In Part 1 of the tutorial, we described how to build a Node.js application and deploy it to AppFog on our CenturyLink Cloud Platform. The application uses Passport to register and authenticate users against a CenturyLink Orchestrate database.

In this installment, we'll add a document storage system to the application. We'll use CenturyLink Object Storage to store documents in the cloud. We will store document metadata separately in Orchestrate.

CenturyLink Object Storage is a robust, geo-distributed cloud storage system capable of storing any type of digital content. It is compatible with the Amazon S3 API, which means we can use a number of available libraries to access cloud storage from our application. While it is a powerful solution for data backups and media distribution, for this application we'll use it as digital storage for a web application front-end.

Creating an Object Storage Bucket

Before we can use Object Storage, we need to add an Object Storage user and bucket.

  1. Go to the CenturyLink Cloud Control Panel and log in.

  2. Hover over the green bar to access the drop-down menu.

  3. Select Object Storage.

    Object Storage Menu

  4. Click the Users tab.

  5. Click Create User. Fill out the form, but keep in mind that the email address you use should be unique across the Object Storage platform and cannot be changed or reused with a different CenturyLink Cloud account.

    Create User Form

  6. Once the user is created, you are taken to the Users tab. Click on the account you just created.

  7. The next screen shows you the access key id and secret access key. Make note of these and save them for later! You will need them to configure the web application to access object storage.

    Secret Keys

  8. Click on the Buckets tab to create a new Object Storage bucket.

    Object Storage Bucket

  9. Select the user you created in Step 3 from the Owner drop-down menu.

  10. Select "Canada" in the Region drop-down menu.

  11. Record the bucket name that you used in Step 8 as you will need it to configure your application later.

Add S3 Libraries to The Web Application Next, add a library to the Node.js application so it can access your new bucket. Since we need our library to easily use alternate endpoints, we will use the Knox Amazon S3 library.

Note: Since Object Storage is Amazon S3 compatible, many of the available S3 libraries and tools can be used to access it. For an example, read our S3CMD - Object Storage Management article in the Knowledge Base.

Add Knox to Package Dependencies If you want to follow along with our already-completed example, download the code for this project. It is located in the Git repository. The code for this part of the tutorial is in the “part1” tag.

Clone the Git repository with the following commands:

    $ git clone https://github.com/pymander/clc-nodejs-tutorial
    $ cd clc-nodejs-tutorial
    $ git checkout tags/part2
  1. In your shell, change to your project directory.
  2. Run the following command:
    $ npm install knox --save
    
    This adds the Knox library to the list of dependencies in project.json and installs a local copy of the library for our application to use.

Add Object Storage Configuration The next step is to add Object Storage configuration to your project's private configuration.

  1. Open config.js in your favorite text editor.
  2. Add configuration options for the access key, secret access key, and bucket. Your configuration file now looks like this:
     // config.js
     // This file contains private configuration details.
     // Do not add it to your Git repository.
        module.exports = {
        "orchestrate_token": "<your-orchestrate-token>",
        "object_storage_key" : "<your access key ID>",
        "object_storage_secret" : "<your secret access key>",
        "object_storage_bucket" : "<your Object Storage bucket name>",
        "object_storage_endpoint" : "canada.os.ctl.io"
        }
    
  3. Replace <your access key ID>, <your secret access key>, and <your Object Storage bucket name> with the values you saved from "Creating an Object Storage Bucket" in Steps 7 through 11.

If you forget your access key id, secret access key, or bucket name, you can find them in the Object Storage Control Panel. The access key id and secret access key are on the Users tab, while the bucket name can be found on the Buckets tab. See the "Creating an Object Storage Bucket" (earlier in this tutorial) for instructions.

Add Multipart Form Handler Library Newer versions of Express don't have built-in handlers for multipart forms. File uploads typically use multipart HTML forms, so we need to add the Connect-busboy library to our project.

Run the following command to add the required libraries:

    $ npm install connect-busboy --save

Add File Uploading The next sections add document uploading to your application.

Create a View for Adding Documents Now we'll create a new view containing a form for uploading and describing documents.

  1. Create a file called views/add.jade.
  2. Edit the file so it contains the following code:

    extends layout
    
    block content
      h1= title
    
      form(method='post', action='/documents/add', enctype="multipart/form-data")
        label Document Title:
          input(type='text', name='title', size='20')
        br
        label Keywords:
          input(type='text', name='keywords', size='40')
        br
        input(type='file', name='document')
        br
        input(type='submit', value='Upload')
    

    Create a Route for Adding Documents Now we need to add a new route to our project to handle document uploads. This route handles both GET and POST requests. A GET request displays the add view, but a POST request processes a multipart form submission and does our file uploads.

  3. Create a file called routes/documents.js.

  4. Edit the file so it contains the following code:

    var express = require('express');
    var router = express.Router();
    var fs = require('fs');
    
    // Load private configuration
    var config = require('../config.js');
    
    /* Object Storage configuration */
    var knox = require('knox'),
        client = knox.createClient({
          key      : config.object_storage_key,
          secret   : config.object_storage_secret,
          bucket   : config.object_storage_bucket,
          endpoint : config.object_storage_endpoint
        });
    
    // Orchestrate connection
    var oio = require('orchestrate');
    var db = oio(config.orchestrate_token, process.env.npm_package_config_datacenter);
    
    /* GET Document listing */
    router.get('/', function(req, res, next) {
      if (!req.user) { res.redirect('/'); }
    
      res.send('Document list');
    });
    
    /* GET Document add form */
    router.get('/add', function(req, res, next) {
      if (!req.user) { res.redirect('/'); }
    
      var tpl = { title : 'Add A Document',
                  user : req.user,
                  session : req.session };
    
      res.render('add', tpl);
    });
    
    /* POST Document upload handler */
    router.post('/add', function(req, res, next) {
      if (!req.user) { return res.redirect('/'); }
    
      // Clear status messages.
      delete req.session.success;
      delete req.session.error;
      delete req.session.notice;
    
      var tpl = { title : 'Add A Document',
                  user : req.user,
                  session : req.session };
      var docinfo = {
        creator : "CenturyLink Cloud example application"
      };
    
      /* There are two steps:
         1. Upload document to Object Storage
         2. Add metadata to Orchestrate */
      if (req.busboy) {
        // Handle all POST fields
        req.busboy.on('field', function(fieldname, val, fieldnameTruncated, valTruncated) {
          console.log('Field [' + fieldname + ']: value: ' + val);
          docinfo[fieldname] = val;
        });
    
        // Handle FILE fields.
        req.busboy.on('file', function (fieldname, file, filename, encoding, mimetype) {
          var buffer = new Buffer(0);
          var headers = {
            'x-amz-acl' : 'public-read',
            'Content-Type' : 'application/pdf' // Force everything to be a PDF.
          };
    
          if ('' === filename) {
            console.log("DEBUG: No filename specified");
            return;
          }
    
          // The knox library doesn't handle spaces in filenames well.
          filename = filename.replace(" ", "_");
    
          // Save document info for later.
          docinfo.filename = filename;
    
          // This is triggered while there is still file data left to be read.
          file.on('data', function (data) {
            buffer = Buffer.concat([buffer, data]);
          });
    
          // This is triggered when the file finishes being read.
          file.on('end', function () {
            headers["Content-Length"] = buffer.length;
    
            // Send our file to Object Storage
            var store = client.put(docinfo.filename, headers);
    
            store.on('response', function(res){
              // Are we successful?
              if (200 == res.statusCode) {
                console.log('STORED: %s', store.url);
                req.session.success = "Your last upload was " + filename;
    
                // Flesh out our metadata.
                docinfo.url  = store.url;
                docinfo.user = req.user.username;
    
                // Now we store metadata in Orchestrate.
                db.put('documents', docinfo.filename, docinfo)
                  .then(function (result) {
                    console.log("SAVED METADATA");
                  })
                  .fail(function (err) {
                    console.log("METADATA FAILURE: " + err.body.message);
                  });
              }
            });
            store.end(buffer);
    
          });
        });
    
        req.pipe(req.busboy);
      }
    
      res.render('add', tpl);
    });
    
    module.exports = router;
    

    Load Routes Into Application Next, the application needs to load the new routes.

  5. Open app.js in your project directory.

  6. Locate the line that says var routes = require('./routes/index');.
  7. Below that line, insert the following code:
    var documents = require('./routes/documents');
    
    Add a Document List Now that you can upload documents, you want to be able to list them.

Create a View for Listing Documents

  1. Create a file called views/list.jade.
  2. Edit the file so it contains the following code:

    extends layout
    
    block content
      h1= title
    
      ul
        each doc in items
          li
            a(href="#{doc.value.url}") #{doc.value.title}
            |  uploaded by #{doc.value.user}
    

    Create a Route for Listing Documents

  3. Open routes/documents.js.
  4. Locate the line that says module.exports = router;.
  5. Above that line, insert the following code:

    /* LIST documents using Orchestrate and Object Storage */
    router.get('/list', function(req, res, next) {
      if (!req.user) { res.redirect('/'); }
    
      var tpl = { title : 'List Documents',
                  user : req.user,
                  session : req.session };
    
      // Fetch our list of documents.
      db.list('documents', { limit: 20 })
        .then(function (result) {
          tpl.items = result.body.results;
    
          res.render('list', tpl);
        })
        .fail(function (err) {
          throw new Error(err);
        });
    });
    

    Link to the New Routes You need to change views/index.jade so it contains a link to the new route.

  6. Open views/index.jade in your text editor.

  7. Add a link to /documents/add to the template. The template look like this:

    extends layout
    
    block content
      h1= title
    
      p
        | This example shows you how to authenticate against an
        a(href='orchestrate.io') Orchestrate
        | database.
      if user
        p
          strong Welcome back, #{user.username}!
        ul
          li
            a(href='/documents/add') Add a document
            |  to the archive.
          li
            a(href='/documents/list') List documents
            |  in the archive.
          li
            a(href='/logout') Logout
      else
        p You have never been here before.
          a(href='/signin') Please sign in
    

    Add a Convenient Menu To give users a convenient navigation method, we'll add a menu to the top-level layout of our views.

  8. Open views/layout.jade in your text editor.

  9. Edit the file so it looks like the following:

    doctype html
    html
      head
        title= title
        link(rel='stylesheet', href='/stylesheets/style.css')
      body
        div
          a(href='/') Home
          |  |
          a(href='/documents/add') Add Document
          |  |
          a(href='/documents/list') List Documents
    
        block content
    

    Run and Test File Uploading You should now be able to login to your application to upload and list files.

  10. Go to http://localhost:3000/ in your web browser.

  11. Click Please sign in.
  12. Enter your username and password.
  13. Click Login.
  14. After signing in, click Add a document.

    Localhost

Your application now includes the Add A Document screen.

Deploying the Application Once you've confirmed that the application is working correctly, it's time to deploy it to the cloud.

Run the following command in your project directory to deploy to the cloud. Replace <your-project-name> with the name of your project.

        $ cf push <your-project-name>

Your updated application is now deployed.

The Next Step: Document Search

Our application is on the way to becoming a useful document storage solution. It now uses CenturyLink Object Storage to store documents in the cloud, while keeping document metadata in a CenturyLink Orchestrate database. In our next tutorial, we will use Orchestrate to search document metadata and provide a fast, efficient solution for locating the documents you’ve stored.

Links to the Complete Tutorial Series

Part 1 - Store and Authenticate User Credentials Part 2 - Build a Document Storage System Part 3 - Include Powerful Search Capabilities Part 4 - Add a Customized PDF Viewer and Comment System Part 5 - The Next Step for Your Web Application