Skip to content
StartJS
Web Development Blog
  • Blog
  • Contact
  • FCC Bonfires
  • Privacy Policy
  • GorkaJS on Twitter
  • StringeLeeroy on GitHub
  • Search
FCC Zipline Series, Free Code Camp
by strongeleeroy4th Nov 20163:52 pm31st Jul 2018

Create a File Metadata Microservice with Node.js

File Metadata Microservice

Inspecting file metadata is something we could easily do with just good ol’ browser JavaScript. But also creates a chance to explore the world of fullstack development as a nice and easy to implement application.

We are going to be coding a fullstack application (frontend + backend) that will let the user upload a file and let us visualize relevant file metadata such as file type or size.

We’ll be serving both the front-end assets and API on the same Express based Node.js server, with some ECMAScript 2015 flavour added to it with Babel.

The client application code is up to you. I’m going to go bare-minimum here, but feel free to use a front-end framework if you want to go all out.

Finally, we’ll deploy the application to Heroku. If you want a quick guide to deploying to Heroku, take a look at this post.


Setting up the environment

As always, start off with a quick npm init and follow up by sorting out the dependencies:

npm install --save express multer babel-require babel-core babel-preset-es2015

The new guy in the block here is multer. Multer is a Node.js middleware that handles multipart form data such as file uploads. By the time we’re done with the file metadata service, you’ll be in a a better position to reuse this middleware in the future.

Since we are going to be serving both the client application and API on the same server, separating them properly sounds like a good idea; let’s do that by using a project structure such as the following:

/file-metadata-microservice
    |
    | -- /node_modules
    | -- .babelrc // Babel config
    | -- .gitignore
    | -- package.json
    | -- Procfile // Heroku deployment instructions
    | -- index.js // Application entry file
    | -- /server
    |       |
    |       | -- app.js // API implementation
    |
    | -- /client
            | -- index.html // Main HTML file
            | -- styles.css // Optional stylesheet
            | -- app.js // Client-side application logic

Before we get to far, make sure that you’ve set up Babel:

{
  "presets": ["es2015"]
}

Optionally, tell Heroku what to do with your app once it is pushed:

web: node index.js

Entry file

Our entry file, index.js, is the very same used by the previous Node.js app articles. You can go ahead and have a look at them or use the one below:

"use strict";

require('babel-register');

const app = require('./server/app').app,
      PORT = process.env.PORT || 8000;

app.listen(PORT, function() {
	console.log('File metadata microservice listening on port', PORT);
});

 

We’re finally set up for success, we’ll start with the backend. Here we go:


File Metadata API

We’re going to start off by creating an express application and adding a POST endpoint:

import express from 'express';

export const app = express();

app.post('/upload', (req, res) => {
  // Endpoint logic will go here
});

This endpoint is pretty lacklustre at the moment, we’re going to add some Multer to the mix. To set multer up, we need to create some sort of storage. Multer accepts both disk and memory storage, and as you have probably guessed, disk storage can be used as a means of permanent file storage.

But we don’t need that this time around, we just need to get a file, analize it and forget about it. Once the storage is created, we can instantiate  a multer middleware instance and have every request made to our endpoint pass through it:

import express from 'express';
import multer from 'multer'; // Import multer

export const app = express();

var storage = multer.memoryStorage(); // Create memory storage
var upload = multer({ storage: storage }); // Create middleware with the storage above

app.post('/upload', upload.single('data'), (req, res) => {
  // File is now accessible @ req.file
});</pre>
See that <strong>upload.single('data')</strong> bit in the endpoint? That means we are passing each and every request made to our endpoint through the multer middleware.

The <strong>single</strong> option means that it will only accept <strong>one file</strong> with the given <strong>fieldname</strong> (<em>'data'</em>). If you want to know more about <strong>multer</strong>, take a look at <a href="https://github.com/expressjs/multer" target="_blank" rel="noopener">the GitHub repository</a>.

Now, we can finally access the file within the endpoint through the <strong>req.file </strong>object and return the to the user:
import express from 'express';
import multer from 'multer';

export const app = express();

var storage = multer.memoryStorage();
var upload = multer({ storage: storage });

app.post('/upload', upload.single('data'), (req, res) =&gt; {
  if (req.file) {
    res.status(200).json({
      filename: req.file.originalname,
      size: req.file.size,
      type: req.file.mimetype
    });
  } else {
    res.status(500).json({ error: `No file was provided in the 'data' field` });
  }
});

This is everything that we need to do for the API code. But we still need to add an additional endpoint that we'll use as a means to serve the client application that we'll add to the client directory.

We're going to use a built-in Express middleware for this purpose. The static middleware will serve static files from the specified directory as requested, it will basically create a static webserver at the given path:

import express from 'express';
import multer from 'multer';

export const app = express();

var storage = multer.memoryStorage();
var upload = multer({ storage: storage });

// We serve static assets when the root directory is requested
// keep in mind that the asset path (client) is relative to the entry file
app.use('/', express.static('client'));

app.post('/upload', upload.single('data'), (req, res) => {
  if (req.file) {
    res.status(200).json({
      filename: req.file.originalname,
      size: req.file.size,
      type: req.file.mimetype
    });
  } else {
    res.status(500).json({ error: `No file was provided in the 'data' field` });
  }
});

We are now done with the backend. Let's move on to...


The Client Application

To enable a user to upload a file and then visualize the file metadata, we're going to create a simple client-side application. I'm going to use plain JavaScript, but feel free to use anything here, be it jQuery, React, Angular or X.js. (X stands for anything, but I bet Googling for X.js would end up finding you an actual library these days...)

First of all, we'll be adding an index.html file to the client directory:



    
    File Metadata Microservice


    <h4>Upload a file using the form below</h4>
    <hr />

    <br />
    Upload

http://app.js

Next, we'll handle the upload event in client/app.js as follows:

(function() {
  var submit = document.getElementById('submit-upload'),
      fileInput = document.getElementById('file-input'),
      resultDisplay = document.getElementById('result');

  submit.addEventListener('click', function() {
    if (fileInput.files.length > 0) {  // Make sure a file was chosen
      uploadFile(fileInput.files[0]); // Grab the first file as it is the only one
    }
  });

  function uploadFile(file) {
    var http = new XMLHttpRequest(),
        formData = new FormData(), // Note: This won't work on older browsers
        url = 'upload'; // Our endpoint address

    formData.append('data', file); // Remember we defined 'data' as the fieldname in multer
    http.open('POST', url, true);
    http.send(formData);
    http.onload = function() {
      // Response is available here as this.responseText
      resultDisplay.innerHTML = this.responseText;
    };
  }
}());

At this point, it may be worth to style and improve upon the client application UI. The current front-end is ugly, so very ugly, but it works.

Go ahead and run the app with node index.js or deploy it to Heroku. Navigate to the defined port and you should be able to select a local file, and then click on Upload to view the resulting JSON in the result <div>.

Share this:

  • Twitter
  • Facebook

Like this:

Like Loading...

Related

Tagged with: backend ECMAScript 2015 ES6 express file metadata file upload FreeCodeCamp frontend fullstack http request JavaScript multer Node.js Solution

1Comment

Add yours
  1. 1
    Chato on 20th Nov 2016 at 8:11 am
    Reply

    Conoces alguna forma de enviar el FormData mediante un $.post o $.ajax de jQuery? Tengo ya tiempo intentándolo pero siempre me aparecen errores al serializar automáticamente el objeto formData desde el form.

    $.ajax({
    url: “/upload”,
    type: “POST”,
    data: new formData(‘formulario’), <<< ESTO ES LO QUE BUSCO SERIALIZAR EN AUTOMATICO
    //processData: false, // tell jQuery not to process the data
    //contentType: false // tell jQuery not to set contentType
    success: function (response) {
    console.log(response);
    }
    });

    En php siempre me ha funcionado pero acá con multer no me va, tb he probado formidable y tampoco. Terminé utilizando el objeto xmlhttprequest pero vuelvo a lo mismo que es hacer append a los valores :|. Te ha ocurrido ese error? Por cierto tu material es excelente!

    LikeLike

Leave a Reply Cancel reply

Fill in your details below or click an icon to log in:

Gravatar
WordPress.com Logo

You are commenting using your WordPress.com account. ( Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. ( Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. ( Log Out /  Change )

Cancel

Connecting to %s

Enter your email to get email notifications every once in a while. Only relevant content updates will go out, I hate spam as much as you do.

Follow Me

  • Twitter
  • LinkedIn

Categories

  • AngularJS (1)
  • CSS & Sass (1)
  • ECMAScript 6 (2)
  • FCC Algorithm Challenges (51)
  • FCC Zipline Series (10)
  • Free Code Camp (3)
  • JavaScript Core (7)
  • Maintenance (2)
  • MongoDB (3)
  • Node.js (4)
  • React (3)
  • Software (2)

Tags

Algorithm Array Array Methods bonfire bonfire solutions boolean ES6 FreeCodeCamp function Functional Programming JavaScript loop Node.js object properties objects operator Solution Solutions String string methods
Website Powered by WordPress.com.

Begin typing your search above and press return to search. Press Esc to cancel.

  • Follow Following
    • StartJS
    • Already have a WordPress.com account? Log in now.
    • StartJS
    • Customise
    • Follow Following
    • Sign up
    • Log in
    • Copy shortlink
    • Report this content
    • View post in Reader
    • Manage subscriptions
    • Collapse this bar
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
%d bloggers like this: