Today, we are going to be building an image search abstraction layer that is built on top of the Bing search API. For this purpose, we’ll make use of Node.js + Express and Mongoose as an object manager for MongoDB. Additionally, we’ll harness the power of ECMAScript 6 using Babel.
If you are not familiar with Mongoose, take a look at this recent post. If you want to know more about ECMAScript 6 usage in Node.js with Babel, give this other article a read first.
Our will have two endpoints:
- /api/search/:query: Will search for the given string using Bing search and return an array of results.
- /api/latest: Will return a list of the last 10 searches performed.
Set up your environment
It’s going to be quite simple, but first things first, let’s set up the project environment. Here’s what my project will look like:
/image-search-abstraction-layer
|
| -- .babelrc // Babel config file
| -- .gitignore // Usual .gitignore file
| -- index.js // Entry file to the app
| -- package.json // Package definition
| -- /node_modules
| -- /src
|
| -- app.js // Main application logic
| -- /api
| |
| | -- search.js // Endpoint logic for search
|
| -- /models
|
| -- searchHistory.js // Mongoose model logic
Dependencies
Before I go full throttle and get into the code, let’s sort out the dependencies we’ll require for this project:
npm install --save babel-core babel-register babel-preset-es2015 express mongoose bing.search
Here’s a quick recap:
- babel-core: Core Babel library.
- babel-register: Allows on-the-fly Babel transpilation using the require hook. More info here.
- babel-preset-es2015: ECMAScript 2015 (6) syntax preset for Babel.
- express: Express framework library.
- mongoose: MongoDB object mapper library. More info here.
- bing.search: An interface to the Microsoft Bing search engine.
All set? Let’s get going.
The entry file, index.js, is almost identical to every other backend project in the series. It basically acts like an entryway for the application so we can use Babel within, and initialize the application:
require('babel-register');
const app = require('./src/app').app,
PORT = process.env.PORT || 8000;
app.listen(PORT, function() {
console.log('Image search abstraction layer listening on port', PORT);
});
Here’s what .babelrc looks like, it’s exactly the same as the one we’ve used for previous projects:
{
"presets": ["es2015"]
}
The src/app.js file will contain the main logic for the application, we’ll initialize the Mongoose connection and define the Express application:
import express from 'express';
import mongoose from 'mongoose';
const mongodb = process.env.MONGODB_URI || 'mongodb://localhost:27017';
mongooseInit(mongodb); // I've moved Mongoose initialization to a function below
export const app = express(); // Create express app
/* We'll get to the app routes in a second */
/* Mongoose initialization function */
function mongooseInit(mongodb) {
mongoose.Promise = global.Promise;
mongoose.connect(mongodb);
}
Instead of defining the routes in src/app.js, we’re going to create an external file where the API routes will live. In this case, I’ve placed this file in src/api/search.js, and created my routes using the Express Router object as follows:
import express from 'express';
export const searchApi = express.Router();
searchApi.get('/search/:query', (req, res) => {
// Image search logic
});
searchApi.get('/latest', (req, res) => {
// Last searches logic
});
Now, we import this file into src/app.js and make use (this is a very bad joke by the way) of it:
import express from 'express';
import mongoose from 'mongoose';
import { searchApi } from './api/search'; // We import the routes object
const mongodb = process.env.MONGODB_URI || 'mongodb://localhost:27017';
mongooseInit(mongodb); // I've moved Mongoose initialization to a function below
export const app = express(); // Create express app
app.use('/api', searchApi); // We **USE** the routes object for the '/api' route
/* Mongoose initialization function */
function mongooseInit(mongodb) {
mongoose.Promise = global.Promise;
mongoose.connect(mongodb);
}
What we get as a result is basically a root route in the form of /api that will then make use of the search API router:
- /api/search/:query
- /api/latest
Pretty cool. Let’s get to image searching.
The SearchHistory Model
Before we search via the Bing search module, we’re going to set up the mechanism that will allow us to view past search queries that we’ll store in MongoDB. For this purpose, we are going to define a new Mongoose schema and model that I’ll call SearchHistory and will live under src/models/searchHistory.js. This is what it looks like:
import mongoose from 'mongoose';
var searchHistorySchema = mongoose.Schema({
timestamp: Number,
query: String
});
searchHistorySchema.index({ timestamp: 1 });
export var SearchHistory = mongoose.model('SearchHistory', searchHistorySchema);
It’s a pretty simple schema, it will basically store the query string and the unix timestamp for the time the query was performed.
At this point in time, we can already define the /latest endpoint and start populating in the the /search/:query endpoint.
Populating the Search History
Let’s import the SearchHistory Mongoose model we created and start making use of it. Each time a user performs a search, we need to do two things:
- Perform the search using the Bing search module.
- Store the search query in out Mongo database.
We’re going to start with number 2, and will leave the actual image search for last. Here’s what the updated src/api/search.js file looks like:
import express from 'express';
import { SearchHistory } from '../models/searchHistory';
export const searchApi = express.Router();
searchApi.get('/search/:query', (req, res) => {
let query = req.params.query, // Get the query from the path parameters object
timestamp = Date.now(); // Get the unix timestamp
// BING SEARCH MAGIC WILL HAPPEN HERE //
// We save a new search history entry asynchronously
let queryHistory = new SearchHistory({ query, timestamp }); // Notice ES6 here!
queryHistory.save();
});
searchApi.get('/latest', (req, res) => {
// Last searches logic
});
Bear in mind that the user does not need to know anything about the query history being generated or saved. The only operation that we’ll need to wait for before sending a response to the end user is the actual Bing search, and we are not quite there yet.
Getting the Latest Search Queries
We have already added the logic to populate the search query history entries, but have no way of fetching them. Let’s fix that real quick.
We’re going to update the /latest endpoint so it returns the last 10 queries performed. We’ll need to sort them by timestamp so we get the most recent ones first, and them limit the result to 10 before sending a response:
import express from 'express';
import { SearchHistory } from '../models/searchHistory';
export const searchApi = express.Router();
searchApi.get('/search/:query', (req, res) => {
let query = req.params.query,
timestamp = Date.now();
// BING SEARCH MAGIC WILL HAPPEN HERE //
let queryHistory = new SearchHistory({ query, timestamp }); // Notice ES6 here!
queryHistory.save();
});
searchApi.get('/latest', (req, res) => {
SearchHistory
.find() // We search for every entry
.select({ _id: 0, query: 1, timestamp: 1 }) // We want timestamp and query back, but not the default _id field
.sort({ timestamp: -1 }) // Order by DESCENDING timestamp
.limit(10) // Limit the result to 10 entries
.then(results => { // Finally, return the results
res.status(200).json(results);
});
});
Now, we just need to take care of a small… humm, detail: actually searching for images.
Using the Bing search module
To easily access the Bing search API, we’ve installed a node module to take care of the grunt-work for us: bing.search. We’ll import it into src/api/search.js and initialize a new Search object using your own, personal API key.
If you do not have a Bing API key, you will need to register for one. It’s 100% free if you choose the up to 5000 monthly transactions package by following this link.
Once you have registered and signed up for the free plan, you can view it under My Account, as Primary Account Key.
Now that we have a key at hand, we can finally implement the Search interface:
import express from 'express';
import Search from 'bing.search';
import { SearchHistory } from '../models/searchHistory';
var search = new Search('YOUR_API_KEY_GOES_HERE'); // Create the search object
export const searchApi = express.Router();
searchApi.get('/search/:query', (req, res) => {
let query = req.params.query,
timestamp = Date.now();
// Search for the user query
search.images(query, (error, results) => {
if (error) {
res.status(500).json(error); // We return an error code
} else {
res.status(200).json(results); // We return the results
}
});
let queryHistory = new SearchHistory({ query, timestamp }); // Notice ES6 here!
queryHistory.save();
});
searchApi.get('/latest', (req, res) => {
SearchHistory
.find()
.select({ _id: 0, query: 1, timestamp: 1 })
.sort({ timestamp: -1 })
.limit(10)
.then(results => {
res.status(200).json(results);
});
});
Our app is now fully functional, but we’re going to implement a couple additions that will improve it further:
- The user should be able to paginate through the results by adding a query parameter to the URL such as: ?offset=2
- The results from Bing are very verbose and have lots of parameters, we are going to map them through a function that will only give us a few, relevant fields.
Adding an Offset Query Parameter
We’ll start by accepting a query parameter in our endpoint, and passing it through the Bing search object, this is quite simple to implement:
import express from 'express';
import Search from 'bing.search';
import { SearchHistory } from '../models/searchHistory';
var search = new Search('YOUR_API_KEY_GOES_HERE');
export const searchApi = express.Router();
searchApi.get('/search/:query', (req, res) => {
let query = req.params.query,
offset = req.query.offset || 10, // We grab the "offset" query param or set it to 10 by default
timestamp = Date.now();
// Finally, we use the options object top pass the offset in
search.images(query, { top: offset }, (error, results) => {
if (error) {
res.status(500).json(error);
} else {
res.status(200).json(results);
}
});
let queryHistory = new SearchHistory({ query, timestamp }); // Notice ES6 here!
queryHistory.save();
});
searchApi.get('/latest', (req, res) => {
SearchHistory
.find()
.select({ _id: 0, query: 1, timestamp: 1 })
.sort({ timestamp: -1 })
.limit(10)
.then(results => {
res.status(200).json(results);
});
});
We should now be able to access URLs like this one: /api/search/cats?offset=5
Mapping the Results
Let’s now map the results before sending them back, the change to the endpoint is minimal, but we do need to add a function to pass through:
import express from 'express';
import Search from 'bing.search';
import { SearchHistory } from '../models/searchHistory';
var search = new Search('YOUR_API_KEY_GOES_HERE');
export const searchApi = express.Router();
searchApi.get('/search/:query', (req, res) => {
let query = req.params.query,
offset = req.query.offset || 10,
timestamp = Date.now();
search.images(query, { top: offset }, (error, results) => {
if (error) {
res.status(500).json(error);
} else {
res.status(200).json(results.map(createResults));
}
});
let queryHistory = new SearchHistory({ query, timestamp }); // Notice ES6 here!
queryHistory.save();
});
searchApi.get('/latest', (req, res) => {
SearchHistory
.find()
.select({ _id: 0, query: 1, timestamp: 1 })
.sort({ timestamp: -1 })
.limit(10)
.then(results => {
res.status(200).json(results);
});
});
// We grab the values that interest us and ignore the rest
function createResults(image) {
return {
url: image.url,
title: image.title,
thumbnail: image.thumbnail.url,
source: image.sourceUrl,
type: image.type
}
}
And that’s it! We have a fully working image search abstraction layer in place.
StartJS 