The 6th article in the Soaring Through the Clouds series continues the exploration of an extensive demo created by a group of Oracle ACE directors that allows users to nominate and vote for music artists they want to perform at a special concert. Part 6 looks at the role of Node.js and the ACC instance it is deployed on in the demo.
By Oracle ACE Director Lucas Jellema
Table of Contents
- Lightweight Application Container Cloud Service for Rich API implementation with Node.js
Introduction
The Application Container Cloud (ACC) is a cloud service with which light weight containers can be provisioned for running complete applications created in technologies such as Node.js and Java SE. Soon, ACC will also offer containers for PHP, Ruby and Python, followed by more technologies. ACC leverages Docker container technology under the covers.
In our multi-cloud demo scenario, ACC played two roles: one as the deployment platform for the Oracle JET rich web application (to be discussed in Part Seven of this series) and the other as the host of a container with a Node.js application that exposes a REST API through which detailed artist records can be retrieved. The implementation of this API leverages several 3rd party services, such as the Spotify and Echo Nest APIs.
In this article, we will discuss this Node.js application and the ACC instance it is deployed on. The orchestration and enrichment service running on SOA CS calls out to the API published by this application to create a very rich artist record that is subsequently saved to the database in the cloud.
Lean API Juggling in Node.js
The challenge for which we used Node.js and ACC consisted of gathering rich details on the proposed artist, such as a biography, an image and a discography of released albums and pictures of their covers. This information is to be recorded from SOA CS through the database adapter in the DBaaS instance. Various APIs are available on the internet for retrieving data on artists; two examples are the Spotify API (https://developer.spotify.com/web-api/ ) and the Echo Nest API (http://developer.echonest.com/docs/v4 ). These APIs expose resources that can be accessed using REST-style interactions and are represented by JSON messages. Various API calls have to be made to collect all the bits of information required. Some of the API calls have limitations, returning a maximum number of records or accepting a restricted list of input parameters. For example, the API operation we invoke to gather the release date and cover image of albums handles no more than 20 albums per call.
Implementing the Artist API in SOA Suite 12c could probably be done as well, but not as easily as in Node.js. The types and the number of REST calls (some sequentially because of dependencies and others in parallel) and the shape of the data to be handled make a Node.js application a very good fit for our challenge: the asynchronous nature of the Node.js platform allows for easy coordination of parallel synchronous calls; the support for making HTTP(S) calls with various headers is easy in Node.js; and manipulating JSON data structures is very straightforward in JavaScript. The large number of relevant code samples and the many community borne Node packages are an extra bonus.
The figure below visualizes the Artist API Node.js application: it exposes a very simple REST API and executes over a dozen REST calls to external APIs. The SOA Composite running on SOA CS makes one simple REST call to the Artist API to collect the complete enriched artist record and merge it with the artist proposal, and persists it to DBaaS before calling out to ICS to tweet about it.
The code for the Node.js Artist API application is available on GitHub: https://github.com/lucasjellema/aced-cloud-demo-ofmw2016-valencia/tree/master/ACC/artist-enricher-api . Like many Node.js applications, we use the express framework to handle incoming HTTP requests in a convenient way. This makes it easy to interpret REST calls—resource URL paths and parameters, headers–and respond to them. The core module http is used for the outbound API calls.
Of special interest in the application is the use of the async module to organize the sequential and parallel activities that are themselves asynchronous in nature. Check out the overarching structure of the main flow through the code, illustrated below.
Two major parallel branches can be discerned, the first dealing with the Spotify API, the second one for the Echo Nest API. The Spotify branch in the code uses the async.waterfall to organize a series of three sequential units. The third unit uses the aync.forEachOf to perform parallel calls for each of the elements in the albumArrays collection—which can contain up to four elements, resulting in four parallel calls from within that forEachOf section. The Echo Nest branch in async.parallel does two subsequent calls, organized in the traditional way: the second request is made in the callback function for the first one.It works as well, but is not as elegant as the use of the asynch module.
- async.parallel([
// first function: call out to Spotify
function(callback){
// execute sequential asynchronous calls to functions as waterfall
async.waterfall([
function(callback) {
// invoke Spotify Search API to find the Artist and the spotify identifier;
request( artistUrl, function (error, response, body) {
… // handle Spotify response
callback(null, artist.spotifyId); // notify waterfall this unit is done
});// request
}, // go collect list of albums by this artist
function(artistSpotifyId, callback) {
/\* get discography - most recent 50 albums (max number in one call) \*/
request(albumsURL, function (error, response, body) {
… // handle list of albums returned in response
callback(null, artist.albums); // inform waterfall this unit is done
}); //request
}, // go get details on the albums in the list artist.albums
function(albums, callback) {
… prepare albumArrays – collections of album ids
// parallel execution of each of the album arrays
// note: we can invoke the API with a max number of album id values
**async.forEachOf(**albumArrays, function (value, key, callback) {
…
// invoke REST API to retrieve details for a set of albums
request(albumsDetailsURL, function (error, response, body) {
… //handle response
callback(); // inform forEachOf that this unit is done
}); //request
}, function (err) { // completion of the forEachOf
// forEachOf is done - all branches have done their callback()
callback(err, albums); // notify the waterfall that we are done here
}
);// forEachOf
\],function (err, result) {// completion of waterfall
callback(err,'waterfall'); // inform async.parallel that is branch is complete
}
);//waterfall
},
// second function in parallel: call out to EchoNest
function(callback){
// get artist biography from EchoNest in two steps
// first get echonestArtistId, then using that id, get the biography
request(searchURL, function (error, response, body) {
// process response from EchoNest, to get the EchoNestArtistId
…
request(bioURL, function (error, response, body) {
… // process response
callback(null, 'Echonest'); // notify ASYNC that this unit is complete
});// request for bio
});// request echonest search plus callback
}// 2nd function in parallel
], function(err, results){ // parallel is complete
// take the artist record as it stands right now and compose the respons
composeArtisResponse(res, artist);
}); //parallel
A visualization of the program structure would be like this, with every request an asynchronous operation:
Deploying and Running the Node.js Application on ACC
The Node.js application consists of a single JavaScript file supported by a number of packages installed using NPM in the folder node_modules.
The file and the modules along with a descriptive file called manifest.json are packaged in an archive for deployment to ACC.
The manifest file contains a description of the application, including the command to be used for running the application:
{
"runtime":{
"majorVersion":"0.12"
},
"command": "node artist-enricher-api.js",
"release": {},
"notes": "A REST API (GET requests) to retrieve rich information about a musical artist: e.g. http://host:port/artists/get?artist=the+beatles"
}
Deployment of the application is easily done through the ACC Cloud Service Console. Click on Create Application. In the popup, we can choose between Java SE and NodeJS (and, soon, other technologies). Click on NodeJS.
The next dialog allows me to specify the details for the application and for the ACC subscription’s billing regime as well as the instance sizing parameters. After uploading the application archive, I can click create and have the Node.js container provisioned and my application installed into it.
When the provisioning and deployment are complete, I can inspect the application in the Service Console. In this console, we can define environment variables and service bindings (see next section on integration with DBaaS).
When the deployment is complete, the API can be accessed at the URL indicated in the Service Console. You can make sample calls such as https://artist-enricher-api-lucasjellema.apaas.em2.oraclecloud.com/artists/get?artist=the+beatles or https://artist-enricher-api-lucasjellema.apaas.em2.oraclecloud.com/artists/get?artist=rihanna , and get exciting responses such as the one shown below.
More details on how to get started with the Application Container Cloud service for Node.js containerized applications are in this article: https://technology.amis.nl/2016/02/03/deploying-a-node-js-application-to-the-oracle-application-container-cloud-service/ .
Built-in DBaaS integration support
One valuable feature of the Node.js application container provided by ACC is the pre-installed Node.js Oracle Database driver. This driver is an open-source Node package provided by Oracle Corporation to facilitate interaction from Node.js applications with local or remote Oracle Database instances. Through this driver, a Node.js application can perform queries, DML operations, DDL statements and PL/SQL calls in a way that is very similar to how Java applications use JDBC to connect to a relational database.
Applications deployed to an ACC Node.js container automatically have this database driver at their disposal. The container can be configured to leverage a DBaaS instance, with which a Cloud Service Binding is created, as shown in the next figure. Running a Node.js application that works with a cloud-based database instance becomes a breeze. The figure shows such a Node.js application that exposes a REST API and is powered by a database instance providing the persistent back end. This article (https://technology.amis.nl/2016/02/06/linking-application-container-cloud-to-dbaas-expose-rest-api-from-node-js-application-leveraging-node-oracle-database-driver/) discusses the detailed implementation and configuration steps.
Conclusion
Node.js is a server-side JavaScript platform that makes it very easy to implement REST/JSON APIs. Handling incoming HTTP requests and making multiple parallel calls; processing and composing JSON data structures; and dealing with complexities around binary data, custom HTTP headers, certificates and more are pieces of cake.
Running a Node.js application on the Oracle Public Cloud infrastructure is well supported by the Application Container Cloud (ACC). Any Node.js application can simply be packaged as a zip file and quickly deployed to a container in ACC. In the integrated multi-cloud scenario, the Artist API Node.js application leverages several 3rd party APIs to provide a simple interface for rich artist data to the orchestration running on SOA CS.
About the Author
Oracle ACE Director Lucas Jellema is solution architect and CTO at AMIS, based in the Netherlands. An Oracle, Java, and SOA specialist, he works as a consultant, architect, and instructor in diverse areas such as Database & SQL, Java/Java EE, SOA/BPM and PaaS and SaaS Cloud Solutions. The running theme through most of his activities is the transfer of knowledge and enthusiasm (and live demos). Lucas is a well-known speaker at Oracle OpenWorld, JavaOne and various Oracle User Group conferences around the world. His articles have appeared in OTN, OTech and the AMIS Technology Weblog, and he is the author of Oracle SOA Suite 11g Handbook (2014, McGraw Hill) and the Oracle SOA Suite 12c Handbook (2015, McGraw Hill).
This article represents the expertise, findings, and opinion of the author. It has been published by Oracle in this space as part of a larger effort to encourage the exchange of such information within this Community, and to promote evaluation and commentary by peers. This article has not been reviewed by the relevant Oracle product team for compliance with Oracle's standards and practices, and its publication should not be interpreted as an endorsement by Oracle of the statements expressed therein.