Showing posts with label nodejs. Show all posts
Showing posts with label nodejs. Show all posts

Tuesday, August 30, 2016


Hi, today we are going to discovery about Amazon Alexa skill using Node.js, it is quite simple, but requires many steps, and a few of them are bit cumbersome.
If you’re not familiar, the Echo is an always-on voice interface for your home that allows developers to extend its functionality with “skills”. These skills get the benefit of Amazon’s full speech analysis engine and they work through a simple JSON web API. In this post I’ll will show you how we can use Node.js to write the skill (a small web app) and what else we need to do to get up and running. You don’t need to own an Echo to write a skill… although they sure are fun to play with!
You might first be asking what’s the difference between the “Echo” and “Alexa” - “Echo” refers to the device, but “Alexa” is the name given to the voice interface software itself. This manifests in the keyword (my term, not theirs) that an end user must say in order to trigger the device to take any action. For example, you can ask Alexa facts about our world. While the device is on, simply utter the phrase:
Alexa, what’s the population of the United States?
Notice that we identified what we want (“population”) and input data (the location: “United States”) - these are the keywords Alexa listens for. The device (in the “Alexa” voice) will then respond with:
The population of the United States is about 324 million.
Let’s go…

Custom Skills List

Alexa ships with a number of built in abilities such as replying with fact answers, converting measurements, adding to your shopping list, giving you the weather, and even controlling certain home automation devices like the Philips Hue light bulbs. Of course, these built in abilities aren’t going to cover everything you want to do with a voice interface, which is why they quickly opened up a platform for writing custom “skills” for Alexa which can be published and installed by the end users.
All of these custom skills are invoked with a particular keyword (or multiple words) in addition to the user saying “Alexa” in order to identify the request as being for that skill. For example, you can install the “Bartender” skill from the Alexa app on your phone (or on the web) which allows you to say something like:
Alexa, ask the bartender how to make a whiskey old fashioned.
Notice the word “bartender” in bold above, that’s because it is the “invocation” name of the skill - the thing users must say in order to get Alexa to perform that skill.
I mentioned that the skill is “installed” earlier, but to be clear: none of these skills are actually “installed” on your Echo device. They are simply added to your account. In fact, very little processing is actually done on the Echo device itself, almost all of the work is done on Amazon’s servers (and our own). Here is a simple breakdown of the workflow:
  1. The user speaks a phrase, beginning with “Alexa” which the Echo hears
  2. The audio is sent to Amazon’s servers for processing
  3. An Alexa skill request is sent to your server for business logic processing
  4. Your server responds with a JSON payload including text to speak
  5. Amazon’s server sends your text back to the device for voice output
So how do we create custom skills? That’s what this article is all about! While we’ll be creating our skill using a custom Node.js application (and server), you could also do this with an Amazon Lamda function (written in Node or other languages). This does allow you to bypass the request validation (more on that later), but also means you’re stuck on Amazon versus being portable and able to completely customize your app/server.
Let’s dive in!

Defining the Skill

Before we can use our skill, we must create it in the Amazon Developer Portal. As mentioned previously, our skills are not installed on the Echo, but rather simply added to your Amazon profile. As such, using our custom skills doesn’t require us to be in “development” mode on our Echo or anything. You simply add your skill to your developer account on the portal and you’re done!
The first step is to define your voice interface. This is not a short step, and you should take time in thinking about how end users will actually speak to Alexa. What would they say? How would they say it? Are there regional dialects that you should consider? What about slang or contractions? The point here is do not wait to think about how your users will speak to Alexa, do this step first. It can help you develop your application.
So what do we need to do to define our skill on the portal? Not too much! Let’s look at each piece:

1. Specify an “Invocation Name”

The “invocation name” of your skill is what end users will say, along with “Alexa”, to hit your web application. You should take great care in selecting it! That said, Amazon has many rules for what this can be. You should read the documentation on selecting an invocation name carefully. In general, they should be two words (or one if it’s your brand name) with no articles, Alexa launch words, or connectors (like “the”, “a”, “ask”, “echo”, or “from”).
Once you’ve selected a name, head over to the developer portal and create your skill, plugging in your chosen name… and hope it isn’t taken!

2. Define Expected “Intents” and “Slots”

An “intent” is basically what it says: something the end user “intends” to do. For example, you can define a “Forecast” intent which gets weather for a given date. Intents have a name and any number of “slots” which represent the data required for that intent to work properly - in our case the “date” for the weather forecast.
You’ll need to input this configuration into the developer portal, but I’d also recommend checking it into your source control system along with your code. We’ll have to use those intents and slots in our code anyway. Here’s a simple configuration for our skill:
{
  "intents": [
    {
      "intent": "AMAZON.CancelIntent"
    },
    {
      "intent": "Forecast",
      "slots": [
        { "name": "Day", "type": "AMAZON.DATE" }
      ]
    }
  ]
}

In the configuration above you can see that I’ve included one of Amazon’s built-in intents which means you don’t have to define a voice interface for it. That makes allowing a user to “cancel” their action much easier. There are other built-in intents for things like allowing the user to say “Yes” and “No” (in a variety of ways), ask for “Help”, and more.
Our own intent has a name of “Forecast” and one “slot” defined named “Day” which is of type “AMAZON.DATE” - one of the built-in slot types. (There’s also AMAZON.TIME, AMAZON.NUMBER, and a few other helpful ones.) What’s neat about the AMAZON.DATE slot type is that the end user can say “today”, “tomorrow”, “Friday”, “next Tuesday”, or others and Amazon will - through its speech analysis software - derive the actual date in YYYYY-MM-DD format which it sends to our server.
Custom Slots
You can create custom slot types as well, but your mileage may vary when using these. They are best when your input data has a well-defined, finite value list. For example, a list of acceptable colors. Creating an open-ended speech slot is tricky and will not produce reliably predictable behavior.

3. Provide Sample Utterances

The last step in defining your skill is to provide a list of sample utterances to flesh out the voice interaction model for Amazon’s speech analysis engine. This is required because a voice interface does not allow for the strictness that a visual interface does. On a web site you have input boxes with validation logic that can inform a user in real time what they need to fix. With a voice interface we have to wait until they are finished speaking to do any validations and ask the user for better information.
Take our simple forecasting skill for example, we want our users to ask for the weather on a given day. But how exactly will they speak that request? Here are some options:
  • what’s the weather for tomorrow
  • what is the forecast for next Tuesday
  • how’s the weather gonna be today
Even with just those three examples we can see how different our users might interact with our skill. Because of this, we need to define our voice interface with examples of what the user might say. The “sample utterances” should be specified with one entry per line, beginning with the intent name and including any named slots in {curly braces}:
Forecast  what's the weather for {Day}
Forecast  what is the forecast for {Day}
Forecast  how's the weather gonna be {Day}
.....
...
..

Those dots don’t mean you should include dots… they are a series of ellipses… you need to include a lot more samples than that. It isn’t uncommon for skills to have hundreds of sample utterances per intent, especially if there are multiple slots.

Creating Your Skill

In this section we’ll discuss building our web application for the skill. Technically speaking, this is just a plain old Node.js server. You can host/deploy this anywhere you like, and it can do whatever you want. The only real restrictions are that it must accept POST requests over HTTPS and respond with very specifically organized JSON. If you plan to get your skill certified for publication then you must also verify requests (read about that a bit later).

Setting Up the Project

To ease our development (at least for this demo app), we’ll use Express for our framework. This makes it very easy to handle POST requests and JSON payloads (both in and out). Don’t forget to create a package.json file with npm init and add your dependencies (express and body-parser). After that, let’s create an entry point for our Node application called app.js:
let express = require('express'),
    bodyParser = require('body-parser'),
    app = express();

app.use(bodyParser.json());
app.post('/forecast', function(req, res) {
    // We'll fill this out later!
    res.json({ hello: 'world' });
});
app.listen(3000);

Notice that our basic file here pulls in (requires) Express and the body parser and then creates the Express application and adds the body parser middleware to parse the request as JSON. Our one and only route is for any POST request to the /forecast endpoint which currently responds with a simple JSON string: { "hello": "world" } (Express will do the JSON stringifying for us).

Verifying Requests

Our next step is an annoying one, but necessary if you plan to certify your skill and publish it. In order to pass certification, your application (skill) must verify that each and every request actually came from Amazon and is valid. This process is not short:
  1. Check the SignatureCertChainUrl header for validity.
  2. Retrieve the certificate file from the SignatureCertChainUrl header URL.
  3. Check the certificate file for validity (PEM-encoded X.509).
  4. Extract the public key from certificate file.
  5. Decode the encrypted Signature header (it’s base64 encoded).
  6. Use the public key to decrypt the signature and retrieve a hash.
  7. Compare the hash in the signature to a SHA-1 hash of entire raw request body.
  8. Check the timestamp of request and reject it if older than 150 seconds.
ಠ_ಠ
Or you can use this awesome library: https://github.com/mreinstein/alexa-verifier
So, presuming you want to use Mike Reinstein’s Alexa verification library you’ll need to do four things: npm install --save alexa-verifier is the first. The next step is to get the raw body from the request, and finally add a piece of Express middleware before our /forecast route. In your app.js file you can add this code (replacing our bodyParser.json() line):
app.use(bodyParser.json({
    verify: function getRawBody(req, res, buf) {
        req.rawBody = buf.toString();
    }
}));

This tells the body parser to give you access to the raw request Buffer object after processing, which we can then add as a string property value to our request object. This makes it accessible in the next step: create a function (a piece of middleware) to call Mike’s library:
let alexaVerifier = require('alexa-verifier'); // at the top of our file

function requestVerifier(req, res, next) {
    alexaVerifier(
        req.headers.signaturecertchainurl,
        req.headers.signature,
        req.rawBody,
        function verificationCallback(err) {
            if (err) {
                res.status(401).json({ message: 'Verification Failure', error: err });
            } else {
                next();
            }
        }
    );
}

The function above simply passes our request information to Mike’s library and waits for a result. If there is no error in the verificationCallback we provide then we call next() which tells Express that we can process the route. So how do we use this piece of middleware? Easy! Simply replace our previous route definition with one added argument:
app.post('/forecast', requestVerifier, function(req, res) {
    // We'll fill this out later!
    res.json({ hello: 'world' });
});

Now all requests going to the /forecast POST route will be verified first.

Request JSON Format

You might recall that our Node application will be accepting JSON in the request body and giving back JSON in the response. Express (and the body parser) makes this easy to handle, all we have to do is inspect the object coming in and perform any business logic. The request body will have three pieces, only two of which we really care about: session, request, and version:
{
    "session": { ... },
    "request": { ... },
    "version": "1.0"
}

Note that the “version” above is the version of the Alexa API we’re using. The “session” object will look something like this:
"session": {
    "sessionId": "SessionId.6a4789.....",
    "application": { "applicationId": "amzn1.ask.skill.2ec93....." },
    "attributes": { /* Any persisted session data... */ },
    "user": { "userId": "amzn1.ask.account.AFP3ZWK564FDOQ6....." },
    "new": true
}

The “attributes” in the session will contain any data we specify in the response that should persist from one request to another within a single Alexa interaction. For example, if the user asks for information about Beijing, we might respond by asking what fact that want to know. We would expect the user to then tell us “population” perhaps. We need to hold onto the city they were asking about, and that happens via the session “attributes”. You can read more about the session object on the Alexa developer documentation.
The “request” object within the HTTP request body is probably what we care more about. It contains some meta information about the request and the intent, along with the slot values. What you might find surprising here is that Amazon does not send the full text of the speech. In order to support the “intent” of the end user, Amazon performs an analysis of the speech and derives the named intent from the speech as well as the values for any slots:
"request": {
    "type": "IntentRequest",
    "requestId": "EdwRequestId.ba6dbc3f.....",
    "locale": "en-US",
    "timestamp": "2016-08-14T20:15:27Z",
    "intent": {
        "name": "Weather",
        "slots": {
            "Day": { "name": "Day", "value": "2016-08-18" }
        }
    }
}

Again, note that you will not have access to the raw text the user spoke. This is why the definition of the voice interface on Amazon’s developer portal is so important!
Request Types
You may have noticed that the “request” object has a “type” property. That will have one of three values, each of which your application should handle!! Those are:
  • “LaunchRequest” - sent if a user says something like: “Alexa, start the forecaster”
  • “IntentRequest” - our main logic, we’ll see to handle that soon
  • “SessionEndedRequest” - sent when the user ends a session manually, or there is an error

Response JSON Format

Just as our request body comes in as JSON, our response body goes out as JSON. The format is very straight forward, with only three top-level properties, and not too many variations within those.
{
  "version": "1.0",
  "sessionAttributes": { /* Any data you want to persist... */ },
  "response": { /* What Alexa should do... */ }
}

The first property (“version”) is pretty obvious, but just to reiterate, that’s the version of the Alexa API, not your application/skill version. This should be “1.0” for now. The second property are the “sessionAttributes” which is exactly what you expect if you read the section on the request format: it’s the data you want to persist from one voice input to the next. Our example earlier was if the user had asked for information about “Beijing”: you might respond with “What do you want to know?” and the user could say: “population”. In this case, you would want to hold onto the city while the conversation is ongoing… that’s what we use the session attributes for.
Obviously our main focus is on the “response” property of the JSON we return. You can see below that this object has four possible sub-properties, but only one of those is required: “shouldEndSession”. That property is a boolean which indicates if the conversation is over with the end user. Note that if this is true, any “sessionAttributes” will be cleared out.
"response": {
  "shouldEndSession": true,
    "outputSpeech": {
        "type": "SSML",
        "ssml": "<speak>Tomorrow it could be <break time=\"1s\"/> nasty.</speak>"
    },
    "card": { /* OPTIONAL */ },
    "reprompt": { /* OPTIONAL */ }
}

Speaking Back

As you might expect, our main focus here is to produce the output speech that Alexa will say to the end user in response to their query. For that we use the “outputSpeech” property of the response! This can come in two types: “PlainText” or “SSML”. I’ve chosen Speech Synthesis Markup Language (SSML) above (and I do in my own code) because it allows for better handling of how Alexa should speak my output. As you can see above, I’m able to insert a brief pause in the outgoing audio. I can also specify phonetic pronunciation, short audio clips, and more.
I’m not going to cover cards or reprompts in this post, but they are pretty simple as well. You can read more about them on the documentation linked at the beginning of the response format section.

Handling Requests and Intents

Now that we know what format our data is coming in, and what format we need to send back, all we have to do is detect what type of request we have, what the intent is, and produce our desired JSON! This just comes down to code organization and flow so I won’t go too much into it, but here’s how you might start in your /forecast route handler:
app.post('/forecast', requestVerifier, function(req, res) {
  if (req.body.request.type === 'LaunchRequest') {
    res.json({
      "version": "1.0",
      "response": {
        "shouldEndSession": true,
        "outputSpeech": {
          "type": "SSML",
          "ssml": "<speak>Hmm <break time=\"1s\"/> What day do you want to know about?</speak>"
        }
      }
    });
  }
  // ...
});

Don’t forget about our requestVerifier that we created earlier - that piece of middleware happens before our route. Inside our route handler we check the “type” of the request first, handling a “LaunchRequest” a little different than the others. Next we might handle the “SessionEndedRequest”, which is also unique:
app.post('/forecast', requestVerifier, function(req, res) {
  if (req.body.request.type === 'LaunchRequest') { /* ... */ }
  else if (req.body.request.type === 'SessionEndedRequest') {
    // Per the documentation, we do NOT send ANY response... I know, awkward.
    console.log('Session ended', req.body.request.reason);
  }
  // ...
});

Lastly, we handle our own IntentRequests. A single skill can have many different Intents, so you will probably want to look at the name of the intent before proceeding. Of course, don’t forget to validate that you have all the data (“slots”) that you need as well!
app.post('/forecast', requestVerifier, function(req, res) {
  if (req.body.request.type === 'LaunchRequest') { /* ... */ }
  else if (req.body.request.type === 'SessionEndedRequest') { /* ... */ }
  else if (req.body.request.type === 'IntentRequest' &&
           req.body.request.intent.name === 'Forecast') {

    if (!req.body.request.intent.slots.Day ||
        !req.body.request.intent.slots.Day.value) {
      // Handle this error by producing a response like:
      // "Hmm, what day do you want to know the forecast for?"
    }
    let day = new Date(req.body.request.intent.slots.Day.value);

    // Do your business logic to get weather data here!
    // Then send a JSON response...

    res.json({
      "version": "1.0",
      "response": {
        "shouldEndSession": true,
        "outputSpeech": {
          "type": "SSML",
          "ssml": "<speak>Looks like a great day!</speak>"
        }
      }
    });
  }
});

Testing Out Your Skillz

Amazon Echo
Now that we’ve defined our skill on Amazon’s developer portal and built our Node.js application we’re ready to try it out! Make sure that you’ve deployed your Node app somewhere (Heroku maybe?) and that you’ve specified that URL for your skill in the developer portal. You should now be able to invoke your skill and use it from your own Echo device!
Of course, if you don’t have an echo device (or aren’t working from a place where you can easily interact with it, or just don’t want to…) you can test your skill a few other ways. First, you should probably write some unit tests for your code. You will most likely want to bypass the verification step (middleware) in your development or testing environment, but otherwise this would work just like any other web app in Node.
Next, you can test individual requests to your app from Amazon itself. Head over to the developer portal and into your new skill; then click on the “Test” option in the left sidebar. This page has the “Voice Simulator” where you can put in some SSML and hear how Alexa would say it; and the “Service Simulator” where you can have Amazon send your server requests based on what a user might say.
The Service Simulator is great for seeing exactly what JSON will get sent to your server and what JSON your server gives back in response. I typically do this, then snag that JSON and reuse it in my automated tests later.
The last tool I’ll point out is <echosim.io>. This site is basically a full online replica of an Amazon Echo device, connected to your Amazon account (so your skill is “installed”), and will emulate all of the input and output speech functionality. I highly encourage using this to test your skill in the final steps before you submit it for certification.</echosim.io>

Wrapping Things Up and Caveats

Thanks for reading all the way to the bottom! ( Or for skipping down here to see how things turned out. ;) ) Now that we’ve got something up and running, you may want to get your skill certified in order for other end users to add it to their Echo! Currently, you cannot charge for skills, but of course, you could require a user to have a “premium” account with your service/site/etc.
The last two caveats are just about some limitations of the Alexa platform. First, there is no way to have Alexa speak to the end user without an active request. In other words, you can’t have Alexa inform the user when the weather forecast changes. (This is more broadly known as “server push”, which is something we - the community - think Alexa is moving toward, but no guarantees.)
Second, there is no easy way to accept arbitrarily formatted input. For example, if you wanted to allow the user to add reminders that your server would email out at specific dates and times it would be easy to get the date (and time) for the reminder, but the text of it would be difficult. The custom slots discussed earlier are the best way to do something like this, but if you recall, those work best with a finite list of possible values (like a list of colors).
If you want to have the “reminder” text you would need to add many, many more entries to your possible slot values. Amazon will do some inference from these defined values, so it is not a concrete list, but that inference is not very broadly applied. Custom slots can have a maximum of 50,000 values defined - across all slots in total - but even then you might not be able to cover anything a user could say. Because of this, “free text” slots are not really viable currently.

Do I have to write my skill in Node?

You most certainly do not have to write your skill in Node. So long as your application supports POST requests over HTTPS and can process and respond with JSON, you’re good to go! In fact, many people are writing skills on Amazon’s on platform: Lamda. One of the benefits here is that you do not need to verify the incoming HTTP requests because Amazon trusts its own servers!

Can I see your demo skill?


If you’re curious to see the simple forecasting demo skill, check it out on GitHub! It is not considered complete, but feel free to fork it and hack away!

Suggest For You


How To Building an Amazon Alexa Skill with Node.js

Read More


Hi Bro,
Today we are going to learn how to build a simple “Universal JavaScript” app (a.k.a. “Isomorphic”) using React RouterExpress and React,
Enjoy your passion and… it’s going to be fun now!

About the author

Hi, I am Luciano and I am the co-author of Node.js Design Patterns Second Edition (Packt), a book that will take you on a journey across various ideas and components, and the challenges you would commonly encounter while designing and developing software using the Node.js platform. In this book you will discover the “Node.js way” of dealing with design and coding decisions. It also features an entire chapter dedicated to Universal JavaScript. If this is the first time you read this term, keep reading, you are going to love this article!

About Universal JavaScript

One of the advantages of having Node.js as runtime for the backend of a web application is that we have to deal only with JavaScript as a single language across the web stack. With this capability it is totally legit to be willing to share some code between the frontend and the backend to reduce the code duplication between the browser and the server to the bare minimun. The art of creating JavaScript code that is “environment agnostic” is today being recognized as “Universal JavaScript”, term that — after a very long debate — seems to have won a war against the original name “Isomorphic JavaScript”.
The main concerns that we generally have to face when building an Universal JavaScript application are:
  • Module sharing: how to use Node.js modules also in the browser.
  • Universal rendering: how to render the views of the application from the server (during the initialization of the app) and then keep rendering the other views directly in the browser (avoiding a full page refresh) while the user keep navigating across the diffent sections.
  • Universal routing: how to recognize the view associated to the current route from both the server and the browser.
  • Universal data retrival: how to access data (typically through APIs) from both the server and the browser.
Universal JavaScript is still a pretty fresh field and there is no framework or approach that emerged as a “de-facto” standard with ready-made solutions for all these problems yet. Altough, there are already a miriad of stable and well known libraries and tools that can be combined to successfully build a Universal JavaScript web application.
In this article we are going to use React (with its companion library React Router) and Express to build a simple application focused on showcasing universal rendering and routing. We will also use Babel to take advantage of the lovely EcmaScript 2015 syntax and Webpack to build our code for the browser.

What we are going to build

I am a Judo fan and so the app we are going to build today is “Judo Heroes”, a web app that showcases some the most famous Judo athletes and their collection of medals from the Olympic Games and other prestigious international tournaments.
This app has essentially two views:
An index page where you can select the athletes:
Universal JavaScript demo app
And an athlete page that showcases their medals and some other details:
Universal JavaScript demo app
To understand better how it works you can have a look at the demo app and navigate across the views.
What’s the matter with it anyway, you are probably asking yourself! Yes, it looks like a very simple app, with some data and a couple of views…
Well there’s something very peculiar that happens behind the scenes that will be hardly noticed by a regular user but it makes developement super interesting: this app is using universal rendering and routing!
We can prove this using the developers tools of the browser. When we initially load a page in the browser (any page, not necessarily the home page, try for example this one) the server provides the full HTML code of the view and the browser only needs to download linked resources (images, stylesheets and scripts):
Universal JavaScript demo app
Then, from there, when we switch to another view, everything happens only on the browser: no HTML code is loaded from the server and only the new resources (3 new images in the following example) are loaded by the browser:
Universal JavaScript demo app
We can do another quick test (if you are still not convinced) from the command line using curl:
curl -sS "https://judo-heroes.herokuapp.com/athlete/teddy-riner"
You will see the full HTML page (including the code rendered by React) being generated directly from the server:
Universal JavaScript demo app
I bet you are now convinced enough and eager to get your hands dirty, so let’s start coding!

Folder structure

At the end of this tutorial our project structure will look like in the following tree:
├── package.json
├── webpack.config.js
├── src
│   ├── app-client.js
│   ├── routes.js
│   ├── server.js
│   ├── components
│   │   ├── AppRoutes.js
│   │   ├── AthletePage.js
│   │   ├── AthletePreview.js
│   │   ├── AthletesMenu.js
│   │   ├── Flag.js
│   │   ├── IndexPage.js
│   │   ├── Layout.js
│   │   ├── Medal.js
│   │   └── NotFoundPage.js
│   ├── data
│   │   └── athletes.js
│   ├── static
│   │   ├── index.html
│   │   ├── css
│   │   ├── favicon.ico
│   │   ├── img
│   │   └── js
│   └── views
 `       └──  index.ejs
In the main level we have our package.json (to describe the project and define the dependencies) and webpack.config.js (Webpack configuration file).
All the rest of the code will be stored inside the folder src, which contains the main files needed for routing (routes.js) and rendering (app-client.js and server.js). It also contains 4 subfolders:
  • components: contains all the React components
  • data: contains our data “module”
  • static: contains all the static files needed for our application (css, js, images, etc.) and an index.html that we will use initially to test our app.
  • views: contains the template that we will use from the server to render the HTML content from the server.

Project initialization

The only requisite here is to have Node.js (version 6 is preferred) and NPM installed in your machine.
Let’s create a new folder called judo-heroes somewhere in the disk and point the terminal there, then launch:
npm init
This will bootstrap our Node.js project allowing us to add all the needed dependencies.
We will need to have babel, ejs, express, react and react-router installed. To do so you can run the following command:
npm install --save babel-cli@6.11.x babel-core@6.13.x  \
  babel-preset-es2015@6.13.x babel-preset-react@6.11.x ejs@2.5.x \
  express@4.14.x react@15.3.x react-dom@15.3.x react-router@2.6.x
We will also need to install Webpack(with its Babel loader extension) and http-server as a development dependencies:
npm install --save-dev webpack@1.13.x babel-loader@6.2.x http-server@0.9.x

The HTML boilerplate

From now on, I am assuming you have a basic knowledge of React and JSX and its component based approach. If not you can read an excellent article on React components or have a look at all the other React related articles on Codequs.com.
Initially we will focus only on creating a functional “Single Page Application” (with only client side rendering). Later we will see how to improve it by adding universal rendering and routing.
So the first thing we need is an HTML boilerplate to “host” our app that we will store in src/static/index.html:
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Judo Heroes - A Universal JavaScript demo application with React</title>
    <link rel="stylesheet" href="/css/style.css">
  </head>
  <body>
    <div id="main"></div>
    <script src="/js/bundle.js"></script>
  </body>
</html>
Nothing special here. Only two main things to underline:
  • We are using a simple “hand-made” stylesheet that you might want to download and save it under src/static/css/.
  • We also reference a /js/bundle.js file that contains all our JavaScript frontend code. We will see later in the article how to generate it using Webpack and Babel, so you don’t need to worry about it now.

The data module

In a real world application we would probably use an API to obtain the data necessary for our application.
In this case we have a very small dataset with only 5 athletes and some related information, so we can keep things simple and embed the data into a JavaScript module. This way we can easily import the data into any other component or module synchronously, avoiding the added complexity and the pitfalls of managing asynchronous APIs in an Universal JavaScript project, which is not the goal of the article.
Let’s see how the module looks like:
// src/data/athletes.js
const athletes = [
  {
    'id': 'driulis-gonzalez',
    'name': 'Driulis González',
    'country': 'cu',
    'birth': '1973',
    'image': 'driulis-gonzalez.jpg',
    'cover': 'driulis-gonzalez-cover.jpg',
    'link': 'https://en.wikipedia.org/wiki/Driulis_González',
    'medals': [
      { 'year': '1992', 'type': 'B', 'city': 'Barcelona', 'event': 'Olympic Games', 'category': '-57kg' },
      { 'year': '1993', 'type': 'B', 'city': 'Hamilton', 'event': 'World Championships', 'category': '-57kg' },
      { 'year': '1995', 'type': 'G', 'city': 'Chiba', 'event': 'World Championships', 'category': '-57kg' },
      { 'year': '1995', 'type': 'G', 'city': 'Mar del Plata', 'event': 'Pan American Games', 'category': '-57kg' },
      { 'year': '1996', 'type': 'G', 'city': 'Atlanta', 'event': 'Olympic Games', 'category': '-57kg' },
      { 'year': '1997', 'type': 'S', 'city': 'Osaka', 'event': 'World Championships', 'category': '-57kg' },
      { 'year': '1999', 'type': 'G', 'city': 'Birmingham', 'event': 'World Championships', 'category': '-57kg' },
      { 'year': '2000', 'type': 'S', 'city': 'Sydney', 'event': 'Olympic Games', 'category': '-57kg' },
      { 'year': '2003', 'type': 'G', 'city': 'S Domingo', 'event': 'Pan American Games', 'category': '-63kg' },
      { 'year': '2003', 'type': 'S', 'city': 'Osaka', 'event': 'World Championships', 'category': '-63kg' },
      { 'year': '2004', 'type': 'B', 'city': 'Athens', 'event': 'Olympic Games', 'category': '-63kg' },
      { 'year': '2005', 'type': 'B', 'city': 'Cairo', 'event': 'World Championships', 'category': '-63kg' },
      { 'year': '2006', 'type': 'G', 'city': 'Cartagena', 'event': 'Central American and Caribbean Games', 'category': '-63kg' },
      { 'year': '2006', 'type': 'G', 'city': 'Cartagena', 'event': 'Central American and Caribbean Games', 'category': 'Tema' },
      { 'year': '2007', 'type': 'G', 'city': 'Rio de Janeiro', 'event': 'Pan American Games', 'category': '-63kg' },
      { 'year': '2007', 'type': 'G', 'city': 'Rio de Janeiro', 'event': 'World Championships', 'category': '-63kg' },
    ],
  },
  {
    // ...
  }
];

export default athletes;
For brevity the file here has been truncated, and we are displaying just the data of one of the five athletes. If you want to see the full code check it out on the official repository. You can download the file into src/data/athletes.js.
As you can see, the file contains an array of objects where every object represents an athlete containing some generic information like id, name and country and another array of objects representing the medals won by that athlete.
You might also want to grab all the image files from the repository and copy them under: src/static/img/.

React components

We are going to organize the views of our application into several components:
  • A set of small UI components used to build the views: AthletePreview, Flag, Medal and AthletesMenu.
  • A Layout component that is used as master component to define the generic appearence of the application (header, content and footer blocks).
  • Two main components that represent the main sections: IndexPage and AthletePage.
  • An extra “page” component that we will use as 404 page: NotFoundPage
  • The AppRoutes component that uses React Router to manage the routing between views.

Flag component

The first component that we are going to build allows us to display a nice flag and, optionally, the name of the country that it represents:
// src/components/Flag.js
import React from 'react';

const data = {
  'cu': {
    'name': 'Cuba',
    'icon': 'flag-cu.png',
  },
  'fr': {
    'name': 'France',
    'icon': 'flag-fr.png',
  },
  'jp': {
    'name': 'Japan',
    'icon': 'flag-jp.png',
  },
  'nl': {
    'name': 'Netherlands',
    'icon': 'flag-nl.png',
  },
  'uz': {
    'name': 'Uzbekistan',
    'icon': 'flag-uz.png',
  }
};

export default class Flag extends React.Component {
  render() {
    const name = data[this.props.code].name;
    const icon = data[this.props.code].icon;
    return (
      <span className="flag">
        <img className="icon" title={name} src={`/img/${icon}`}/>
        {this.props.showName && <span className="name"> {name}</span>}
      </span>
    );
  }
}
As you might have noticed this component uses a small array of countries as data source. Again this makes sense only because we need a very small data set which, for the sake of this demo app, is not going to change. In a real application with a larger and more complex data set you might want to use an API or a different mechanism to connect the data to the component.
In this component it’s also important to notice that we are using two different props, code and showName. The first one is mandatory and must be passed to the component to select which flag will be shown among the ones supported. The showName props is instead optional and if set to a truthy value the component will also display the name of the country just after the flag.
If you want to build a more refined reusable component for a real world app you might also want to add to it props validation and defaults, but we are going to skip this step here as this is not the goal for the app we want to build.

Medal component

The Medal component is similar to the Flag component. It receives some props that represent the data related to a medal: the type (G for gold, S for silver and B for bronze), the year when it was won, the name of the event and the city where the tournament was hosted and the category where the athlete who won the medal competed.
// src/components/Medal.js
import React from 'react';

const typeMap = {
  'G': 'Gold',
  'S': 'Silver',
  'B': 'Bronze'
};

export default class Medal extends React.Component {
  render() {
    return (
      <li className="medal">
        <span className={`symbol symbol-${this.props.type}`} title={typeMap[this.props.type]}>{this.props.type}</span>
        <span className="year">{this.props.year}</span>
        <span className="city"> {this.props.city}</span>
        <span className="event"> ({this.props.event})</span>
        <span className="category"> {this.props.category}</span>
      </li>
    );
  }
}
As for the previous component here we also use a small object to map the codes of the medal types to descriptive names.

Athletes Menu component

In this section we are going to build the menu that is displayed on top of every athlete page to allow the user to easily switch to another athlete without going back to the index:
// src/components/AthletesMenu.js
import React from 'react';
import { Link } from 'react-router';
import athletes from '../data/athletes';

export default class AthletesMenu extends React.Component {
  render() {
    return (
      <nav className="atheletes-menu">
        {athletes.map(menuAthlete => {
          return <Link key={menuAthlete.id} to={`/athlete/${menuAthlete.id}`} activeClassName="active">
            {menuAthlete.name}
          </Link>;
        })}
      </nav>
    );
  }
}
The component is very simple, but there are some key points to underline:
  • We are importing our data module directly into the component to have access to the list of athletes available in the app.
  • We use the map method to iterate over all the athletes and generate for every one of them a Link.
  • Link is a special component provided by React Router to create links between views.
  • Finally, we use the prop activeClassName to use the class active when the current route matches the path of the link.

Athlete Preview component

The AthletePreview component is used in the index to display the pictures and the names of the athletes. Let’s see its code:
// src/components/AthletePreview.js
import React from 'react';
import { Link } from 'react-router';

export default class AthletePreview extends React.Component {
  render() {
    return (
      <Link to={`/athlete/${this.props.id}`}>
        <div className="athlete-preview">
          <img src={`img/${this.props.image}`}/>
          <h2 className="name">{this.props.name}</h2>
          <span className="medals-count"><img src="/img/medal.png"/> {this.props.medals.length}</span>
        </div>
      </Link>
    );
  }
}
The code is quite simple. We expect to receive a number of props that describe the attributes of the athlete we want to display like id, image, name and medals. Note that again we are using the Link component to create a link to the athlete page.

Layout component

Now that we built all our basic components let’s move to creating those that give the visual structure to the application. The first one is the Layout component, which has the only purpose of providing a display template to the whole application defining an header, a space for the main content and a footer:
// src/components/Layout.js
import React from 'react';
import { Link } from 'react-router';

export default class Layout extends React.Component {
  render() {
    return (
      <div className="app-container">
        <header>
          <Link to="/">
            <img className="logo" src="/img/logo-judo-heroes.png"/>
          </Link>
        </header>
        <div className="app-content">{this.props.children}</div>
        <footer>
          <p>
            This is a demo app to showcase universal rendering and routing with <strong>React</strong> and <strong>Express</strong>.
          </p>
        </footer>
      </div>
    );
  }
}
The component is pretty simple and we should understand how it works just by looking at the code. There’s though a very interesting prop that we are using here, the children prop. This is a special property that React provides to every component and allows to nest components one inside another.
We are going to see in the routing section how the React Router will make sure to nest the components into the Layout component.

Index Page component

This component constitutes the full index page and it contains some of the components we previously defined:
// src/components/IndexPage.js
import React from 'react';
import AthletePreview from './AthletePreview';
import athletes from '../data/athletes';

export default class IndexPage extends React.Component {
  render() {
    return (
      <div className="home">
        <div className="athletes-selector">
          {athletes.map(athleteData => <AthletePreview key={athleteData.id} {...athleteData} ></AthletePreview>)}
        </div>
      </div>
    );
  }
}
Note that in this component we are using the AthletePreview component we created previously. Basically we are iterating over all the available athletes from our data module and creating an AthletePreview component for each of them. The AthletePreview component is data agnostic, so we need to pass all the information about the current athlete as props using the JSX spread operator ({...object}).

Athlete Page component

In a similar fashion we can define the AthletePage component:
// src/components/AthletePage.js
import React from 'react';
import { Link } from 'react-router';
import NotFoundPage from './NotFoundPage';
import AthletesMenu from './AthletesMenu';
import Medal from './Medal';
import Flag from './Flag';
import athletes from '../data/athletes';

export default class AthletePage extends React.Component {
  render() {
    const id = this.props.params.id;
    const athlete = athletes.filter((athlete) => athlete.id === id)[0];
    if (!athlete) {
      return <NotFoundPage></NotFoundPage>;
    }
    const headerStyle = { backgroundImage: `url(/img/${athlete.cover})` };
    return (
      <div className="athlete-full">
        <AthletesMenu></AthletesMenu>
        <div className="athlete">
          <header style={headerStyle}></header>
          <div className="picture-container">
            <img src={`/img/${athlete.image}`}/>
            <h2 className="name">{athlete.name}</h2>
          </div>
          <section className="description">
            Olympic medalist from <strong><Flag code={athlete.country} showName="true"></Flag></strong>,
            born in {athlete.birth} (Find out more on <a href={athlete.link} target="_blank">Wikipedia</a>).
          </section>
          <section className="medals">
            <p>Winner of <strong>{athlete.medals.length}</strong> medals:</p>
            <ul>{
              athlete.medals.map((medal, i) => <Medal key={i} {...medal}></Medal>)
            }</ul>
          </section>
        </div>
        <div className="navigateBack">
          <Link to="/">« Back to the index</Link>
        </div>
      </div>
    );
  }
}
By now, you must be able to understand most of the code shown here and how the other components are used to build this view. What might be important to underline is that this page component accepts from the outside only the id of the athlete, so we include the data module to be able to retrieve the related information. We do this at the beginning of the render method using the function filter on the data set. We are also considering the case where the received id does not exist in our data module, in this case we render NotFoundPage, a component that we are going to create in the next section.
One last important detail is that here we are accessing the id with this.props.params.id (instead of simply this.props.id): params is a special object created by React Router when using a component from a Route and it allows to propagate routing parameters into components. It will be easier to understand this concept when we will see how to setup the routing part of the application.

Not Found Page component

Now let’s see the NotFoundPage component, which acts as a template to generate the code of our 404 pages:
// src/components/NotFoundPage.js
import React from 'react';
import { Link } from 'react-router';

export default class NotFoundPage extends React.Component {
  render() {
    return (
      <div className="not-found">
        <h1>404</h1>
        <h2>Page not found!</h2>
        <p>
          <Link to="/">Go back to the main page</Link>
        </p>
      </div>
    );
  }
}

App Routes component

The last component we need to create is the AppRoutes component which is the master component that renders all the other views using internally the React Router. This component will use the routes module, so let’s have a quick look at it first:
// src/routes.js
import React from 'react'
import { Route, IndexRoute } from 'react-router'
import Layout from './components/Layout';
import IndexPage from './components/IndexPage';
import AthletePage from './components/AthletePage';
import NotFoundPage from './components/NotFoundPage';

const routes = (
  <Route path="/" component={Layout}>
    <IndexRoute component={IndexPage}></IndexRoute>
    <Route path="athlete/:id" component={AthletePage}></Route>
    <Route path="*" component={NotFoundPage}></Route>
  </Route>
);

export default routes;
In this file we are basically using the React Router Route component to map a set of routes to the page components we defined before. Note how the routes are nested inside a main Route component. Let’s explain how this works:
  • The root route maps the path / to the Layout component. This allows us to use our custom layout in every section of our application. The components defined into the nested routes will be rendered inside the Layout component in place of the this.props.children property that we discussed before.
  • The first child route is an IndexRoute which is a special route used to define the component that will be rendered when we are viewing the index page of the parent route (/ in this case). We use our IndexPage component as index route.
  • The path athlete/:id is mapped to the AthletePage. Note here that we are using a named parameter :id. So this route will match all the paths with the prefix /athlete/, the remaining part will be associated to the params id and will be available inside the component in this.props.params.id.
  • Finally the match-all route * maps every other path to the NotFoundPage component. This route must be defined as the last one.
Let’s see now how to use these routes with the React Router inside our AppRoutes component:
// src/components/AppRoutes.js
import React from 'react';
import { Router, browserHistory } from 'react-router';
import routes from '../routes';

export default class AppRoutes extends React.Component {
  render() {
    return (
      <Router history={browserHistory} routes={routes} onUpdate={() => window.scrollTo(0, 0)}/>
    );
  }
}
Basically we only need to import the Router component and add it inside our render function. The router receives our routes mapping in the router prop. We also configure the history prop to specify that we want to use the HTML5 browser history for the routing (as an alternative you could also use hashHistory).
Finally we also added an onUpdate callback to reset the scrolling of the window to the top everytime a link is clicked.

The application entry point

The last bit of code to complete our first version of the application is to define the JavaScript logic that initializes the whole app in the browser:
// src/app-client.js
import React from 'react';
import ReactDOM from 'react-dom';
import AppRoutes from './components/AppRoutes';

window.onload = () => {
  ReactDOM.render(<AppRoutes></AppRoutes>, document.getElementById('main'));
};
The only thing we do here is to import our master AppRoutes component and render it using the ReactDOM.render method. The React app will be living inside our #main DOM element.

Setting up Webpack and Babel

Before we are able to run our application we need to generate the bundle.js file containing all our React components with Webpack. This file will be executed by the browser so Webpack will make sure to convert all the modules into code that can be executed in the most common browser environments. Webpack will convert ES2015 and React JSX syntax to equivalent ES5 syntax (using Babel), which can be executed practically by every browser. Furthermore we can use Webpack to apply a number of optimizations to the resulting code like combining all the scripts files into one file and minifying the resulting bundle.
Let’s write our webpack configuration file:
// webpack.config.js
const webpack = require('webpack');
const path = require('path');

module.exports = {
  entry: path.join(__dirname, 'src', 'app-client.js'),
  output: {
    path: path.join(__dirname, 'src', 'static', 'js'),
    filename: 'bundle.js'
  },
  module: {
    loaders: [{
      test: path.join(__dirname, 'src'),
      loader: ['babel-loader'],
      query: {
        cacheDirectory: 'babel_cache',
        presets: ['react', 'es2015']
      }
    }]
  },
  plugins: [
    new webpack.DefinePlugin({
      'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV)
    }),
    new webpack.optimize.DedupePlugin(),
    new webpack.optimize.OccurenceOrderPlugin(),
    new webpack.optimize.UglifyJsPlugin({
      compress: { warnings: false },
      mangle: true,
      sourcemap: false,
      beautify: false,
      dead_code: true
    })
  ]
};
In the first part of the configuration file we define what is the entry point and the otput file. The entry point is the main JavaScript file that initializes the application. Webpack will recursively resolve all the included/imported resources to determine which files will go into the final bundle.
The module.loaders section allows to specify transformations on specific files. Here we want to use Babel with the react and es2015 presets to convert all the included JavaScript files to ES5 code.
In the final section we use plugins to declare and configure all the optimizations plugins we want to use:
  • DefinePlugin allows us to define the NODE_ENV variable as a global variable in the bundling process as if it was defined in one of the scripts. Some modules (e.g. React) relies on it to enable or disable specific features for the current environment (production or development).
  • DedupePlugin removes all the duplicated files (modules imported in more than one module).
  • OccurenceOrderPlugin helps in reducing the file size of the resulting bundle.
  • UglifyJsPlugin minifies and obfuscates the resulting bundle using UglifyJs.
Now we are ready to generate our bundle file, you just need to run:
NODE_ENV=production node_modules/.bin/webpack -p
The NODE_ENV environment variable and the -p option are used to generate the bundle in production mode, which will apply a number of additional optimizations, for example removing all the debug code from the React library.
If everything went fine you will now have your bundle file in src/static/js/bundle.js.

Playing with the single page app

We are finally ready to play with the first version of our app!
We don’t have a Node.js web server yet, so for now we can just use the module http-server (previously installed as development dependency) to run a simple static file web server:
node_modules/.bin/http-server src/static
And your app will be magically available on http://localhost:8080.
Ok, now take some time to play with it, click on all the links and explore all the sections.
Does everything seem to work allright? Well, almost! There’s just a little caveat… If you refresh the page in a section different from the index you will get a 404 error from the server.
There are a number of ways to address this problem. In our case it will be solved as soon as we implement our universal routing and rendering solution, so let’s move on to the next section.

Routing and rendering on the server with Express

Ok, we are now ready to evolve our application to the next level and build the missing server side part.
In order to have server side routing and rendering we will use Express with a relatively small server script that we will see in a moment.
The rendering part will use an ejs template as replacement for our index.html file that we will save in src/views/index.ejs:
<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Judo Heroes - A Universal JavaScript demo application with React</title>
    <link rel="stylesheet" href="/css/style.css">
  </head>
  <body>
    <div id="main"><%- markup -%></div>
    <script src="/js/bundle.js"></script>
  </body>
</html>
The only difference with the original HTML file is that we are using the template variable <%- markup -%> inside the #main div in order to include the React markup into the server-generated HTML code.
Now we are ready to write our server application:
// src/server.js

import path from 'path';
import { Server } from 'http';
import Express from 'express';
import React from 'react';
import { renderToString } from 'react-dom/server';
import { match, RouterContext } from 'react-router';
import routes from './routes';
import NotFoundPage from './components/NotFoundPage';

// initialize the server and configure support for ejs templates
const app = new Express();
const server = new Server(app);
app.set('view engine', 'ejs');
app.set('views', path.join(__dirname, 'views'));

// define the folder that will be used for static assets
app.use(Express.static(path.join(__dirname, 'static')));

// universal routing and rendering
app.get('*', (req, res) => {
  match(
    { routes, location: req.url },
    (err, redirectLocation, renderProps) => {

      // in case of error display the error message
      if (err) {
        return res.status(500).send(err.message);
      }

      // in case of redirect propagate the redirect to the browser
      if (redirectLocation) {
        return res.redirect(302, redirectLocation.pathname + redirectLocation.search);
      }

      // generate the React markup for the current route
      let markup;
      if (renderProps) {
        // if the current route matched we have renderProps
        markup = renderToString(<RouterContext {...renderProps}></RouterContext>);
      } else {
        // otherwise we can render a 404 page
        markup = renderToString(<NotFoundPage></NotFoundPage>);
        res.status(404);
      }

      // render the index template with the embedded React markup
      return res.render('index', { markup });
    }
  );
});

// start the server
const port = process.env.PORT || 3000;
const env = process.env.NODE_ENV || 'production';
server.listen(port, err => {
  if (err) {
    return console.error(err);
  }
  console.info(`Server running on http://localhost:${port} [${env}]`);
});
The code is commented, so it shouldn’t be hard to get a general understanding of what is going on here.
The important part of the code here is the Express route defined with app.get('*', (req, res) => {...}). This is an Express catch-all route that will intercept all the GET requests to every URL in the server. Inside this route, we take care of delegating the routing logic to the React Router match function.
ReactRouter.match accepts two parameters: the first one is a configuration object and the second is a callback function. The configuration object must have two keys:
  • routes: used to pass the React Router routes configuration. Here, we are passing the exact same configuration that we used for the client-side rendering.
  • location: This is used to specify the currently requested URL.
The callback function is called at the end of the matching. It will receive three arguments, error, redirectLocation, and renderProps, that we can use to determine what exactly the result of the match operation was.
We can have four different cases that we need to handle:
  • The first case is when we have an error during the routing resolution. To handle this case, we simply return a 500 internal server error response to the browser.
  • The second case is when we match a route that is a redirect route. In this case, we need to create a server redirect message (302 redirect) to tell the browser to go to the new destination (this is not really happening in our application because we are not using redirect routes in our React Router configuration, but it’s good to have it ready in case we decide to keep evolving our application).
  • The third case is when we match a route and we have to render the associated component. In this case, the argument renderProps is an object that contains the data we need to use to render the component. The component we are rendering is RouterContext (contained in the React Router module), which is responsible for rendering the full component tree using the values in renderProps.
  • The last case is when the route is not matched, and here we can simply return a 404 not found error to the browser.
This is the core of our server- side routing mechanism and we use the ReactDOM.renderToString function to be able to render the HTML code that represents the component associated to the currently matched route.
Finally, we inject the resulting HTML into the index.ejs template we defined before to obtain the full HTML page that we send to the browser.
Now we are ready to run our server.js script, but because it’s using the JSX syntax we cannot simply run it with the node interpreter. We need to use babel-node and the full command (from the root folder of our project) looks like this:
NODE_ENV=production node_modules/.bin/babel-node --presets 'react,es2015' src/server.js

Running the complete app

At this stage your app is available at http://localhost:3000 and, for the sake of this tutorial, it can be considered complete.
Again feel free to check it out and try all the sections and links. You will notice that this time we can refresh every page and the server will be able to identify the current route and render the right page.
Small advice: don’t forget to check out the 404 page by entering a random non-existing URL!

Wrapping up

Code

Demo

Suggest for you:


How To Build a Universal React and Node App

Read More

Copyright © WISE CODE DECK | Designed With By Blogger Templates
Scroll To Top