Saturday, August 22, 2015

Write your own Chat service with Vert.x and RiotJS - Part 2

Welcome back!

Glad you made it! After all I promised a fully functioning chat service and haven’t delivered yet!
Let’s see what part deux will do for you. Our goal is to have something chat-like working and be able to deploy it.

Piercing the layers

We are following a tiered approach to architecting this software. There will be a front-end, a web-server and some kind of database.

Instead of completely developing all three tiers after the other, I’m a firm believer in focusing on one or two use-cases and following them through all the layers.
We neglected this a little bit in part 1, starting with the middle tier, but we will try to follow this principle for the rest of the tutorial.

RiotJS 2.x - the good parts of React and Polymer

In order to keep the learning curve small, I opted for RiotJS for our browser-based UI.
It is simple, easy to understand and keeps the amount of magic behind the scenes to a minimum.
We will be using it to develop components (i.e. custom tags) that communicate with each other through events.

A component is basically a piece of HTML template with view/controller logic inside.

Each component will also have a shared API object to talk to our back-end.
It is a pretty simple but battle-tested pattern.

Here’s what a typical interaction with our UI would look like:

Created with Raphaël 2.1.2UserUserRiotTagRiotTagAPIAPIVert.xVert.x<click>channels()GET /chatty/channels{channels:[...]}channelsCallback()update()

Wow, sequence diagrams!

For styling we’ll stick to Bootstrap, since everybody seems to know that one.

Preparation

Add a web directory to your project directory and…
Well, this is getting too long already. Just grab the code from Github.

If you have a checkout already:

git pull
git checkout part2

If you are starting from scratch:

git clone https://github.com/beders/chatty.git
cd chatty
git checkout part2

In essence, we are adding a stub index.html, some custom tags, some local JavaScript files and are loading a few more from various CDNs.
The two most important JavaScript files will be vertxbus.js that comes with Vert.x and the implementation of our API in api.js.

Chatty UI

What components does a chat application have?
Maybe a main menu, a list of chat channels, a chat history for the currently selected channel and an input field to chat?
With RiotJS, we can start with something simple like this and evolve it easily:

<chatty>
    <chatty-menu/>
    <channel-list/>
    <channel-history>
        <chat-input/>
    </channel-history>
</chatty>

All we need to do is add a browser-side API, put these new custom tags in their own files and knit it all together in index.html

index.html

Our index.html looks deceptively simple:

<head>
... bootstrap, sockjs, jquery, riot ...
    <script src="js/vertxbus.js"></script>
    <script src="js/api.js"></script>
    <script src="tag/chatty.html" type="riot/tag"></script>
    <script src="tag/chatty-menu.html" type="riot/tag"></script>
    <script src="tag/chat-history.html" type="riot/tag"></script>
    <script src="tag/chat-input.html" type="riot/tag"></script>
<body>
<chatty></chatty>
<script>
    var eb = new vertx.EventBus('/chatty/channel-events');
    eb.onopen = function() {
        console.log("Event Bus is ready");
        api.eventBus = eb;
        riot.mount('chatty', {api: api});
    }
</script>
</body>

Ignore the head section for a moment.
There’s the new chatty tag, which we will initialize in a second.

We are setting up an event bus connection to vert.x first. This is the same event bus you used in part 1, all the way extended to the browser (more details about that at the end of this part).

We need to wait until we have a connection established before showing the UI, which is why the rest of the setup is in a callback function. (We’ll not be bothering with AMD or other module loaders for now).

We store a reference to the event bus in our API and then start the RiotJS magic.

The ‘head’ section loads various custom tags as typeriot/tag, one of them being the chatty custom tag.
Calling riot.mount('chatty', { api: api }) tells Riot to compile the contents of tag/chatty.html and insert it into the DOM. We also pass in options (available as opts in the custom tag code) which is our little api object that sits between the components and the server-side.

Here’s tag/chatty.html:

<chatty>
    <chatty-menu api="{opts.api}"></chatty-menu>
    <div class="container-fluid">
        <div class="row">
            <div class="col-md-1" style="height:100%;"><channel-list api={opts.api}></channel-list></div>
            <div class="col-md-11"><chat-history api="{opts.api}"></chat-history></div>
        </div>
    </div>
</chatty>

We added some scaffolding around our new custom tags (and you can probably tell, that I’m not super good at this). Let’s look at them more closely.

(inside chatty.html)

Things get slightly more interesting here, as we define what a channel list is and how we are getting the data.
As you can see, we have the actual template and the code controlling it in the same place, which makes it really easy to maintain and understand.
(We could even omit the script tag and use ES6 syntax if we wanted to)

<channel-list>
    <div style="margin-top:60px; margin-left:8px;">
    <ul class="nav nav-pills">
        <li>CHANNELS</li>
        <li each="{channels}"><a href="#" onclick="{parent.selectChannel}">{name}</a></li>
    </ul>
    </div>
    <script>
        var self = this;
        this.channels = [];

        opts.api.channels(function(ch) {
            self.channels = ch.channels;
            self.update();
        });

        this.selectChannel = function(event) {
            console.log(event.item);
            opts.api.trigger('channel.selected', event.item);
        }

    </script>
</channel-list>

The HTML part looks fairly standard with the exception of new attributes like each and substitutions inside curly braces. RiotJS supports any JS expression inside those.
What we want to do here is to get the list of channels from our server and display them inside an ul tag.

The element that has the each attribute is then repeated for each entry of our list. It also establishes a new context, i.e. the expression {name} refers to the current object in the {channels} list. (This is also the reason the onclick handler needs to refer to the parent context.)

If the list of channels changes, RiotJS will figure out how to re-arrange and change the DOM for us.

The script section has all the logic we need for this component. The opts variable contains any options passed into the component. As part of our design we chose to always have the api object passed in.
We call its channels function and provide a callback to get the list of channels via the REST API we wrote in part 1.
When we receive it, we store the result in our channels variable and update the component.

We also provided an onclick event handler: If a channel is clicked, selectChannel will be called with the channel object in event.item.

Now instead of trying to figure out which parts of the UI need to be updated, we simply send an channel.selected event via the api object with the currently selected channel and hope someone picks up on the other end.

This keeps the component clean of any other dependencies and makes it very simple.
Which is a good guideline: If your component’s code becomes too large, you might need to add a few more sub-components or re-think your component model.

Someone clicked on a channel! What now?

chatty-history.html

Well, in that case we want to switch to the chat history of that channel and show any incoming chat messages.

<chat-history>
    <div class="container" style="height:100%; overflow: scroll; margin-top:60px;">
        <div class="row" each="{chat in chatLines}">
            <div class="col-md-12">
                <system-message if="{chat.type=='system'}" message={chat}></system-message>
                <chat-message if={chat.type=='chat'} message={chat}></chat-message>
            </div>
        </div>
        <div class="row" style="height:40px;" id="empty"></div>
    </div>
    <chat-input api="{opts.api}"></chat-input>
    <script>
        var self = this;
        this.channel = {};
        this.chatLines = [];

        opts.api.on('channel.selected', function(channel){
            console.log("Switch to channel: " + JSON.stringify(channel));
            self.channel = channel;
            opts.api.subscribe(channel);
            self.chatLines.push({alias:'system', type:'system',
                message: '--- ' + self.channel.name + ' ---'});
            self.update();
        });

        opts.api.on('message.received', function(msg, channel) {
            if (channel.address == self.channel.address) {
                self.chatLines.push(msg);
                self.update();
                self.empty.scrollIntoView({
                    behavior: "smooth",
                    block: "end"
                });

            }
        });
    </script>
</chat-history>

The crucial part of this component is each="{chat in chatLines}".
We store all the chat lines in chatLines and we are inserting custom-tags for each of those. Depending on the type of chat message, we insert <system-message> or <chat-message> (check the if attribute).
Finally, we add a custom tag for users to enter a chat line.

The script part is equally simple: We register for internal events being sent via the api object (which is an Observable) and meet our old friend ‘channel.selected’.
Here we use api.subscribe(..) to register for any event bus messages being sent over this channel.

Lastly we register for any messages being sent from the API. If we receive one, we just stuff it into the chatLines array, let RiotJS update our DOM and then scroll the new line into view.

self.empty is automatically set to the DOM element with that ID. No need to look up elements via jQuery.

Custom tags like <chat-message> take care of the actual layout of the chat message. We are using bootstrap’s media classes for this.

chat-input.html

Time to actually provide an input field to type a message!

<chat-input>

    <footer class="footer navbar-fixed-bottom" style="left: 81px;">
    <form class="form-inline" action="#" style="margin:4px;">
        <div class="form-group">
            <div class="input-group">
                <div class="input-group-addon"><i class="glyphicon glyphicon-chevron-right"></i></div>
                <textarea disabled="{channel == null}" type="text" class="form-control" rows="1" cols="60" id="chatLine" placeholder="{channel ? 'Get chatty':'Select channel'}" onkeydown="{keydown}"></textarea>
                <!--<div class="input-group-addon"><i class=""</i></div>-->
            </div>
        </div>
    </form>
    </footer>

    <script>
        var self = this;
        self.channel = null;
        opts.api.on('channel.selected', function(channel){
            self.channel = channel;
            self.update();
            self.chatLine.focus();
        });

        this.keydown = function(event) {
            if (event.keyCode == 13) {
                if (event.shiftKey) {
                    self.chatLine.rows++;
                    return true;
                } else {
                    var line = self.chatLine.value.replace(/$/mg,'<br>');
                    var chat = {type: 'chat', message: line};
                    opts.api.sendChat(chat, self.channel);
                    self.chatLine.value = "";
                    self.chatLine.rows = 1;
                    return false;
                }
            }
            return true;
        }
    </script>
</chat-input>

The interesting parts are the attributes on the textarea tag and the component’s code. We have a few short JavaScript expressions in there.
If those get too long, we would simply add appropriate functions to the script part.

We register for the channel.selected event to focus on the textarea and we register a handler to receive keystrokes. We want the input field to grow by a line whenever shift+enter is pressed.
Note that in event handlers, RiotJS will call update() automatically.

chat-menu.html

We’ll skip that for now since we don’t have any proper user management yet. In the demo app, just fill in an e-mail address which will be used as chat alias (and avatar).

api.js

There is no vert.x or backend specific stuff in any of the UI components. All components consider the api object as the only facade into the dirty world of network communication and caching etc.

If we decide to replace the backend, we could do so if we stick to the interface defined by the api object.

api = (function() {
    var api = riot.observable();
    api.subscriptions = {};
    api.useAlias = function(alias)  {
        this.alias = alias;
    };

    api.channels = function(callback) {
        $.getJSON("/chatty/channels", callback);
    };

    api.subscribe = function(channel) {
        if (!api.subscriptions[channel.address]) {
            api.subscriptions[channel.address] = function (msg) {
                api.trigger('message.received', msg, channel);
            };
            this.eventBus.registerHandler(channel.address, api.subscriptions[channel.address]);
            console.log("Subscribed to " + channel);
        }
    };

    api.sendChat = function(message, channel) {
        message.alias = this.alias;
        message.avatar = this.avatar;
        this.eventBus.publish(channel.address, message);
    };


    // helper functions


    return api;
})();

(We don’t mess with modules yet, so excuse the global variable)
Our actual api object is an observable, which means it has on and trigger functions which are used to communicate with the UI components as seen.

Here we finally have some Vert.x specific code.
this.eventBus is the gateway to the server-side event bus and we use it to register for channel addresses (map subscriptions ensures we only register once).

Whenever a message from Vert.x is received, we simply broadcast it via the RiotJS event system to interested parties like <chat-history>.

We could provide our own chat message cache here, if we wanted to. Maybe synchronize it with the server-side when coming out of offline mode? Ahh, dreams for later tutorials.

sendChat is where we are actually sending off a chat message. We use broadcast here, which means all interested listeners will receive the message, i.e. each user of chatty that has a subscription for the current channel!
Vert.x takes care of the server-side distribution (we’ll see how in a moment).

There’s a few more functions:

  • useAlias just registers the chat alias to use
  • channels uses jQuery to make an AJAX call to our backend to retrieve the list of channels. Note that we also could use an event bus messages for this! Also, we could broadcast to the UI components that a new channel list has arrived. We’ll take care of that later when we can actually add/remove channels

That basically wraps it up for the client-side for now.

We have added a new use case to our app: sending a chat message.
But haven’t changed the backend yet to deal with it.

Vert.x EventBus via SockJS

Meanwhile, most of the actual web server code can be found in ChattyServer.java
I ditched our HelloChatty.js file and moved the REST API code.

We need to add support for sending event bus messages all the way to the browser and back. Vert.x comes with a SockJS server (and client) to accomplish this. SockJS allows us to notify connected clients (and will automatically figure out the best way to do this: short polling, long polling, websocket, etc.)

Let’s add a bridge that connects the internal Vert.x event bus to the outer world:

SockJSHandler sockJSHandler = SockJSHandler.create(vertx);
router.route("/chatty/channel-events/*").handler(sockJSHandler);

Voilá!
With this and vertxbus.js in your web/js directory, we can make SockJS connections and send any message.
And that is a little bit of a problem.

The client can’t be trusted.

Exactly. The client code can’t be trusted and you don’t want client-side code to be able to send and receive any message.
That’s why we can add filters to the bridge that allows connected clients to only send and receive specific messages.

BridgeOptions options = new BridgeOptions();                                          
options.addOutboundPermitted(new PermittedOptions().setAddressRegex("channel\\..*")); 
options.addInboundPermitted(new PermittedOptions().setAddressRegex("channel\\..*"));  
sockJSHandler.bridge(options);                                                        

Now, only addresses that start with channel. are allowed on the event bus bridge.

Serving Static Content

Let’s have Vert.x also serve all of our juciy UI code. For this, we are adding a static handler that is part of the vertx-web module.

StaticHandler web = StaticHandler.create("web").setFilesReadOnly(true).setDirectoryListing(false);
router.route("/chatty/app/*").handler(web);

This will serve the contents of our web directory as static file.

Let’s give it a go. Start ChattyServer
vertx run us.monoid.vertx.chatty.ChattyServer

and point your browser to:
http://localhost:8888/chatty/app/index.html

Pick an alias (if you use an e-mail address, we’ll get an avatar from gravatar) in the main menu, select a channel and chat away.
Open a second tab to the same address and chat a bit with yourself.
It’s therapeutic.
It will help you understand why you keep reading overly long tutorials.

Deployment

We’ll be using Heroku to deploy our little chat app.
If you haven’t done so yet, sign up for Heroku and install the Heroku toolbelt.
Heroku is free for our little test instance and it automates a lot of things for us.

Building a stand-alone JAR file

We’ll be using the lovely maven-shade plugin to package our app, along with the web pages and the kitchen-sink into a single JAR file.

Check the pom.xml plugin section for <artifactId>maven-shade-plugin</artifactId>

We need to configure it with a manifest entry, so it knows how to deploy our app:

<manifestEntries>
 <Main-Class>io.vertx.core.Starter</Main-Class>
 <Main-Verticle>us.monoid.vertx.chatty.ChattyServer</Main-Verticle>
</manifestEntries>

The shade plugin adds itself to the standard mvn package goal, so if we run mvn package now, we end up with a ‘fat’ jar. About 4.4MB for our little app. Not too bad!

Now we can run our app like this:

java -jar target/chatty-fat.jar

Neat!

Heroku deployment

We are going to use the Heroku deployment plug-in for deployment.
Since we are only deploying a single JAR file, we tell it to ignore all the rest.

<plugin>
   <groupId>com.heroku.sdk</groupId>
   <artifactId>heroku-maven-plugin</artifactId>
     <version>0.4.4</version>
     <configuration>
        <processTypes>
         <web>java -jar target/${project.artifactId}-fat.jar</web>
        </processTypes>
        <includeTarget>false</includeTarget>
           <includes>
          <include>target/${project.artifactId}-fat.jar</include>
          </includes></configuration></plugin>

One more thing we need to change: Heroku expects your app to use the PORT environment variable for your web application.

We’ll change our start verticle accordingly:

private int port = 8888;
    @Override
    public void start() throws Exception {
        Optional.ofNullable(System.getenv("PORT")).ifPresent(it -> port = Integer.parseInt(it));
        deployVerticles();
        startWebServer();
    }

and in startWebServer()

vertx.createHttpServer().requestHandler(router::accept).listen(port);

Still not quite there: We should add a forward from ‘/’ to the actual homepage of Chatty. There’s no forward() function available yet, so we have to write our own little handler.

router.route("/").handler(rc -> {
  if (rc.request().uri().equals("/")) {
    String uri = rc.request().absoluteURI();
    rc.response().setStatusCode(301).putHeader("Location", uri + "/chatty/app/index.html" ).end();
   } else {
    rc.next();
  }
 });

Next, go to the project root directory and run

heroku login
heroku create

This creates the app on the Heroku servers.
You only need to do this once!

Now, to build and deploy, run this:

mvn clean heroku:deploy

then run

heroku open

to see Chatty in all its (non)-glory!

And that concludes the second part. We still haven’t talked about persistence and there’s so much wrong with this app before it becomes a real chat app,
but we have a start.

Written with StackEdit.

6 comments :

  1. Is it possible to have some sort of authentication for EventBus client before is allowed to perform actions on the bus?

    You said it "The client can’t be trusted."

    ReplyDelete
  2. Using routes and handlers, you can secure any URL. Yes.

    ReplyDelete
  3. great! thanks!
    a question, can SockJSHandler accept NetSocket's message?
    that is, in a NetSocket Verticle, eventBus publish some message, can riotjs accept message from web client?

    ReplyDelete
  4. Any message put on the event bus can be read by the web client (if permissions are set that way)

    ReplyDelete
  5. How would one handle a client disconnecting from a room with Vert.x and sockJS? I would like to display a message saying that so-and-so has left the room if they close their browser window.

    ReplyDelete
  6. you have to run pings between connected clients and servers.
    if the server hasn't heard from a client for a while, broadcast a disconnect message to clients.
    There's no sure-fire way for when people just close their browser window.

    ReplyDelete