Saturday, August 22, 2015

Write your own Chat service with Vert.x and RiotJS - Part 2

Welcome back!

Glad you made it! After all I promised a fully functioning chat service and haven’t delivered yet!
Let’s see what part deux will do for you. Our goal is to have something chat-like working and be able to deploy it.

Piercing the layers

We are following a tiered approach to architecting this software. There will be a front-end, a web-server and some kind of database.

Instead of completely developing all three tiers after the other, I’m a firm believer in focusing on one or two use-cases and following them through all the layers.
We neglected this a little bit in part 1, starting with the middle tier, but we will try to follow this principle for the rest of the tutorial.

RiotJS 2.x - the good parts of React and Polymer

In order to keep the learning curve small, I opted for RiotJS for our browser-based UI.
It is simple, easy to understand and keeps the amount of magic behind the scenes to a minimum.
We will be using it to develop components (i.e. custom tags) that communicate with each other through events.

A component is basically a piece of HTML template with view/controller logic inside.

Each component will also have a shared API object to talk to our back-end.
It is a pretty simple but battle-tested pattern.

Here’s what a typical interaction with our UI would look like:

Created with Raphaël 2.1.2UserUserRiotTagRiotTagAPIAPIVert.xVert.x<click>channels()GET /chatty/channels{channels:[...]}channelsCallback()update()

Wow, sequence diagrams!

For styling we’ll stick to Bootstrap, since everybody seems to know that one.

Preparation

Add a web directory to your project directory and…
Well, this is getting too long already. Just grab the code from Github.

If you have a checkout already:

git pull
git checkout part2

If you are starting from scratch:

git clone https://github.com/beders/chatty.git
cd chatty
git checkout part2

In essence, we are adding a stub index.html, some custom tags, some local JavaScript files and are loading a few more from various CDNs.
The two most important JavaScript files will be vertxbus.js that comes with Vert.x and the implementation of our API in api.js.

Chatty UI

What components does a chat application have?
Maybe a main menu, a list of chat channels, a chat history for the currently selected channel and an input field to chat?
With RiotJS, we can start with something simple like this and evolve it easily:

<chatty>
    <chatty-menu/>
    <channel-list/>
    <channel-history>
        <chat-input/>
    </channel-history>
</chatty>

All we need to do is add a browser-side API, put these new custom tags in their own files and knit it all together in index.html

index.html

Our index.html looks deceptively simple:

<head>
... bootstrap, sockjs, jquery, riot ...
    <script src="js/vertxbus.js"></script>
    <script src="js/api.js"></script>
    <script src="tag/chatty.html" type="riot/tag"></script>
    <script src="tag/chatty-menu.html" type="riot/tag"></script>
    <script src="tag/chat-history.html" type="riot/tag"></script>
    <script src="tag/chat-input.html" type="riot/tag"></script>
<body>
<chatty></chatty>
<script>
    var eb = new vertx.EventBus('/chatty/channel-events');
    eb.onopen = function() {
        console.log("Event Bus is ready");
        api.eventBus = eb;
        riot.mount('chatty', {api: api});
    }
</script>
</body>

Ignore the head section for a moment.
There’s the new chatty tag, which we will initialize in a second.

We are setting up an event bus connection to vert.x first. This is the same event bus you used in part 1, all the way extended to the browser (more details about that at the end of this part).

We need to wait until we have a connection established before showing the UI, which is why the rest of the setup is in a callback function. (We’ll not be bothering with AMD or other module loaders for now).

We store a reference to the event bus in our API and then start the RiotJS magic.

The ‘head’ section loads various custom tags as typeriot/tag, one of them being the chatty custom tag.
Calling riot.mount('chatty', { api: api }) tells Riot to compile the contents of tag/chatty.html and insert it into the DOM. We also pass in options (available as opts in the custom tag code) which is our little api object that sits between the components and the server-side.

Here’s tag/chatty.html:

<chatty>
    <chatty-menu api="{opts.api}"></chatty-menu>
    <div class="container-fluid">
        <div class="row">
            <div class="col-md-1" style="height:100%;"><channel-list api={opts.api}></channel-list></div>
            <div class="col-md-11"><chat-history api="{opts.api}"></chat-history></div>
        </div>
    </div>
</chatty>

We added some scaffolding around our new custom tags (and you can probably tell, that I’m not super good at this). Let’s look at them more closely.

(inside chatty.html)

Things get slightly more interesting here, as we define what a channel list is and how we are getting the data.
As you can see, we have the actual template and the code controlling it in the same place, which makes it really easy to maintain and understand.
(We could even omit the script tag and use ES6 syntax if we wanted to)

<channel-list>
    <div style="margin-top:60px; margin-left:8px;">
    <ul class="nav nav-pills">
        <li>CHANNELS</li>
        <li each="{channels}"><a href="#" onclick="{parent.selectChannel}">{name}</a></li>
    </ul>
    </div>
    <script>
        var self = this;
        this.channels = [];

        opts.api.channels(function(ch) {
            self.channels = ch.channels;
            self.update();
        });

        this.selectChannel = function(event) {
            console.log(event.item);
            opts.api.trigger('channel.selected', event.item);
        }

    </script>
</channel-list>

The HTML part looks fairly standard with the exception of new attributes like each and substitutions inside curly braces. RiotJS supports any JS expression inside those.
What we want to do here is to get the list of channels from our server and display them inside an ul tag.

The element that has the each attribute is then repeated for each entry of our list. It also establishes a new context, i.e. the expression {name} refers to the current object in the {channels} list. (This is also the reason the onclick handler needs to refer to the parent context.)

If the list of channels changes, RiotJS will figure out how to re-arrange and change the DOM for us.

The script section has all the logic we need for this component. The opts variable contains any options passed into the component. As part of our design we chose to always have the api object passed in.
We call its channels function and provide a callback to get the list of channels via the REST API we wrote in part 1.
When we receive it, we store the result in our channels variable and update the component.

We also provided an onclick event handler: If a channel is clicked, selectChannel will be called with the channel object in event.item.

Now instead of trying to figure out which parts of the UI need to be updated, we simply send an channel.selected event via the api object with the currently selected channel and hope someone picks up on the other end.

This keeps the component clean of any other dependencies and makes it very simple.
Which is a good guideline: If your component’s code becomes too large, you might need to add a few more sub-components or re-think your component model.

Someone clicked on a channel! What now?

chatty-history.html

Well, in that case we want to switch to the chat history of that channel and show any incoming chat messages.

<chat-history>
    <div class="container" style="height:100%; overflow: scroll; margin-top:60px;">
        <div class="row" each="{chat in chatLines}">
            <div class="col-md-12">
                <system-message if="{chat.type=='system'}" message={chat}></system-message>
                <chat-message if={chat.type=='chat'} message={chat}></chat-message>
            </div>
        </div>
        <div class="row" style="height:40px;" id="empty"></div>
    </div>
    <chat-input api="{opts.api}"></chat-input>
    <script>
        var self = this;
        this.channel = {};
        this.chatLines = [];

        opts.api.on('channel.selected', function(channel){
            console.log("Switch to channel: " + JSON.stringify(channel));
            self.channel = channel;
            opts.api.subscribe(channel);
            self.chatLines.push({alias:'system', type:'system',
                message: '--- ' + self.channel.name + ' ---'});
            self.update();
        });

        opts.api.on('message.received', function(msg, channel) {
            if (channel.address == self.channel.address) {
                self.chatLines.push(msg);
                self.update();
                self.empty.scrollIntoView({
                    behavior: "smooth",
                    block: "end"
                });

            }
        });
    </script>
</chat-history>

The crucial part of this component is each="{chat in chatLines}".
We store all the chat lines in chatLines and we are inserting custom-tags for each of those. Depending on the type of chat message, we insert <system-message> or <chat-message> (check the if attribute).
Finally, we add a custom tag for users to enter a chat line.

The script part is equally simple: We register for internal events being sent via the api object (which is an Observable) and meet our old friend ‘channel.selected’.
Here we use api.subscribe(..) to register for any event bus messages being sent over this channel.

Lastly we register for any messages being sent from the API. If we receive one, we just stuff it into the chatLines array, let RiotJS update our DOM and then scroll the new line into view.

self.empty is automatically set to the DOM element with that ID. No need to look up elements via jQuery.

Custom tags like <chat-message> take care of the actual layout of the chat message. We are using bootstrap’s media classes for this.

chat-input.html

Time to actually provide an input field to type a message!

<chat-input>

    <footer class="footer navbar-fixed-bottom" style="left: 81px;">
    <form class="form-inline" action="#" style="margin:4px;">
        <div class="form-group">
            <div class="input-group">
                <div class="input-group-addon"><i class="glyphicon glyphicon-chevron-right"></i></div>
                <textarea disabled="{channel == null}" type="text" class="form-control" rows="1" cols="60" id="chatLine" placeholder="{channel ? 'Get chatty':'Select channel'}" onkeydown="{keydown}"></textarea>
                <!--<div class="input-group-addon"><i class=""</i></div>-->
            </div>
        </div>
    </form>
    </footer>

    <script>
        var self = this;
        self.channel = null;
        opts.api.on('channel.selected', function(channel){
            self.channel = channel;
            self.update();
            self.chatLine.focus();
        });

        this.keydown = function(event) {
            if (event.keyCode == 13) {
                if (event.shiftKey) {
                    self.chatLine.rows++;
                    return true;
                } else {
                    var line = self.chatLine.value.replace(/$/mg,'<br>');
                    var chat = {type: 'chat', message: line};
                    opts.api.sendChat(chat, self.channel);
                    self.chatLine.value = "";
                    self.chatLine.rows = 1;
                    return false;
                }
            }
            return true;
        }
    </script>
</chat-input>

The interesting parts are the attributes on the textarea tag and the component’s code. We have a few short JavaScript expressions in there.
If those get too long, we would simply add appropriate functions to the script part.

We register for the channel.selected event to focus on the textarea and we register a handler to receive keystrokes. We want the input field to grow by a line whenever shift+enter is pressed.
Note that in event handlers, RiotJS will call update() automatically.

chat-menu.html

We’ll skip that for now since we don’t have any proper user management yet. In the demo app, just fill in an e-mail address which will be used as chat alias (and avatar).

api.js

There is no vert.x or backend specific stuff in any of the UI components. All components consider the api object as the only facade into the dirty world of network communication and caching etc.

If we decide to replace the backend, we could do so if we stick to the interface defined by the api object.

api = (function() {
    var api = riot.observable();
    api.subscriptions = {};
    api.useAlias = function(alias)  {
        this.alias = alias;
    };

    api.channels = function(callback) {
        $.getJSON("/chatty/channels", callback);
    };

    api.subscribe = function(channel) {
        if (!api.subscriptions[channel.address]) {
            api.subscriptions[channel.address] = function (msg) {
                api.trigger('message.received', msg, channel);
            };
            this.eventBus.registerHandler(channel.address, api.subscriptions[channel.address]);
            console.log("Subscribed to " + channel);
        }
    };

    api.sendChat = function(message, channel) {
        message.alias = this.alias;
        message.avatar = this.avatar;
        this.eventBus.publish(channel.address, message);
    };


    // helper functions


    return api;
})();

(We don’t mess with modules yet, so excuse the global variable)
Our actual api object is an observable, which means it has on and trigger functions which are used to communicate with the UI components as seen.

Here we finally have some Vert.x specific code.
this.eventBus is the gateway to the server-side event bus and we use it to register for channel addresses (map subscriptions ensures we only register once).

Whenever a message from Vert.x is received, we simply broadcast it via the RiotJS event system to interested parties like <chat-history>.

We could provide our own chat message cache here, if we wanted to. Maybe synchronize it with the server-side when coming out of offline mode? Ahh, dreams for later tutorials.

sendChat is where we are actually sending off a chat message. We use broadcast here, which means all interested listeners will receive the message, i.e. each user of chatty that has a subscription for the current channel!
Vert.x takes care of the server-side distribution (we’ll see how in a moment).

There’s a few more functions:

  • useAlias just registers the chat alias to use
  • channels uses jQuery to make an AJAX call to our backend to retrieve the list of channels. Note that we also could use an event bus messages for this! Also, we could broadcast to the UI components that a new channel list has arrived. We’ll take care of that later when we can actually add/remove channels

That basically wraps it up for the client-side for now.

We have added a new use case to our app: sending a chat message.
But haven’t changed the backend yet to deal with it.

Vert.x EventBus via SockJS

Meanwhile, most of the actual web server code can be found in ChattyServer.java
I ditched our HelloChatty.js file and moved the REST API code.

We need to add support for sending event bus messages all the way to the browser and back. Vert.x comes with a SockJS server (and client) to accomplish this. SockJS allows us to notify connected clients (and will automatically figure out the best way to do this: short polling, long polling, websocket, etc.)

Let’s add a bridge that connects the internal Vert.x event bus to the outer world:

SockJSHandler sockJSHandler = SockJSHandler.create(vertx);
router.route("/chatty/channel-events/*").handler(sockJSHandler);

Voilá!
With this and vertxbus.js in your web/js directory, we can make SockJS connections and send any message.
And that is a little bit of a problem.

The client can’t be trusted.

Exactly. The client code can’t be trusted and you don’t want client-side code to be able to send and receive any message.
That’s why we can add filters to the bridge that allows connected clients to only send and receive specific messages.

BridgeOptions options = new BridgeOptions();                                          
options.addOutboundPermitted(new PermittedOptions().setAddressRegex("channel\\..*")); 
options.addInboundPermitted(new PermittedOptions().setAddressRegex("channel\\..*"));  
sockJSHandler.bridge(options);                                                        

Now, only addresses that start with channel. are allowed on the event bus bridge.

Serving Static Content

Let’s have Vert.x also serve all of our juciy UI code. For this, we are adding a static handler that is part of the vertx-web module.

StaticHandler web = StaticHandler.create("web").setFilesReadOnly(true).setDirectoryListing(false);
router.route("/chatty/app/*").handler(web);

This will serve the contents of our web directory as static file.

Let’s give it a go. Start ChattyServer
vertx run us.monoid.vertx.chatty.ChattyServer

and point your browser to:
http://localhost:8888/chatty/app/index.html

Pick an alias (if you use an e-mail address, we’ll get an avatar from gravatar) in the main menu, select a channel and chat away.
Open a second tab to the same address and chat a bit with yourself.
It’s therapeutic.
It will help you understand why you keep reading overly long tutorials.

Deployment

We’ll be using Heroku to deploy our little chat app.
If you haven’t done so yet, sign up for Heroku and install the Heroku toolbelt.
Heroku is free for our little test instance and it automates a lot of things for us.

Building a stand-alone JAR file

We’ll be using the lovely maven-shade plugin to package our app, along with the web pages and the kitchen-sink into a single JAR file.

Check the pom.xml plugin section for <artifactId>maven-shade-plugin</artifactId>

We need to configure it with a manifest entry, so it knows how to deploy our app:

<manifestEntries>
 <Main-Class>io.vertx.core.Starter</Main-Class>
 <Main-Verticle>us.monoid.vertx.chatty.ChattyServer</Main-Verticle>
</manifestEntries>

The shade plugin adds itself to the standard mvn package goal, so if we run mvn package now, we end up with a ‘fat’ jar. About 4.4MB for our little app. Not too bad!

Now we can run our app like this:

java -jar target/chatty-fat.jar

Neat!

Heroku deployment

We are going to use the Heroku deployment plug-in for deployment.
Since we are only deploying a single JAR file, we tell it to ignore all the rest.

<plugin>
   <groupId>com.heroku.sdk</groupId>
   <artifactId>heroku-maven-plugin</artifactId>
     <version>0.4.4</version>
     <configuration>
        <processTypes>
         <web>java -jar target/${project.artifactId}-fat.jar</web>
        </processTypes>
        <includeTarget>false</includeTarget>
           <includes>
          <include>target/${project.artifactId}-fat.jar</include>
          </includes></configuration></plugin>

One more thing we need to change: Heroku expects your app to use the PORT environment variable for your web application.

We’ll change our start verticle accordingly:

private int port = 8888;
    @Override
    public void start() throws Exception {
        Optional.ofNullable(System.getenv("PORT")).ifPresent(it -> port = Integer.parseInt(it));
        deployVerticles();
        startWebServer();
    }

and in startWebServer()

vertx.createHttpServer().requestHandler(router::accept).listen(port);

Still not quite there: We should add a forward from ‘/’ to the actual homepage of Chatty. There’s no forward() function available yet, so we have to write our own little handler.

router.route("/").handler(rc -> {
  if (rc.request().uri().equals("/")) {
    String uri = rc.request().absoluteURI();
    rc.response().setStatusCode(301).putHeader("Location", uri + "/chatty/app/index.html" ).end();
   } else {
    rc.next();
  }
 });

Next, go to the project root directory and run

heroku login
heroku create

This creates the app on the Heroku servers.
You only need to do this once!

Now, to build and deploy, run this:

mvn clean heroku:deploy

then run

heroku open

to see Chatty in all its (non)-glory!

And that concludes the second part. We still haven’t talked about persistence and there’s so much wrong with this app before it becomes a real chat app,
but we have a start.

Written with StackEdit.

Sunday, August 16, 2015

Write your own Chat service with Vert.x 3.0 - Part 1

Introduction

This is a multi-part tutorial on how to write a chat service with Vert.x 3.0.
Bring some knowledge about Java, JavaScript and maven (or gradle) and tolerance for overly long sentences and let’s have some fun!

What is Vert.x 3 again?

Vert.x is a reactive, actor-based, event-bus driven, asynchronous, polyglot tool-kit for the JVM. (Wow, that’s a mouthful.) Let’s look at what this means in detail:
Event-Bus Driven
At the heart of Vert.x lies an event bus with a very simple address scheme that supports publishing and sending messages, along with replying to messages. Payload is usually JSON or plain-text.
Reactive
Vert.x supports the popular RxJava library that allows you to write observers that react to sequences of events.
Actor-based
Most code in Vert.x is written as a so-called “Verticle”. In its simplest form a Verticle is a piece of single-threaded code that acts on events from the event bus.
Asynchronous
Any I/O in Vert.x is performed asynchronously. Instead of blocking the current thread and waiting for data to arrive, you usually specify a callback-handler that is called when data arrives. You can still run blocking code though (in separate worker threads).
Polyglot
Vert.x’s API is available in several programming languages. Verticles deployed in Java, JavaScript, Groovy, Python or Ruby can seamlessly communicate via the event bus. You are free to use – say – JavaScript to write your REST API and do the heavy lifting in some Java-based Verticles.
You can also think of Vert.x as the ‘multi-threaded Node.js of the Java world’. Kinda.
A few useful links
Vert.x documentation
Vert.x download

Another Chat service? Noooooo.

I understand. But a chat service is a good showcase for the capabilities of Vert.x. One of those being the event bus, which can be extended all the way to the browser.
Let’s quickly summarize some features we would like to have for our initial prototype:
  • Create a chat alias
  • See a list of channels
  • Join or create a chat channel
  • Send a message
  • Leave a chat channel
We’ll cover more features, such as presence and typing indicators, in later parts of this tutorial.
We’ll also be writing some funky HTML later on using RiotJS. Because, why not?
It looks like our list above already has identified some actors we need. For this exercise we’ll turn them into Vert.x actors and see what we can do with them.
We have a User using an alias, we have a Channel and there are Messages exchanged between Users of a Channel.
That seems easy enough.
Let’s get coding!
The source code for this tutorial is on github. Each part will be tagged as a release.

New Project! New Project! New Project!

I’m using Intellij here to set up a new Maven-based project. Feel free to use your favorite IDE.
Create a new Maven project called Chatty, pick any group ID, and add this to your pom.xml:
<properties>
    <vertx-version>3.0.0</vertx-version>
</properties>

<dependencies>
    <dependency>
        <groupId>io.vertx</groupId>
        <artifactId>vertx-core</artifactId>
        <version>${vertx-version}</version>
    </dependency>

    <dependency>
        <groupId>io.vertx</groupId>
        <artifactId>vertx-web</artifactId>
        <version>${vertx-version}</version>
    </dependency>

    <dependency>
        <groupId>io.vertx</groupId>
        <artifactId>vertx-lang-js</artifactId>
        <version>${vertx-version}</version>
    </dependency>

</dependencies>
This will give us the core Vert.x libraries, the freshly baked web libraries for our REST API, and the JavaScript language binding.
We also need to tell Maven that we are living in the future and want to use Java 8 for compilation and where our JavaScript code will be hiding (see <resources> tag)
<build>
<resources>
        <resource><directory>src/main/js</directory></resource>
    </resources>
<pluginManagement>
    <plugins>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.1</version>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
        </plugin>
    </plugins>
</pluginManagement>
</build>
All set.
Let’s give it a spin.
If you are tired of copying and pasting stuff, you can get the code like this:
git clone https://github.com/beders/chatty.git
cd chatty
git checkout part1

Hello Chatty

Let’s start with a simple HTTP service written in JavaScript.
If you haven’t cleverly anticipated it yet, create a js directory in src/main and add a HelloChatty.js file to it. Here’s our first working HTTP server written with Vert.x!
var html = '<link rel="stylesheet" href="https://goo.gl/fUS2lx">'
    + '<h1 style="color:navy;position:relative;top:50%;transform:translateY(-50%);"'
    + 'align="center" class="animated infinite rubberBand">Hello Chatty</h1>';

var Router = require("vertx-web-js/router");
var router = Router.router(vertx);
router.route('/chatty').handler(function (routingContext) {
    routingContext.response().putHeader("Content-Type", "text/html")
        .end(html);
});
vertx.createHttpServer().requestHandler(router.accept).listen(8888);
print('Go to http://localhost:8888/chatty');
You can run this via command line from your project’s root directory (if you have downloaded and unpacked the full version of Vert.x 3.0):
$ ~/Downloads/vert.x-3.0.0/bin/vertx run src/main/js/HelloChatty.js 
Or you can run it from your IDE using io.vertx.core.Starter as main class and adding run HelloChatty.js as arguments. (In IDEA, use an Application configuration)
Go to http://localhost:8888/chatty and admire the bad color choice and obnoxious CSS animation!
Wait! What? A full web-server in 4 lines of code?
Let’s look at the details.
Ignoring the bad HTML in the first few lines, let’s look at this Router thingie more closely:
var Router = require("vertx-web-js/router");
This loads the router module from the web module (a full list is available here)
var router = Router.router(vertx);
The JavaScript environment contains a pre-defined vertx object which is used to access the vertx core classes and functions. Here we create a new router and tell it to use this vertx object. We’ll see how to do this with Java later on.
What is a router anyway?
A router allows you to attach functions to the individual parts of an HTTP request! In the simplest case, each part of an URL can be matched to one or more routes.
A handler function can then be attached to a route to handle the HTTP request.
In short, a router sits between HTTP requests and your code and handles mapping the request method, path, header and parameters to your functions.
In our example, we define a route and attach a handler for the /chatty part of the URL:
router.route('/chatty').handler(function (routingContext) {
    routingContext.response().putHeader("Content-type", "text/html")
        .end(html);
});
This in essence means, whenever any kind of HTTP request is received that starts with /chatty, call the handler function.
The function takes a single parameter routingContext which gives you access to the HTTP request and response.
In our simple example, we are declaring the response to be proper HTML and then use the end(...) to end the response with a piece of not-really-HTML-conformant-garbage-a-browser-will-still-display.
vertx.createHttpServer().requestHandler(router.accept).listen(8888);
Last but not least we set up the actual HttpServer listening to port 8888. The requestHandler function expects a handler for all incoming HTTP requests which our router provides.

Very Verticle!

Let’s write a Java verticle now to illustrate how they are in essence single-threaded actors which use the event bus to communicate.
For a real chat service, we need some function that provides a list of channels and returns it. A perfect excuse for a Verticle.
A Java verticle is simply an instance of a subclass of AbstractVerticle.
Vert.x gives a Verticle a few excellent guarantees:
  • once deployed, its code will always run in the same event-loop thread which makes synchronizing on instance data unnecessary
  • messages will be received in order
But it also demands a few things from your code:
  • don’t block! No Thread.sleep, no synchronous I/O etc. in your Verticle. There are still ways to do it (in worker threads), but we usually can just rely on the Vert.x API
  • don’t call other verticles directly (use the event bus instead). This is important for scalability issues as the event bus will find an available verticle for you (if you deployed more than one of the same kind)

Channel Manager - does almost nothing

That said, let’s look at our Verticle ‘ChannelManager’:
 public class ChannelManager extends AbstractVerticle {
    Map<String, ChannelInfo> channelMap = new HashMap<>();

    @Override
    public void start() throws Exception {
        new ChannelInfo("#general").registerWith(channelMap);
        vertx.eventBus().consumer("channels.list", msg -> {
            msg.reply(new JsonObject().put("channels", new JsonArray(channelMap.values().stream().map(ci -> ci.toJson()).collect(Collectors.toList()))));
        });
        vertx.eventBus().consumer("channels.join", (Message<JsonObject> msg) -> {
            // sanity checks for the values
            ChannelInfo channel = channelMap.computeIfAbsent(msg.body().getString("name"), name -> new ChannelInfo(name));
            msg.reply(channel.toJson());
        });
    }

    static class ChannelInfo {
        String name, address;

        ChannelInfo(String name) {
            this.name = name;
            this.address = "channel." + name; // we could use UUID.getRandom().toString here as well
        }
        JsonObject toJson() {
            return new JsonObject().put("address", address).put("name", name);
        }

        void registerWith(Map<String, ChannelInfo> channelMap) {
            channelMap.put(name, this);
        }
    }
}
We use a simple Map<String,ChannelInfo> to store information about all existing channels.
In start(), stuff is getting more interesting.
First we add a standard channel (so we have something to send).
Then we register with the even bus to consume any messages arriving on address ‘channels.list’. The message handler is a simple lambda expression that gets passed a Message<?> object.
We can use that object to reply to the sender of the message! In this case, we use some stream() magic to assemble a JSON object with the list of channels.

Test, Decker-Unit!

Let’s get fancy and write a unit test for this. We will be using bog-standard JUnit 4 for this:
public class ChannelManagerTest {
    private static Vertx vertx;

    @BeforeClass
    public static void setUp() throws Exception {
        CountDownLatch latch = new CountDownLatch(1);
        vertx = Vertx.vertx();
        vertx.deployVerticle(new ChannelManager(), res -> {
            latch.countDown();
        });
        latch.await();
    }

    @Test
    public void listChannels() throws InterruptedException {
        CountDownLatch latch = new CountDownLatch(1);
        vertx.eventBus().send("channels.list", null, reply -> {
            assertNotNull(reply.result().body());
            System.out.println(reply.result().body());
            latch.countDown();
        });
        if (!latch.await(5, TimeUnit.SECONDS)) {
            fail();
        }
    }
}
@BeforeClass is where we set up our test environment and where we deploy a single instance of the ChannelManager. Note that deployment is asynchronous in nature, we are registering a callback that then counts down on the latch to allow the JUnit thread to finish set up.
We’ll be using this little trick a few more times (There are a few extensions for JUnit out there that deal with the asynch nature of things. We’ll try to keep in simple by relying on a CountDownLatch.
Test method listChannels uses the Vert.x event bus to send an (empty) message to the address bus address ‘“channels.list”’. Luckily, that is exactly the address the ChannelManager is listening on!
Note that listChannels has no idea about the existence of ChannelManager. It just knows that when it sends something to "channels.list", it will get a reply.
We could exchange our implementation of ChannelManager with a Ruby one and as long as we keep the message format the same, we are good to go!
Our test code also has a reply handler with a lambda expression:
It expects to get a reply with a list of channels.
Run the test and hopefully it will complete and give you this output:
{"channels":[{"name":"#general","address":"channel.#general"}]}
Neat!
We have a list of channel objects with a name and an event bus address we will use later.

Listing Channels the traditional way

Getting the list of channels via the event bus works now, but we also want to expose it as a REST resource.
Open HelloChatty.js and add the following on line 5 or so:
vertx.deployVerticle('us.monoid.vertx.chatty.ChannelManager');
This deploys a single instance of the ChannelManager verticle.
And we need to add a simple REST API to get a list of channels.
router.route('GET', '/chatty/channels').handler(function(routingContext) {
    vertx.eventBus().send('channels.list', null, function(reply, err) {
        if (err == null) {
            var channelList = reply.body();
            routingContext.response().putHeader("Content-Type", "application/json").end(JSON.stringify(channelList));
        }
    });
});
If a client does a GET request on /chatty/channels, we will send a message to the event bus, register a function as reply handler and - if we get a valid reply -(which is automatically converted into a JS object), stringify the channel list and send it as payload.
Now, run HelloChatty again.
If your IDE is clever, the classpath is already set up.
If you run vert.x from the command line, try this:
~/Downloads/vert.x-3.0.0/bin/vertx run src/main/js/HelloChatty.js -cp target/classes
Then point your browser to http://localhost:8888/chatty/channels
Working? Not working?
Let me know in the comments.

Sending a chat message

With a list of channels in hand we have enough information to actually chat directly via the event bus (in later parts we might re-visit that decision).
vertx.eventBus().publish(channel.address, {message:'Hello World', alias: 'beders' });
This snippet above will send a message to ALL listeners of a chat channel, which is what we will be using in part 2, where we’ll have a first prototype ready to go that can actually - you know - send chat messages between users.

Tuesday, July 7, 2015

Optional Parameters in Java 8 Lambda expressions

There is no such thing.

Still, using method overloading, polymorphism and default methods we can have something that looks a bit like optional parameters in lambda-expression and avoids instanceof when calling them.

Examples, the lazy way to grok things. Here’s one

Imagine a generic event bus that sends events and an optional header to registered event handlers.

Here’s what we would like to achieve:

bus.register(event -> System.out.println("I gots an event"));
// or!
bus.register((event,header) -> System.out.println("I gots an event w/ header"));

The first line registers an event handler that doesn’t really care about the headers and the second event handler wants it all: event and headers.

Obviously we need to use method overloading and support a functional interface with one parameter in the first case and with two parameters in the second case.

Here’s how Bus.java could look like so far.
(And for now we are ignoring thread-safety etc.)

public class Bus {
    public static class Event {}
    public static class Header {}
    List<Object> handlers = new ArrayList();

    public void register(Consumer<Event> handler) {
        handlers.add(handler);
    }

    public void register(BiConsumer<Event,Header> handler) {
        handlers.add(handler);
    }
...
}

The ‘trick’ here is to use method overloading and since Java 8 comes with a bunch of generic interfaces (Consumer, BiConsumer), we simply use those.

Problem solved!

Almost.

Note that handlers needs to be of type Object (and we don’t want two handler lists either).

Let’s look at the event dispatcher of our hypothetical event bus.

public void dispatch(Event event, Header header) {
        for (Object obj : handlers) {
            if (obj instanceof Consumer) {
                Consumer<Event> c = (Consumer<Event>) obj;
                c.accept(event);
            } else if (obj instanceof BiConsumer) {
                BiConsumer<Event, Header> b = (BiConsumer<Event, Header>) obj;
                b.accept(event, header);
            }
        }
    }

That looks ugly. Do we really have to deal with instanceof and funny looking, error-prone else-if constructs?

There’s no common super-interface of Consumer and BiConsumer and even if there was, it couldn’t have a common method since that would violate the Single Abstract Method rule for lambda expressions.

Default methods to the rescue!

While a functional interface can only have one abstract method, it can have as many default methods as it wants and we’ll use that to make our dispatch method very simple:

public void dispatch(Event event, Header header) {
    handlers.forEach(handler -> handler.dispatch(event, header));
}

Here’s how.
We need to define our own interfaces for the handlers:

// our common super-interface, defining a default method
public static interface BaseHandler {
    default void dispatch(Event evt, Header header) {}
}
// the one-argument handler
public static interface EventHandler extends BaseHandler {
    void handle(Event evt);
    default void dispatch(Event evt, Header header) {
        handle(evt); // ignore header
    }
}
// the two-argument handler
public static interface EventAndHeaderHandler extends   BaseHandler {
    void handle(Event evt, Header header);
    default void dispatch(Event evt, Header header) {
        handle(evt, header); 
    }
}

We delegate the actual call to the handle method to the overridden default method dispatch (which simply ignores the header argument in EventHandler)

Now we can change our handlers list to be of type List<BaseHandler> and fix the register methods.

List<BaseHandler> handlers = new ArrayList();

public void register(EventHandler handler) {
    handlers.add(handler);
}

public void register(EventAndHeaderHandler handler) {
    handlers.add(handler); 
}

The initial code snippet still works as intended and we have hopefully made it a bit easier for our users to use the event bus (and don’t get warnings about un-used parameters, for example).

Cheers!


Here's a gist with the full source code:


Written with StackEdit.

Saturday, June 20, 2015

Nashorn Scripts in Mac OS Yosemite's Notification Center

Or rather: Another use-case for Today Scripts

Users of Yosemite enjoy the rather useful new side-bar called 'Today' which shows a configurable list of widgets.
Here's how you can use a combination of Today-Scripts  and Nashorn to display information.

This little tip is only useful to you if you run Mac OS Yosemite (obviously) have a recent JDK installed.


Install Today Scripts

This very useful swiss knifes of widgets by Sam Rothenberg allows you to run any bash script and display its output (with ANSI-color support!).
Note that you either have to lower your security settings to run Today Script.app or simply right-click and Open the app. Once it is run, you have a new widget available for the 'Today' section.

Write some Nashorn scripts

jjs, the Nashorn script interpreter should be in /usr/bin/jjs on your system. 
I'm putting my scripts in ~/.jjs and here's a demo script that will show a list of currently running java processes with PID and main class. Save it as jps.js




Now let's add that to Today Script.
Create a new script, use /bin/bash and as script use this:

jjs -scripting ~/.jjs/jps.js

Here's a screenshot of the script configuration.





And that is all there is to it.
Pretty simple, but really the shortest way of running some juicy JavaScript/Java combo in your Notification Center.

Saturday, September 27, 2014

Swift: Fun with operators and delegation (updated)

Apple's new programming language has a quite extensive bag of tricks to make you more productive.
One of the more controversial ones is the ability to define custom operators.

Apple's official guide has a short overview of custom operators and is recommended reading.

Let's look at an example of how operators can be useful (misused?) in the context of iOS programming.

How would you read this?

pickerView --> myDelegate

Read on to confirm your hypothesis.

Pick me up


It is quite common to setup your view controller as delegate as well as data source for components like UITableView, UIPickerView, UICollectionView etc.

In Storyboard you simply drag the delegate/datasource connector to your view controller and you are done.
This is all fine and dandy, but becomes smelly if you are using two or more views of the same type,
for example, two UIPickerViews.

Looking at the solutions that were proposed on StackOverflow, a common trick is to either use tags or compare outlets in the common implementation of the delegate or data source methods like this:

// returns the number of 'columns' to display.
    func numberOfComponentsInPickerView(pickerView: UIPickerView) -> Int {
        if (pickerView == picker1) {
             return 4
        } else if (pickerView == picker2) {
             return 1
        }
// or using switch
//        switch pickerView {
//            case timer: return 4
//            default: // you need a default case
//                return 1
//        }
    }
    

Might get ugly fast. Wouldn't it be nice to have two delegates/data sources. One for each view, that is still somehow part of your view controller? Also, wouldn't it be nice to easily move this logic to a different view controller, if necessary?

I didn't find an elegant way to solve this in Storyboard.
(UPDATE: Someone pointed out I can add any kind of object to a Storyboard and wire those as delegate/data source. That's cool! I still would like to have my view controller code in one place and can't use inner classes in Storyboard)

Also, there's no common protocol implemented by UIViews that indicate they have delegate/datasource properties to set.

Here's a solution that uses inner classes to act as delegates/data sources as well as a new operator that connects views with their delegates/data sources just for funsies.

First the new custom operator which takes care of setting up the delegation chain.

import UiKit

infix operator --> { associativity left }
func --> (left: NSObject, right: NSObject) {
    if (left.respondsToSelector("delegate")) {
        left.setValue(right, forKey: "delegate")
    }
    if (left.respondsToSelector("dataSource")) {
        left.setValue(right, forKey: "dataSource")
    }
}

This creates a new infix operator '-->' which takes care of delegation for you.
It uses key-value coding offered by NSObject to determine if the left hand side of the operator supports a delegate and/or data source and then assigns the right hand side accordingly.
The operator is defined with left associativity and the default precedence value (100).

Note that the current version of Swift doesn't offer much in terms of reflection or metadata programming. Luckily NSObject offers the necessary machinery behind the scenes for us. Also note how selector strings are implicitly converted to Selector objects (since Selector implements StringLiteralConvertible)!

Next up is a delegate/data source for UIPickerView. Note that it has to inherit from NSObject.

    class PickerDelegate : NSObject, UIPickerViewDelegate, UIPickerViewDataSource {
        func numberOfComponentsInPickerView(pickerView: UIPickerView) -> Int {
            return 1
        }
        func pickerView(pickerView: UIPickerView, numberOfRowsInComponent component: Int) -> Int {
            return 10
        }
        func pickerView(pickerView: UIPickerView, titleForRow row: Int, forComponent component: Int) -> String! {
            return "Pick me \(row)"
        }
    }

Now consider adding one or more of those as inner classes to your UIViewController.
In order to wire everything together, two things are missing:
  1. Creating an instance of the delegate
  2. Setting up the delegate/data source chain in viewDidLoad
...
   @IBOutlet weak var picker: UIPickerView! // hook this up via Storyboard
   // 1
   let pickerDelegate = PickerDelegate()

   //2
   override func viewDidLoad() {
        super.viewDidLoad()
        picker --> pickerDelegate   
    }
    ...
Note that if you don't store a reference to the delegate in your view controller, the garbage collector might throw it away since UIViews are not hanging on to them (the property is marked as unowned(unsafe)). The same is true for the datasource property.

I also attached a complete storyboard that shows the general idea.

Advantages

  • Clear separation of concerns. If two or more views of the same type are associated with your ViewController, you will enjoy not having to write if-else construct or switch statements
  • Easy to refactor. If you decide to move your views to a different view controller, it's easy to move the delegate. If you have it in its own top-level file, it is even easier (than using inner classes)
  • Expressive syntax that indicates calls going from the UIView to a delegate/data-source

Disadvantages

  • Code in operator is relying on reflection.
  • Not much is gained and regular code is type-safe:
  • picker.delegate = pickerDelegate
       picker.datasource = pickerDelegate
       
  • Custom operators are global. Beware of clashes.

Note that I'm still pretty new to Swift development and especially Xcode 6. Let me know if you've found more elegant solutions.

But, but it isn't type safe! - Protocol composition to the rescue


Yes, key-value programming is not type safe.
And unfortunately, there is no protocol defined for UIViews that have a delegate and or datasource.

In order to make the code of the operator type-safe, we will have to define the --> function for every UIView that supports those patterns.
Here's an example for UIPickerView.

   func --> (left:UIPickerView, right: protocol<UIPickerViewDelegate,UIPickerViewDataSource>) {
       left.delegate = right
       left.dataSource = right
   }

Notice the usage of protocol composition for the right hand side of the operator.


And here's the link to the Playground.

Saturday, January 25, 2014

So you are having trouble with Webstart on Java 7u51?

The Joys of Java Webstart and Java 7u51

Well, it seemed like a great idea back in 2004 to adopt Java Webstart for our enterprise solution.
10 years later, we are still using it and Oracle has done it's best to make sure we have to make substantial changes to our desktop app to comply with the latest security baseline.

Starting with Java 7u45 and maximizing the pain with 7u51, there are new restrictions for what Oracle calls RIAs (applets and Webstart applications), that will affect you if you are in a similar situation, especially if your customers are setting the Java security settings in the Java Control panel to High, Very High, Ultra High or Insanely High I Won't Run Anything That Starts With CA FE BA BE.

Here are the stumbling blocks you will encounter and how to solve them.

#0 I don't wanna change my stuff


In this case, good luck convincing the IT department of your Fortune 100 customer to use Exception Site Lists (which may or may not work. In our testing you had to clear out the Webstart cache for every start) or Deployment Rule Sets (yeah, good luck with telling their IT to copy a signed JAR file into  c:\Windows\Sun\Java\Deployment for each desktop).
Deployment Rule Sets do work, but someone needs to pay for a proper certificate for the DeploymentRuleSet.jar signature. Which brings us to ...

#1 You need a proper code-signing certificate

Gone are the times where an innocent self-signed certificate will get you access to the user's local machine. You will have to cough up the cash to have your certificate signed.

From the dozens of vendors our there, chose one that has a root certificate in the JRE's cacerts keystore.

keytool -v -list -keystore /Library/Java/Home/lib/security/cacerts

Here's a tool to get you started that will create a new keystore as well as a certificate signing request (CSR) to send to the code signer.

Note that I don't endorse any of those vendors. They are greedy bastards, making you pay on a yearly basis for a one-time activity. Where else is that ok? Right, if you have children :)

I would also strongly advise to get on the phone with the code signing authority. The information you provided in the Common Name and Company Name field is likely not correct and if you change that information you need to create another CSR. You can save a lot of hassle by getting this out of the way before you submit your CSR.

If you do have to change the information in your key, some vendors offer to re-key your key quickly. If you did everything correct, importing the signed certificate into your keystore should not throw an error message.

Damn. I get an error message running keytool.

keytool -import -trustcacerts -alias server -file your_domain_name.p7b -keystore your_site_name.jks 
 
Well, check the certificate in the file provided with the one in the keystore. The easiest way to do this is to use Keystore Explorer, an excellent little tool that saved me from losing my sanity.
You might have to re-issue a CSR or you might have to insist on a SHA-1 key from the signer.

If all goes well, you have a nice new keystore with a certificate that you should use to sign all JAR files referenced by your JNLP file.

Oh no, you have Bouncy Castle JARs in there that are already signed?
No worries, sign them with your signature as well. The trick how to do it is here!

#2 You need a main JAR and new entries in your META-INF/MANIFEST.MF


Declare one of your JAR files as main in the JNLP file:

<jar href="lib/yourMain.jar" main="true"/>

In yourMain.jar, add a Permissions entry:

Permissions: all-permissions

You can add a Codebase entry as well, but often times the codebase isn't known in advance.
It's ok. You can specify it in the JNLP file:

<jnlp ...  codebase="http://enterprisesoftwareforzewin.com">

Luckily you don't need to add the Permission header to all of your JARs, just the main one.

#3 You need to say good-bye to your old property entries in the JNLP file

This one took the most doing. In its wisdom, Oracle decided that most of the system properties you use are not OK anymore.

Now have fun finding those pieces in your code which read those properties.
Some properties are considered safe, but you'll probably have to rename all your properties and prefix them with either jnlp. or javaws. :

<property name="jnlp.var.dir" value="${user.home}/.mylittleconfigdirectory"/>

That is a particular joy if you use 3rd party libraries that don't expect that.
Like log4j. If you used <property name="log4j.configuration" value="..../log.cfg"/> in your JNLP file, say hello to configuring Log4j in your main method manually!
Especially painful if your log.cfg referenced other system properties.

Here's what I had to do:
String url = System.getProperty("jnlp.log4j.configuration");
      if (url != null) {
          try {
              Properties p = new Properties();
              URL logURL = new URL(url);
              InputStream in = logURL.openStream();
              p.load(in);
              // replace log.dir with jnlp.log.dir
              p.setProperty("log.dir", System.getProperty("jnlp.log.dir"));
              in.close();
               PropertyConfigurator.configure(p);
            } catch (Exception e) {
                System.err.println("Unable to configure log4j with "+ url);
                e.printStackTrace();
            }

Haven't touched the code of our Launcher class since 2004. Joy! Code Archeology!

Trick: If you are unsure what properties are passed to your app, use jconsole to connect to the launcher process (you can do it while it shows you a dialog telling you how bad your app sucks and that it can't run it).

#4 You are done. Nope, you are not. Signing the JNLP file (is optional)


There's much confusion about this.
Here's what will happen if you run your Webstart app now without JNLP signing.
Your users will get a warning message with a nice 'Run' button and your app will start nevertheless.
That happens with Medium, High and Very High security settings.



If you get other error messages, you screwed something up earlier.
Set the security setting to Medium which gives you slightly more informative error messages.

Signing your JNLP file is easy if your JNLP file never changes.
This link describes the process. There is some wiggle room with attributes like codebase in which case you want to use an APPLICATION_TEMPLATE.JNLP template.
Put this stuff in your main JAR file in the right directory (and sign it of course).

If your JNLP file is created dynamically with different resources, you are out of luck buddy. And I'm too, because we use the horrors of JSP to create a JNLP file just for you.

In that case, consider creating a new single main class that starts your usual main class, and just use this class along with the APPLICATION_TEMPLATE.JNLP to start your app. You can probably dynamically create that JAR file and sign it on the fly. (Probably not a good idea to have your keystore file on your webserver though)
You could even try to use an <extension> tag with the rest of your app's resources.

#5 Test and test some more with the different JREs


Goes without saying. You might even have to tell your users to delete their temporary files in the Java Control panel.


Since I'm feeling all 2004ish now, I'll ask you all to share this on MySpace.