Sunday, April 22, 2012

Gatling and Play2.0: continued

This blog entry is a follow-up of this entry where I introduced a spike I did on Play 2.0 stress tested using Gatling-tool.

At the time writing the above entry, I had to quickly hack gatling to use Akka 2.0 as Play 2.0 uses it, and I didn't wanted to have clashes.

But, thanks to Stéphane Landelle, Gatling is now Akka 2.0 enabled (since 1.1.2).

So that, it was time to give the plugin's embryo a refresh. For that I used another project that aims testing the neo4j rest plugin for Play2.0 I'm also writing. In case of, I've also introduced what I did in this project in this post.

Content

Using the Gatling I first tested how I've been able to stress test:

  • simple (get) urls
  • mutating (post) urls using server form underground to compile excepted data.
  • duration tests (stress testing on a given period basis)
What I liked so much, even in this embryos+, is the ease to create stress tests when coupled with Play2.0 functionalities and Gatling's DSL.

Foreword

The Gatling plugin I'm currently building is located on github here and is based on sbt to build it.

But it's based on the version 1.1.4-SNAPSHOT version of Gatling's libraries (due to some fixes the Stéphane did "for me", while he was at devoxx fr, isn't he gentle !!!). 
At the time writing, you'll have to build gatling and gatling-highcharts locally using maven (quick fast!).

How to

Set up

Having created a Play 2.0 app, you now have to powerful of sbt in hands (especially if you've installed the TypeSafe stack 2.0). So, to stress test your app, you'll have first to build to plugin I told above.

Plugin

First of all, clone/fork this library project on github plugin, after what, you'll just have to run sbt publish-local in the related folder. That's it, you now have the plugin in your local repo.

Project

In your Play 2.0 app, you now are able to define the plugin as `test` dependency, using the following
That's it...

Use

Specs

Personally, for testing I use org.specs2.Specification, for using it with the plugin (at the time writing) you'll have to create the following:
This listing is creating a fake server to enable urls to be tested, and some functions to deal with it.

To create the server, the plugin defines a Util object (be.nextlab.play.gatling.Util), which also defines rough helpers to be used in stress tests.

A full spec should look the following:

Simple Url

You saw, in the provided gist above, that Util also defines a way to simply defines a gatling Simulation (basically a Function0 that returns a scenario builder: Gatling DSL result).

Having that in hand, here is the fragment to stress test the root url:

As you can see, it's pretty simple, but nothing can really be checked in the body of the specs (I'm working on having relevant information to check).
But, at least you can run this test to hit 10 times the root url ramping the numbers of users by 2.

Running it (sbt test) you'll have a new gatling folder in your target folder that contains a  results directory where are located all stress results in an html report (with great charts)

And all you had to do is to define the request headers and the url...

Mutating the server

If you have controllers that mutates the server, you should have define POST urls, which are using the Form feature provided by Play 2.0.

Having did so, you'll be able to stress it very easily using Map, JsObject or in the best case your Model.

Let's say we have a controller controllers.Stuffs that uses case classes models.Stuff. The controller defines a stuffForm and a createStuff action.

Your stress test can now be defined like the following:

In the gist, you can see 5 points to note, they are key-clues to create reusable stress tests.

Nothing is really hardcoded, neither the path to the http end point nor the parameters data.

That's http stress testing using type checked requests. CoOl isn't it? Hey, man, we got back our lovin' type cheking (one of the best scala feature).

Heavy check, duration based

This part is more a Gatling feature highlighting.

This last example is an heavy test that uses looping over a configuration for a given period. This gives you how many users could use your application.

Such test might be shaped the following:

Note

For now, it's NOT an official plugin neither a gatling nor a play 2.0 one, but discussions are on the way for that... stay tuned on twitter or here.

Wednesday, April 18, 2012

Still Playing... but new players are in

Why?

Because I love to play, with Play 2.0 and scala (still learning).
But also because I'm currently investigating technologies that I might choose for a new product line currently building in my company, NextLab (in Belgium).

With ?

This time, I've more played with client side libraries or frameworks (no I won't post yet another entry on JQuery...), but I also tried how it is easy to create totally async code (so parallelized) using server side ones.
The technologies that will be quickly introduced in this post can be found hereafter, but everything has been packaged in a github repo, and a running instance on Heroku.

At first the current spike was dedicated to the slowly growing Play 2.0's Neo4J REST plugin, we are creating at NextLab. But, in order to demonstrate what could be done with the coupling of these two technologies, I've extended the spike's scope to something more funny.

So let's introduce the technologies:

Client Side

Twitter Bootstrap
An amazing toolbelt helping building responsive website without having to bother boilerplates in HTML, CSS.

Even if it is neat, complete and well thought, Bootstrap comes with another handy factor... it has been built using LESS. And by chance (I know, chance is not part of the equation) LESS is supported by Play 2.0 by default.

Just a note, LESS will let you reuse color codes, mixins, etc etc that Bootstrap has already defined.
Spine.js
As we'll below, we'll have to discuss with WS (json), upon which a REST interface has been added for meaningful resources.

That's where Spine.js comes in the game. This lightweight MVC library brought me the small tools that I needed for fetching, saving resources without having to write not even a single request by hand...
d3.js
Probably my favorite (that might be my mathematician part who's talking), this powerful Data-Drive Document toolkit has taken the right thing by the right end.

Its functional approach of decoupling data from the document, and link them using layouts helps you to concentrate on each part of the data usage independently:
  • the incoming/rought data mapping to a representative data
  • the mapping from represented data to document (most of the time, one data for one element)

Communication

The communication layer is of course HTTP, with a little help from the emerging HTML5 features. One in particular, Server-Sent Events (here is a great intro).

This, stable, HTML5 feature comes with the handy functionality to let the server sent events to connected client, without hacks; that said Comet or Polling.

Open a connection, push data, and that ONLY when the server needs to.

Server Side

Play 2.0 (Scala)
Of course... But I used some "advanced" feature like,
Async
Async is a Play 2.0 feature that let the server deal with the tasks it has to schedule.

That is, when you think that the server might have to wait for actions being executed before being able to respond a request, Play 2.0 let you, really simply, create Async request (non blocking).

Very handy when you have to call third parties services for instance...
Iteratee
The only way I would consider from now to consume data. Iteratee is a fairly difficult thing to understand (read this wiki) but it gives you the same smart decomposition that in d3.js, that is, decoupling the management of a sources, its useful representation and its computed result.
Akka
Powerful, actor-based, parallelizer, asynchronous task, scheduler library.

Actually I needed, a request to launch async tasks (you know like event generation and dispatching)... so what else!
Neo4J
A database for storing graph... let's use a graph database.

Within NextLab, we started a open sourced Play2.0 plugin for calling the REST api provided by a Neo4J Server (helpful on Heroku). It's still emerging, and continue to evolve a lot because features are implemented on the fly (need), and a re-pass is forecasted to add a meaningful DSL (like FaKod did).

Why do we make this choice to implement a distinct plugin instead of wrapping the java library?
  1. yet another library, which brings me too much (I need REST only)
  2. I want requests being async and under control

Libraries Repo

See below, we'll use Heroku. So in order to deploy this application wich uses our plugin, I needed to publish somewhere.

This is what can offer Cloudbees. Among other great things like git repo, CI and so on, Cloudbees provides you for free four maven repositories that you can make public if you wish.

So I used sbt to publish on my "snapshot" repo on cloudbees, letting heroky has access to it for downloading the plugin dependency.

Platform

Free, reliable, easy to use with scala and Play 2.0... which else than Heroku?

So what

I think that this post is already too long... However, I can let you play with the resulting app here.

Check out the code (and fork it) there.

Depending on the comment, I will expand this post to other ones to respond potential demands (if there is any ^^).

Good play.

Oh yes, one last note, the application is to Create Stuff which contains dummies. Stuff are created using a simple form that must be fulfilled. Stuffs can be linked one to another by clicking the graph.

Please create Stufs and links, it will be a good test for Neo4J, the plugin and Play 2.0.

Hope you've reach this.

SMAK. hehe