Matthew O'Riordan

MaxCDN review - seriously unreliable

URL regular expression for links with or without the protocol

I’ve just come across a pretty common requirement to convert any text that looks like a link into a link within some HTML text.  Strangely, after searching for a good 15 minutes for a regular expression, all I could find was either a regular expressions which detects URLs with a protocol such as http://mattheworiordan.com/, or a regular expression which detects URLs without such as www.mattheworiordan.com.  Why the hell I could not find one which does both is beyond me, so here I go at posting a solution for anyone else to use.

Here is the holy grail: 

/((([A-Za-z]{3,9}:(?:\/\/)?)(?:[-;:&=\+\$,\w]+@)?[A-Za-z0-9.-]+|(?:www.|[-;:&=\+\$,\w]+@)[A-Za-z0-9.-]+)((?:\/[\+~%\/.\w-_]*)?\??(?:[-\+=&;%@.\w_]*)#?(?:[.\!\/\\w]*))?)/

Here is a nice example of this regular expression in action http://jsbin.com/eqocuh/5/edit#source

Please feel free to modify this JSBin, add examples, and update this regular expression, and I will update within this blog post.

Here is an explanation of the regular expression for those who care

(
 ( # brackets covering match for protocol (optional) and domain
  ([A-Za-z]{3,9}:(?:\/\/)?)   # match protocol, allow in format http:// or mailto:
  (?:[\-;:&=\+\$,\w]+@)?   # allow something@ for email addresses
  [A-Za-z0-9\.\-]+   # anything looking at all like a domain, non-unicode domains
  | # or instead of above
  (?:www\.|[\-;:&=\+\$,\w]+@) # starting with something@ or www.
  [A-Za-z0-9\.\-]+   # anything looking at all like a domain
 )
 ( # brackets covering match for path, query string and anchor
  (?:\/[\+~%\/\.\w\-]*)  # allow optional /path
  ?\??(?:[\-\+=&;%@\.\w]*)  # allow optional query string starting with ? 
  #?(?:[\.\!\/\\\w]*) # allow optional anchor #anchor 
 )? # make URL suffix optional
)

Infinite scrolling tab system for JQuery

Google Docs Spreadsheets has a useful worksheet tab system whereby if there are more tabs than the space available, a scroller device automatically appears.  Whilst building the new sprints functionality for my backlog management tool easyBacklog, I went on a search for a scrolling tab system and couldn’t find one.

So I’ve built a JQuery plugin inspired by the Google Docs Spreadsheet tab system, along with the rounded tabs solution by Chris Coyier.

Check out the plugin at https://github.com/mattheworiordan/jquery.infinite.tabs

Comments / feature requests / bug reports welcome.

Setting up a Jenkins (old Hudson) Continuous Integration Server

Following on from my recent full stack integration work on easyBacklog, the Agile Project Management app I am building, I have set up a Continuous Integration server which will run both unit tests and integration tests across the entire app each time I push a commit.  In order to do this, I needed full support for Selenium and Capybara-Webkit, along with Cucumber and RSpec.

I looked at various CI solutions:

• TeamCity - looked really good, but very .NET and Java focussed so that put me off
• Integrity - looks like it’s no longer maintained
• CruiseControl.rb - not much documentation for it, and not a huge amount of community interest it seems
• Jenkins - split from Hudson when they became commercial, and seems to be the big daddy and very well supported in the community.

So I opted for Jenkins and went about setting up a CI server on my Mac Mini with OS X Lion.

As I couldn’t find any easy guides on what to do, I loosely documented the steps I followed to get my CI server up and running so that hopefully others trying to do the same thing will have a good starting point.  Here is a simple chronological log of what I did:

Note: I am pulling my working directory from a local Git repository.  If you want to use Github, then install the Github plugin (see below) and figure that part out yourself!

Log into your Mac with your admin account

Install Homebrew: /usr/bin/ruby -e “$(curl -fsSL https://raw.github.com/gist/323731)”

If you have Homebrew installed already, then I would advise you `brew update` to get the latest recipes

Install Qt (needed by Capybara-Webkit): brew install qt —build-from-source
Note, this will take over an hour to build, so make yourself a nice cuppa and wait.
Whilst this is happening, I would go ahead and install Firefox which is needed for the standard Selenium install. 

Install Jenkins using the native installer, http://mirrors.jenkins-ci.org/

Server should be running, if not, reboot the machine as on Mac OS X a LaunchDaemon is installed by Jenkins.

Go to http://localhost:8080, Manage, Manage Plugins, and install Jenkins Git, Hudson Ruby and Jenkins ruby metrics

I would also advise you set up security for your Jenkins server.  Go to Manage Jenkins, Configure System, choose “Jenkins’s own user database” under “Access Control” and the rest is reasonably self-explanatory.

If you want email notifications when builds succeed or fail, then you will either need an SMTP server you can use, or you can configure OS X to run postfix.  See the following articles to get postfix running on Mac OS X:

• http://www.phase2technology.com/node/667/
• https://discussions.apple.com/thread/3247974?start=0&tstart=0
• http://serverfault.com/questions/42519/how-to-correct-postfix-relay-access-denied-problem

Set up a user called jenkins in OSX (login jenkins).

Modify the file /Library/LaunchDaemon/org.jenkins-ci.plist, remove the GroupName and daemon value and then change UserName to jenkins

You will now need to change the permissions on a folder Jenkins uses, run the following:
sudo chown -R jenkins:wheel /Users/Shared/Jenkins

Now reboot, and make sure Jenkins is still up and running (http://localhost:8080)

Log into your OS X box with the jenkins account

Install RVM: bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

Install ruby: rvm install 1.9.2

Ensure 1.9.2 is default: rvm —default install 1.9.2

Reload your console, and type rvm list.  Ruby 1.9.2 should be selected.  If not, your default Ruby 1.9.2 is not working.

Now install the Bundler gem, `gem install bundler`

Now before you set up your first job in Jenkins, I would advise that you make sure the jenkins user account has everything needed to actually run Rake and RSpec.  So to do this, simply go to the console, create a working folder and git clone git://your-git-respository-url.git to get a working version of the files onto the server.  
Run `bundle install && RAILS_ENV=test bundle exec rake db:migrate && RAILS_ENV=test bundle exec rake`.  Your Cucumber tests (and Unit tests if you have some) should run.  If they don’t pass, figure out why not before you get Jenkins to start automatically building for you.

To set up your CI job, go to Jenkins home page (http://localhost:8080), click New Job, and set the following:

• Name the job, and select “Build a free-style software project”, then click Next
• I opted for “Discard Old Builds” and kept the last 30 builds
• I chose Git and entered the URL (local path) to my Git repository.  This could of course be a public Git repository, but make sure when you are logged in as jenkins on your Mac that the repository is accessible.  Set up public key / private key authentication if necessary.
• Under build triggers, choose Poll SCM.  I entered * * * * which means I poll the SCM every minute.  If this is a remote repository, then I would suggest you do something like 0,15,30,45 * * * to poll every 15 minutes.
• Add a build step, “Execute Shell” and use something like this:

#!/bin/bash -eexport RAILS_ENV=test
source “$HOME/.rvm/scripts/rvm”
rvm use 1.9.2
bundle install —deployment
bundle exec rake db:migrate spec

• Under email notifications I would put in your email address for notifications.
• I have not yet looked into the Rails / Ruby specific plugins and how to use them, so for now leave these blank.
• Save your job

You can then manually start the build by going to the home page, clicking on the title of your build, and clicking Build Now.  You should see the status of the build in the Build History (bottom left), and if you click on the Build in progress, and then on Console Output, you can monitor the build as it happens.

Now that you have SCM polling, each time you push a change to your Git repository, a new build should automatically fire off.  

Do give me feedback if I’ve missed out any steps.  Unfortunately I wrote down a lot of these steps retrospectively.

A library for simulating a drag event with a JQuery UI Sortable widget

I have been working on thorough integration testing of my new Agile Project Management tool called easyBacklog, and came across a problem whereby it seems there is no easy way to test / simulate drag and drop events with a JQuery UI Sortable widget.

Whilst jquery.simulate.js provides functionality to simulate many JQuery and JQuery UI event, it is unable to simulate a drag event for a JQuery UI Sortable widget due to the intricate behaviour needed to make the JQuery UI Sortable widget fire the correct events.

So I’ve developed a library to solve this problem: it allows you to simulate Drag events for a JQuery UI Sortable, and supports a number of useful features.

Go to the Github repository jquery.simulate.drag-sortable.js for the code and the tests.

Testing :focus with JQuery and Selenium or Capybara-Webkit

Recently whilst trying to write integration tests for a JQuery heavy front end application, I came across a very strange issue when testing to see if an element has focus.

What I found is that if an element has focus, yet the actual browser or tab does not have focus, then using the pseudo selector :focus from JQuery (or Sizzle) will not find the focussed item.  Putting it another way, the browser and web page must have the focus in the OS for the :focus selector to work.  

A simple way to replicate this issue is to go to http://jsbin.com/abozas, open up your Javascript console, and type the following commands

$('#inpt').focus();
$('#inpt').is(':focus');

As you have just put the focus on #inpt, you would expect it to retain focus and thus pass a JQuery test of .is(‘:focus’);  However, because the Javascript console has the focus, and not the web page, .is(‘:focus’) will always return false.

Now generally this is not an issue as your web page actually has focus, but unfortunately I discovered that in Rails when using Cucumber, Capaybara and Capybara-Webkit and/or Selenium and testing for :focus, all tests will fail because the browser window does not have the focus of the OS at the time it is run.

So firstly I logged this is an issue with Capybara-Webkit, and in the mean time I’ve worked up a workaround solution by poking around in JQuery and Sizzle (the CSS selector engine that powers not only JQuery, but also Prototype, Dojo, MochiKit, TinyMCE).  I discovered that the issue is in fact not with the Sizzle selector logic, but resides with some optimizations that Sizzle uses for performance reasons.  For the function matchesSelection, it tries to first use the native method webKitMatchesSelector() to match a selector, and if this fails, it falls back to the Sizzle selector logic.  The problem is that the webKitMatchesSelector is OS aware and thus removes focus from an element when the browser does not have focus, whereas Sizzle is unaware of the browser’s focus status and thus matches successfully using the following code:

function( elem ) {
  return elem === elem.ownerDocument.activeElement;
}

So to fix this problem, I have written a small Javascript file which you should include when you are running a test or cucumber Rails environment.  This script will then deactivate the native matches selector and query selector for your browser.

The code to fix the :focus issue can be found at https://gist.github.com/1166821 and simply does the following:

/* Prevent use of native find selector */
document.querySelectorAll = false;

/* Prevent use of native matches selector */
document.documentElement.matchesSelector = false;
document.documentElement.mozMatchesSelector = false;
document.documentElement.webkitMatchesSelector = false;
document.documentElement.msMatchesSelector = false;

If you would like to replicate this issue, review this Gist.  An example set of tests where the issue is resolved can be seen in this Gist.

I hope others find this useful.

Cucumber and Capybara-Webkit automatic screenshots

This post has been superseded by a Gem I have written to do automatic screen shots for Capybara (not just Cucumber, also supports RSpec and Mini-test).  Please go to https://github.com/mattheworiordan/capybara-screenshot

Writing Cucumber front end tests (AJAX & Javascript) can be pretty damned painful and slow, especially when a failure does not come with any useful error message.  Fortunately I’ve been using Capybara-Webkit recently, which brings two very useful features for debugging issues:

• Javascript console messages (console.log) are sent directly to the shell whilst running your tests.  So you can effectively add debugging statements in your Javascript that are sent through to your terminal.  See below.
   
• Capybara allows you to save out PNG screenshots from the current page.  So I’ve written two convenient pieces of code that you may find useful.  

  The first one allows you to write “Then take a snapshot” or “Then take a snapshot and show me the page” within your features, and a PNG and HTML file will be saved into tmp/capyabara.
  
  And insert this into your shared_steps.rb (if you have one) under /features/step_definitions 

  Then /take a snapshot(| and show me the page)/ do |show_me|
    page.driver.render Rails.root.join("tmp/capybara/#{Time.now.strftime('%Y-%m-%d-%H-%M-%S')}.png")
    Then %{show me the page} if show_me.present?
  end
  

  The second one ensures that whenever a scenario fails, a screen shot and HTML version of the page is saved into tmp/capybara automatically.
  
  Place this in your /features/support folder

  def screen_shot_and_save_page
    require 'capybara/util/save_and_open_page'
    path = "/#{Time.now.strftime('%Y-%m-%d-%H-%M-%S')}"
    Capybara.save_page body, "#{path}.html"
    page.driver.render Rails.root.join "#{Capybara.save_and_open_page_path}" "#{path}.png"
  end
  
  begin
    After do |scenario|
      screen_shot_and_save_page if scenario.failed?
    end
  rescue Exception => e
    puts "Snapshots not available for this environment.\n
      Have you got gem 'capybara-webkit' in your Gemfile and have you enabled the javascript driver?"
  end
  

This code is up on a Gist at Github at so feel free to fork away.

Rate limiting Javascript requests

I recently needed to rate limit requests to a particular function in Javascript, and was surprised nothing obvious came up in Google searches so here is my contribution for others who need this.

Note that I am a big fan of Underscore.js and have thus simply extended the _ to include my new rate limiting function.  Also, many people immediately think that _.throttle will achieve rate limiting, but in fact _.throttle is a destructive rate limiting function in that it will disregard all requests that arrive within the time threshold specified.  If therefore you want to ensure that all calls are still executed, but are never executed more than once ever X milliseconds, then I suggest you use my method.

The code is uploaded to a Gist which you can fork at https://gist.github.com/1084831, and is embedded below.

// Rate limit ensures a function is never called more than every [rate]ms
// Unlike underscore's _.throttle function, function calls are queued so that
//   requests are never lost and simply deferred until some other time
//
// Parameters
// * func - function to rate limit
// * rate - minimum time to wait between function calls
// * async - if async is true, we won't wait (rate) for the function to complete
//           before queueing the next request
// 
// Example 
// function showStatus(i) { 
//   console.log(i); 
// } 
// var showStatusRateLimited = _.rateLimit(showStatus, 200); 
// for (var i = 0; i < 10; i++) { 
//   showStatusRateLimited(i); 
// } 
// 
// Dependencies 
// * underscore.js 
// 
_.rateLimit = function(func, rate, async) {
   var queue = [];
   var timeOutRef = false;
   var currentlyEmptyingQueue = false;
   
   var emptyQueue = function() {
     if (queue.length) {
       currentlyEmptyingQueue = true;
       _.delay(function() {
         if (async) {
           _.defer(function() { queue.shift().call(); });
         } else {
           queue.shift().call();
         }
         emptyQueue();
       }, rate);
     } else {
       currentlyEmptyingQueue = false;
     }
   };
   
   return function() {
     // get arguments into an array
     var args = _.map(arguments, function(e) { return e; });
     // call apply so we can pass in arguments as parameters as opposed to an array
     queue.push( _.bind.apply(this, [func, this].concat(args)) );
     if (!currentlyEmptyingQueue) { emptyQueue(); }
   };
};

You can see a working version at http://jsbin.com/upadif/8/edit#preview

Full stack integration testing with Rails 3, Cucumber, RSpec, QUnit and Capybara

I find it enlightening to be part of such an active Ruby and RoR community where Gems, code samples and responsive community support is available, however I also find it hard sometimes for that very same reason to figure out what stack to use for my new projects because there are just too many options.

One of the problems I’ve had for a while now is figuring out what stack to use for my testing, both for my back end code (unit & integration) as well as my front end code (unit & integration).  So this blog post is about my journey in April 2011 to find the best full stack integration components for a Rails project and document it for others to use.  I hope others find this useful.

Background

I use RSpec and Cucumber for all my testing, but my application’s front end is becoming increasingly complex and needs testing as well.  I would ideally like a combination of both unit testing and integration testing on the Javascript and front end elements.  And I want my tests to be as fast as possible because the longer it takes the less likely it is I’ll run them.

The objective of this exercise

• Document the entire process collecting all useful resources together in one place so others can learn from this
• Set up a basic Rails 3 project and push it to Github which has enough back end & front end code to write some meaningful tests.  This must include some asynchronous client-server requests.  And a Javascript framework such as jQuery should be used to make the stack and environment realistic.
• Write RSpec & Cucumber tests for traditional unit testing and integration testing.
• Explore options to unit test and integration test the front end code, documenting the pros & cons for each
• Find the best stack for me and put that up on Github for others to use

My journey to find the ideal full stack integration suite for Rails 3 (as of now)

I installed a clean version of Rails 3.0.3 on Ruby 1.9.2.  See commit.

I like to use RSpec for all unit tests, so I installed RSpec and removed all the old Test::Unit files in /test/.  See commit.  After you add the gems to your Gemfile, remember run `bundle` and also to run the RSpec generator `script/rails generate rspec:install`.  For those of you that use scaffolding (I avoid it), I also modified config/application.rb to instruct Rails 3’s generators to always use RSpec rather than Test::Unit. 

My favourite tool for integration testing is Cucumber.  RSpec can be used for integration testing too, and I know there is a current view from DHH himself that both RSpec and Cucumber are unnecessary, however I like Cucumber and the readable user stories are great.  So I went ahead and installed Cucumber that by default now uses Capybara (it used to default to Webrat).  I followed the Cucumber Rails installation instructions, however to ensure Capybara is used for tests as opposed to Webrat I ran the following generator command `rails generate cucumber:install —capybara`.  See commit.

In order to get the project ready for front end development, I cleared out the index.html file, installed a copy of jQuery and RoR JQuery UJS, removed all javascript files and added the UJS Javascript file), and added in a stripped down jQuery UI code base to allow for some more complex user interactions.  See commit.

I then went ahead and installed SASS because writing traditional CSS just sucks.  I normally use HAML as well instead of ERB, but for the purposes of this example I thought it best to keep things as understandable as possible and it’s quite possible HAML is less understood (and once again, DHH does not like it). See commit.

Building the Calculator app and tests

Following good TDD, I then went ahead and wrote the tests for a new Calculator model class that I am about to build.  The calculator is intentionally very simple and only supports addition, subtraction, multiplication and division.  I kept the tests simple and adequate for the purposes of this project.  I then ran RSpec `rspec spec/models/*.rb` and got a load of errors as the Calculator class does not yet exist.  

In order to satisfy the tests, I built a simple Calculator class that lives in the /app/models/ directory.  I ran RSpec until everything was green, and voila, we now have a test suite for the model which passes.  See commit.

I then built a very simple HTML based calculator and a controller called Calculator to manage all calculator operations.  See commit.  I pushed it up online using Heroku and posted it at http://full-stack-testing.heroku.com/calculator

Once finished, I added some Cucumber integration tests which will run using Capybara’s default driver rack_test as they do not need Javascript support.  Ideally I should have written the features first, but for the purposes of this example I was very much discovering as I built, and had no spec to work from.  Fortunately in most traditional situations I have a spec I can work from and can therefore follow TDD and build out the Cucumber features first.

The first test was extremely simple as follows:

I ran `rake cucumber` and ran into the following issues:

Firstly I received the error:

schema.rb doesn’t exist yet. Run “rake db:migrate” to create it then try again. If you do not intend to use a database, you should instead alter /Users/matthew/Projects/full_stack_integration_testing/config/application.rb to limit the frameworks that will be loaded

To fix that I simply ran rake db:migrate which created an empty schema.rb file.

I then re-ran `rake db:migrate` and ran `rake cucumber` again.  Unfortunately this time I received a rather less intuitive error:

undefined local variable or method `node’ for #<Capybara::Driver::Node tag=”a” path=”/html/body/a”> (NameError)

After a little searching it seems this is a known error.  I found this strange considering I thought I was on the latest stable versions of all gems, and my feature is extremely simple.  However, after looking at a related closed issue in Capybara I see that this issue was corrected, so I clearly don’t have the latest gems.  I had to modify my Gemfile to explicitly request cucumber-rails with a version greater than or equal to 0.4.0 to replace the 0.3.2 version that we was used by default.

I then re-ran `rake cucumber` and unfortunately hit another error as described at http://blog.firsthand.ca/2011/02/cucumber-rails-undefined-local-variable.html

I had to re-run `rails g cucumber:install` to regenerate the cucumber files, and then I was ready to run rake cucumber again.  See commit. 

So I finally ran `rake cucumber` and fortunately my very simple test passed as follows:

So I went on to writing some more expansive Cucumber scenarios to provide better integration testing of the calculator.  And after building only one additional step, I managed to create a Cucumber feature which covered the bulk of the functionality offered including exceptions for the Calculator.  See commit.

Test performance and Spork

So a recurring problem I have is that whilst Cucumber tests report completing quite quickly (in around 2.5s in this case), it took considerably longer than that in reality as the rails environment start up time is very slow.  Whilst I appreciate testing is an absolute necessity, if it’s going to be painfully then that’s going inevitably deter me from testing.  So I have heard about spork on a few of the podcasts I follow, and understand that spork is effectively a library which runs your test suite by forking a new copy of the Rails environment each time you run your tests.  So I went ahead and followed the Spork installation instructions.

Specifically I added the following lines to my Gemfile

I then ran `spork —bootstrap` which modified the file spec/spec_helper.rb and added further installation instructions into that file.  I looked through the instructions and read that out of the box spork works well with rails.  So I left all settings as they were.

I then started up the spork server by simply running `spork` and got the following:

I then ran my first set of RSpec tests with `rspec —drb spec/**/*` and it was damn quick.  Everything was completed in under a second, amazing.

So I then read that to run the cucumber tests using spork all I needed was to run `script/cucumber —drb` so I went ahead and did that but unfortunately was not quite so lucky.  I got the response:

  WARNING: No DRb server is running. Running features locally:

So I did a bit of searching and found an article on using spork with DRB on the cucumber Wiki.  It turns out I need to run the generator again and include spork support.  As I need capybara support too (see above), I ran the following:

rails generate cucumber:install —capybara —spork

It asked to overwrite my paths file, so I let it do that and just reinserted the custom path I had added earlier.

I then ran `script/cucumber —drb’ and was told again:

WARNING: No DRb server is running. Running features locally:

I did a bit of searching and discovered that this was because the spork DRB server was actually running for use with RSpec and it needs to be run for use with Cucumber.  So I stopped the spork server and executed the spork server for cucumber as follows `spork cuc`

I then run `script/cucumber —drb’ and as expected the test ran immediately, considerably quicker than before.

As I want to run both Cucumber tests and RSpec tests as fast as possible, I wanted to find out how I can run spork for both cucumber an RSpec simultaneously.  It turns out this is quite simple, all I need to do is run the following command:
`bundle exec spork cucumber & bundle exec spork rspec &`
which gets spork for cucumber and rspec running as background tasks, and on different ports namely 8989 for RSpec and 8990 for Cucumber.  Note you can access these tasks using `jobs`, or `fg [job number]`.  You can also kill these tasks using `kill %[job number]`

Spork for both environments is something I’m going to use often, so I set up a simple script script/spork which runs spork in the background for both RSpec and Cucumber.  To make this an executable I ran the command chmod 777 script/spork in the root.  I ran `script/spork` and both servers were started in the background.

In order to measure performance improvement of the tests using spork, I set up a basic benchmarking script residing in scripts.  The code is extremely simple and runs the rspec tests and cucumber tests without DRB (that’s what spork uses), and then with DRB to see the difference.  I noticed however that cucumber.yml was configured to run —drb as an option automatically, so I removed this and added -c for colour output.  I then ran script/benchmark_tests and got the following results:

• RSpec without spork took: 9.81s
• RSpec with spork (DRB) took: 1.11s
  Spork saved 8.70s when running RSpec tests
• Cucumber without spork took: 13.69s
• Cucumber with spork (DRB) took: 4.31s
  Spork saved 9.38s when running Cucumber tests

So it looks like spork is doing a great job in saving on the start up time of the tests for both Cucumber and RSpec.  Admittedly as my tests increase in size this start up time saving will become less relevant, but any speed improvement is welcome especially when running RSpec unit tests which are typically quite fast.  See commit.

Automated tests and Watchr

And this is all great, but I don’t really like jumping into the console every time I need to run a test.  What I want is to run tests automatically whenever I change the relevant files in my project.  I’ve heard of two apps which achieve this, autotest and watchr.  Watchr seems to be gaining more traction of late, and my previous experience with autotest has not been brilliant and have always had problems getting it running reliably.  So without much further research, I decided to use Watchr quite simply because influential Rubyists have been talking about it recently so I trust their judgement.  So firstly I added watchr gem to my Gemfile and ran `bundle`

I then found a Gist which seems to roughly achieve what I want in that it supports DRB, Cucumber and RSpec.  I saved the Gist source code to script/watchr.rb and I tried running `watchr script/watchr.rb` but unfortunately was told that Growl was not installed.  I added growl to the Gemfile, ran `bundle` and then reran `watchr script/watchr.rb` and it started up as expected.  However, when changing an RSpec file in spec/models the tests did not run as expected, so I made a few changes to watchr.rb.  Please take a look at my Gist to see the list of changes.

So once I made my changes, everything seemed to be working except that the growl notifications did not have an image displaying, and the output from RSpec was not colour coded.  So firstly to address the Growl issue I downloaded some PNG status icons and saved them into /script/.watchr_images.  In order to resolve the ANSI colors being lost for RSpec issue, I tried a number of alternatives using all methods described in the article “5 ways to run commands from ruby“ but unfortunately none of those solutions worked.  So I posted a question on Stackoverflow  and someone helped me find the solution.  In order to have colour show up in RSpec you need to tell it you are outputting to tty so that colour is supported, so I modified the $spec_cmd in watchr.rb to the following:

Now any changes to features will trigger Cucumber, and any changes to models or unit tests will trigger RSpec and confine to the effected files where possible.  See commit.

Front end unit and integration testing

So onto the more difficult parts, front end unit and integration testing.  So I have gone ahead and added some trivial front end code to the calculator which presents a set of buttons for the calculator so users don’t have to use the text field when entering the number used in operations.  I wanted to ensure that my tests had some AJAX functionality too as this is generally needed in my web applications.  So in summary, the code I added provides us with the following to be tested:

• Simple Javascript class methods that can be unit tested i.e. no DOM is required
• DOM manipulation on load to add additional features to the calculator
• AJAX communication with a server and dynamic updating of the DOM including handling of errors
• Prompt dialog box which requires the user to interact with to continue when performing an operation on zero
• Alert dialog box which prevents a user from dividing by zero
• JQuery UI Dialog box which again needs user to interact with to continue
• Drag & drop functionality (you can drag values from the calculator to a pretty pointless memory bank)
• Use of persistent cookies to persist the values stored in the memory bank

I committed all those changes and you can see the calculator live at http://full-stack-testing.heroku.com/calculator.  Please note that the calculator is not particularly intuitive so you need to enter the value first and then press the operation you wish to use! This was done so that most operations can be performed using simple HTML forms.

My search for the best front end testing libraries

The first library I found was the Harmony gem.  Harmony uses Johnson to access the Mozilla Spidermonkey Javascript engine developed for Firefox.  Harmony then uses env.js to simulate a browser DOM.  Now if we’re just going to be doing unit tests at first, then arguably we don’t need a DOM, however considering most of the JavaScript relies on jQuery and in turn the DOM, I think it’s wise to ensure there is support for a DOM and thus we won’t need to mock or stub too much code to get the unit tests running.  I also came across Holygrail which is a Harmony plugin specifically for Ruby on Rails.  I added the following gems to my Gemfile:

I then run `bundle` but unfortunately I ran into issues installing Johnson.  Looking through the errors (and there were loads), they were all related to the C build of this so not something I wanted to try and dig into and solve.  I then took a look at the last commit dates for Johnson and it was worrying, March 2010, so over a year since anything was contributed.  I did the same for Harmony and the last update was July 2010, so again, closing in on a year since the last update.  I also found some posts indicating that Johnson does not support Ruby 1.9.2 which is the version of Ruby I am using.  Encouragingly env.js was updated days ago, so that project is still very active.  So I abandoned the Johnson / Harmony option and looked for a testing framework that has a more active community and was still being developed.

I found RubyRacer which is constantly being updated and arguably uses the best available Javascript engine as it uses Googles V8.  Unfortunately I could not find any examples where The Ruby Racer had been used for RSpec or Cucumber testing other than a few discussions on the matter, some experiments and some basic examples:

• Looks like Joseph Wilk has the right idea, but no activity since Sep 2010.
• Looks like Cucumber can run tests with the V8 engine, but not sure this is quite what we are after.  Also, I am looking to do Javascript Rspec testing too which this does not address.
• I came across a discussion about Jasmine being integrated with Ruby Racer which allowed testing at, however again it looks like incomplete and not really active anymore.

So after some further research, I found that RSpec itself supports Capybara as well as Webrat.  See the RSpec and Capybara Railscast and the following commit on RSpec Rails.  There are also some examples on the Capybara Github home page.  So I set out to get Capybara running with Rails Rspec to see if we could get it to run an RSpec test.  The problem is that we need to decide which driver to use.  The common options are:

• Selenium (launches and automates browser, brilliant but slow)
• Envjs (after trying to install this gem and capaybara-envjs this there are numerous problems getting this working with Ruby 1.9.2, probably because it relies on Johnson which is not Ruby 1.9.2 compatible, see http://groups.google.com/group/ruby-capybara/browse_thread/thread/1556c7a2b0fe0996)
• Celerity (headless browser in Java, requires JRuby)
• Culerity (headless browser that can be run from normal MRI ruby i.e. not JRuby, but does not appear to support Ruby 1.9.2 at present)
• Akephalos which according to Thoughtbot back in Nov 2010 was the best option out there.
• New (as of 26 Jul 11): Capybara Webkit is the new kid on the block, and supports a full high performance Webkit headless browser.  Read at the bottom of the post for an update on how to get this working after following all the steps in this post.

So I installed Capybara and tried to run an RSpec test placing it in the requests folder (which automatically invokes Capybara).  Unfortunately I wrote some tests which assume Javascript is running, but I could not get Capybara and Akephalos / Selenium to run with RSpec.  I am told that Capybara is being used, but it refuses to use a Javascript driver for some reason even with my RSpec tests metadata set as follows:

So eventually I decided to focus for now on Cucumber tests to ensure that Capybara is set up correctly to use Selenium and Akephalos first.  I wrote some basic tests with tags for @javascript and @selenium to test Javascript functionality.  In my cucumber env.rb file I added the following so that Akephalos is the default Javascript driver and Selenium can be used where necessary:

And I ensured that the following gems were added to the Gemfile and bundle was executed:

Cucumber ran as expected, so the following two simple example tests passed and Firefox was instantiated for the test tagged with @selenium:

So now that I knew that Capybara is working correctly, I wanted to get RSpec working with Capybara.  So instead of using RSpec metadata according to the instructions on the Capybara page under the section “Using Capybara with RSpec”, I manually set the driver in the RSpec tests as follows:

And this worked.  So this led me to believe that for some reason RSpec is simply ignoring the metadata.  I did a bit of searching and found a ticket about Driver swapping with RSpec 2 which pointed to a useful Gist.  So I used the code form the Gist as a base and added the following to my spec_helper.rb file:

With this configuration in place I was able to run the following two simple tests, the first using the default Javascript driver Akephalos, and the second using the driver Selenium as it is explicitly required.

However, when I ran the tests from under watchr using —drb Akephalos failed with an error messages:

Failure/Error: within(“#calculator”) do
  RangeError:
     0x0000008184a6a8 is recycled object

I found an open ticket created around a month ago describing this issue.  It seems that as both Cucumber and RSpec are failing Akephalos tests with spork, we’re going to unfortunately need to disable Spork for now.  Whilst Spork helps speed up the start up time for tests, I have read it actually marginally slows down the tests themselves, and based on how long my test are taking to run now once we start up a headless browser and an actual browser, I don’t think the start up time is that significant anymore.  So until the spork and Akephalos issue is resolved, (see http://spacevatican.org/2011/7/3/sporking-with-akephalos for a workaround) I’m going to opt for using watchr without spork.

Front end unit testing from Rails

Whilst for this calculator app there is not a specific Javascript unit testing requirement, for the purposes of this article I wanted to work how I will be unit testing my front end code in future, and what’s the best way for myself.  On one hand, I can simply insert Javascript code into my RSpec tests using code such as:

however that doesn’t feel quite right to me.  Firstly, if the Javascript test code got complex I’d have lots of Javascript code embedded in my Ruby code which is just messy and hard to debug.  Also, I feel like I’d like a solution where the tests could also be run in my browser so that whilst developing my front end code I could manually initiate tests without having to invoke RSpec each time.  I know there are lots of Javascript testing frameworks out there, but the one I hear of most often (and is used by the JQuery team) is QUnit.  I also came across this QUnit plugin for Rails, however it’s very out of date (last update Dec 2009) so I expect no longer relevant.  I then looked at JsTestDriver which allows you to have multiple browsers running as slaves and simultaneously run tests against all those browsers.  Whilst this sounds useful, I don’t want to overcomplicate what I’m trying to achieve at this stage, so I decided to stick with QUnit for this article.

So first thing I did was download and add qunit.js and qunit.css to my project.  I then wanted to keep my QUnit tests in the specs folder along with all my other tests files, but this was not possible out of the box as those files are not accessible from the browser.  So I set up a simple route and controller for my Javascript tests which serves up the Javascript tests using a special QUnit HTML template I set up.  The controller is simply:

Note that I store all my javascript test in /spec/javascript/ and suffix them all with _spec.js.erb.  And I have also forbidden the running of these tests unless in test or development environments.

I then added a route:

And added a simple HTML template javascript_test.js.erb into layouts for use with QUnit tests.

I then modified my calculator.js class to be a bit more testable for the purposes of Unit tests (i.e. removed dependency on the DOM for various methods including the AJAX functions), and set up my first set of tests in a new file I named calculator_spec.js.erb.  Here is an excerpt from the spec file which shows a normal Unit test as well as one which supports asynchronous AJAX requests:

So I now have QUnit running nicely in my browser when I visit http://localhost:3000/javascript_test/calculator on my dev machine.

However what I really want is for these QUnit tests to be run automatically as part of my RSpec tests.  My rationale for this is that all my unit tests are run from RSpec whilst all integration tests are run from Cucumber.  So I don’t want to now have to manually run QUnit tests as well, I want these tests to be run automatically whenever Watchr picks up changes or RSpec is invoked manually.

So I wrote a new RSpec test which simply iterates through the tests in the /spec/javascript folder, invokes the Akephalos headless browser, and renders the HTML and Javascript for the page.  It then sleeps until the QUnit tests have completed (with a 60 second limit per suite of tests), and once the tests have completed it checks that each QUnit test passed using Nokogiri which parses the HTML.  As QUnit generates an HTML based microformat it’s easy to iterate through each passed/failed test.  As you can see in my commit, test 5 “Wait up to 5 seconds but fail with timeout after 3.5 seconds” fails as expected, whereas all the other tests pass.  To ensure the Javascript tests run automatically, I modified watchr to include the Javascript tests too.  See corrected commit here.

More complex user interactions for the Cucumber integration tests

The last thing I want to do now is have some more complex integration test examples working so that we can simulate a more extensive set of user interactions.  Our calculator already supports some more complex interactions as follows:

• Drag and drop - you can drag the calculator current value into the history area
• Use of cookies - a user’s history of drag and drop should be retained across sessions
• Handle a confirm dialog box which requires user input to continue - I have added a confirm when performing an operation on a zero value.
• Handle an alert dialog box which should appear when dividing by zero.

Confirm dialog boxes are supported natively by Ruby Selenium Webdriver, however this method only works with Firefox and would also then fail for any Akephalos tests.  My feeling is that unless Capybara has a generic way of dealing with alert / confirm dialog boxes, then it would be better to stub out this functionality using Javascript.  I found a an article on this, “How to test a confirm dialog with Cucumber” which led me to the this Gist which I forked to support confirm & alert dialog tests with cucumber.  

So I got confirmation and alert dialog boxes working, along with drag and drop testing in this commit.  It was worth noting that Akephalos & HTMLUnit failed the drag and drop test in that no error was raised when calling drag_to, however I found the drag & drop event was never fired for some reason.  So in order to get all my features passing I have used Selenium for the drag & drop tests, and Akephalos for all other tests.

Finally, I realised that although my application has no database, most applications would so I needed to configure RSpec2 to not use transactional fixtures when running Javascript tests.  See commit.

Updates to this post as of 26 July 2011 - Capybara Webkit

Since I wrote this article, the great folk over at Thoughtbot have indeed come up with a solution called Capybara-Webkit that seems to be superior to both Akephalos and Selenium in many regards.  They have integrated QtWebKit, which appears to be a library to enable cross platform development using the popular WebKit which powers Chrome, Safari and most mobile browsers.  The beauty of the solution is that it seems to be really fast, it supports advanced features like drag & drop which were not possible with Akephalos, yet this can all be achieved without opening up a browser which was required previously through the use of Selenium.  I’ve updated the Git repository with the changes so you can see how little you need to do to get Capybara-Webkit to replace both web drivers.

That’s it

I’ve got a test suite I’m pretty happy with now that reliably tests both back end and front end code with both unit and integration tests.  It works well with the latest version of Ruby 1.9.2 and Rails 3.0.  Thanks certainly goes to Capybara, RSpec2, Cucumber, Selenium, Akephalos (and now Capybara-Webkit), Watchr and Spork.  The only thing I’m not entirely convinced with at the moment is the use of Watchr to automatically invoke Cucumber as the Cucumber tests can be particularly slow, but I’m sure each person will have their preference.

You can see the full source code at https://github.com/mattheworiordan/Full-stack-testing/

The live (and pretty pointless) demo project is up at http://full-stack-testing.heroku.com/

Useful links I came across whilst researching this article

• Harmony - DOM + Javascript testing from within Ruby test frameworks such as mini test, unit test, RSpec.  Please note that this project appears to be outdated and does not support Ruby 1.9.2.  This library depends on http://env-js.appspot.com/, which has been superseded by https://github.com/thatcher/env-js
• Selenium Ruby - looks outdated, last news was 2007
• WebDriver Selenium project page
• Culerity - Celerity without the need to run your Ruby code in JRuby 
• Watir - browser automation in Ruby, similar to Selenium, not clear on why Watir over Selenium, other than lack of Java.  Good article on the differences between Watir and Selenium
• Capybara and Envjs (ruby 1.8.*, no ruby 1.9.2 support)
• JsTestDriver - run multiple unit tests across different browsers simultaneously
• Capybara - testing framework supporting numerous headless and headed browsers
• RSpec rails extension
• Dealing with aSynchronous requests when testing 

What next?

I would like to get this all working with a continuous build server which not only runs the full suite of tests whenever commits are pushed, but also runs the front end tests across multiple browsers including multiple operating systems.  I think I’ll be using Selenium and possibly JsTestDriver with QUnitTestRunnerPlugin, but that’s for the next article.  I’ll be working on that as soon as I can, and I’ll post a new blog post when it’s ready. 

Follow me on @mattheworiordan 

If you have any questions, suggestions, or corrections, please do get in touch Matthew O’Riordan

Do it, donate to Wikipedia, they need it and you can afford it

I decided to donate to Wikipedia today not because I feel especially altruistic today, but because of the following:

• It’s genuinely a brilliant website, and it costs money, lots of money to keep it going.  I have and continue to gain a lot of benefit from its existence, so I want to ensure it continues to live on the web.
• As I continue to embrace open source software and the open source community, I realise how powerful it is not just for software development, but for the mankind and the planet as a whole to collaborate and contribute to a greater good. Wikipedia plays a valuable role in education globally, and provides this freely.
• I don’t want advertising on Wikipedia, or for them to succumb to financial incentives which will inevitably result in the site being directed away from simply presenting information from as many perspectives as possible (which is fortunately completely different to being objective, and is arguably one the reasons why Wikipedia is brilliant and survives)
• Wikipedia is suffering at the moment because editors and moderators are dropping like flies.  Whilst I am not yet ready to take on that responsibility, I do hope that my contribution at least helps them to find ways to solve that problem.
• It’s Christmas dammit, they deserver a present.

Come on, join me and donate to Wikipedia.

Matt