Matthew O'Riordan

How elastic are Amazon Elastic Load Balancers (ELB)? Not very it seems

Tue, 13 Mar 2012 22:56:00 +0000

Update: Since I made this post, Amazon have in fact been in touch and have been extremely helpful looking into this issue and running numerous tests to ensure ELB is performing as it should. As soon as we have completed running all tests and conclusive findings, I will make another follow up post detailing the results.

Recently I’ve been working on a project that will feasibly need to respond to 10,000+ SSL web socket connection requests per second, and I’ve been racking my brain as to figure out what type of infrastructure I would need to cope with that.

I did some initial tests and determined that a single CPU instance on EC2 (with 1 EC2 compute power) can cope with a measly 25 SSL handshakes per second. Based on my 10k target, that means I would need 400 CPU instances on EC2 just to deal with handshakes. I soon learn that the certificate key size is paramount, and dropping from the normal 2048 bit key down to 1024 bit improves performance 5 fold, so that now means I only need 80 CPU instances. Still way too many instances, and to be honest, not a piece of architecture I particularly want to look after either as I will need to build a system that can auto-scale on demand.

So I started experimenting with Amazon’s Elastic Load Balancers on the premise that they are elastic. I ran some quick apache bench tests against an ELB server and unsurprisingly got 25 SSL handshakes per second throughput with a 2048 bit key. So I can deduce that an ELB when it starts out is simply a single CPU instance on EC2. I then tried to run apache bench for 30 minutes to see how ELB scales, but unfortunately quickly discovered that that type of test is quite contrived, and ELB scales out dynamically using DNS. So a single load test client would only ever hit one ELB instance, and as such, ELB may not scale as only one IP is hitting it and even if it did, the client would never be able to benefit from that scaling.

So I went ahead and built a load testing farm of daemons that distribute their “attack” on the ELB from loads of different IP addresses (I spun up 40+ micro instances on EC2), and will cleverly keep querying DNS to see when ELB scales and distribute traffic to the new IP addresses that ELB responds to for its public DNS name.

And my findings were interesting, and disappointing. It seems that ELB does in fact scale, but for some reason I am hitting a hard limit for some reason or another. Perhaps Amazon have set a hard limit for some reason, perhaps I need to have something enabled on my account, or perhaps ELB is just not as elastic as it’s claimed to be. I’ll only find out if Amazon get in touch with me to explain ;)

Here is a graph demonstrating the attempted requests per second from my load testing daemons that scaled up request from 50 per second in minute one, up to 6,000 requests per second in the 120th minute. As you can see, other than a few more small upward blips, ELB pretty much flattens out at around 2,000 requests per second with a 1024 bit key and ELB spread across 3 zones (and thus 3 instances).

If anyone has any suggestions or feedback on how to make ELB scale as it should, please do get in touch. I’ve posted my findings into the Amazon forums here and here, ~~but I have little hope that Amazon themselves will respond~~ (see below, Amazon did get in touch and have been brilliant!).

In the mean time, it looks like I’ll be finding another solution.

Update: Since I made this post, Amazon have in fact been in touch and have been extremely helpful looking into this issue and running numerous tests to ensure ELB is performing as it should. As soon as we have completed running all tests and conclusive findings, I will make another follow up post detailing the results.

MaxCDN versus Amazon Cloudfront

Wed, 22 Feb 2012 14:29:17 +0000

Following on from my bad experience with MaxCDN in November, I thought it would be worth a follow up post for those who are interested in how MaxCDN stacks up against Amazon Cloudfront. All my experience is based on production issues I have had with easyBacklog, a simple backlog management tool designed for agencies.

On a positive note, after the outage in November and my numerous support requests, MaxCDN got in touch, apologised for not responding and resolving the issue in a sensible amount of time, and in fact offered me a credit note for all the money I had spent with them to date. +1 for them caring and doing something about it.

However, things did not get better from that point forwards. As a result of the CDN had going down and taking most of my site with it, I thought I should now add monitoring to the CDN. I must admit, I never thought one would need to monitor a CDN that is designed to achieve the five 9s, 99.999% uptime, but clearly I was not willing to take the chance again.

So I set up two kinds of monitors:

Monitors from 3 global locations to cacheable assets i.e. the CDN should not have to contact my server at all and should be serving these assets from the cache.
Monitors from 3 global locations to non-cacheable assets i.e. the CDN has to reload this asset for every HTTP request.

The reason I took this approach is that whilst it’s great that a CDN may be delivering cached assets efficiently, I also want to be sure that each time I do a deploy, it will load up the new assets efficiently as well.

And this is where it all went wrong with MaxCDN, see the graph I sent to their support team below:

All of those red dots means their CDN server did not respond within 10s. Yes, 10 whole seconds. This is a CDN for god’s sake, what is going on here. I ran simultaneous tests directly to the origin server for the same URL and the response times was never more than 750ms, meaning the CDN can for newly deployed assets simply timeout and never serve the assets.

I spoke with MaxCDN numerous times about this issue, and whilst they were doing their best to help, they never managed to resolve the issue. The only thing they did at one point was to move me onto a new type of infrastructure that did not support querystring cache invalidation, and in the process they took my site down for another hour. Thanks MaxCDN.

So here is why I thought I should write a blog post. Previously I was using Jammit and MaxCDN with Rails 3.0 that uses a querystring after the asset URLs to ensure they are not cached. Unfortunately Amazon Cloudfront ignores the querystring, so assets are never refreshed. Hence, I had to use MaxCDN as they do support querystrings for assets. So I bit the bullet, upgraded to Rails 3.2, removed Jammit and upgraded to the asset pipeline (Sprockets). All assets are now precompiled with fingerprints in the URL such as /assets/my-image-au32h121wehqweq.png meaning I can use Cloudfront.

So I changed over to Amazon Cloudfront yesterday and here is how it stacks up:

CloudFront is bloody amazing, not a single spike for non cacheable assets above a few hundred milliseconds, and regardless of location it’s working brilliantly and super fast.

So in the debate of MaxCDN vs Amazon Cloudfront here is my take:

MaxCDN

Pros

Simple to set up and priced right
Allows the use of custom aliases such as easybacklog.netdna-ssl.com
Supports SSL
Feature rich, supports querystring invalidation amongst other things
Nice admin interfaces

Cons

Unreliable, I have experienced a number of outages
Support is not great when needed
Performance is sub-par

Amazon Cloudfront

Pros

Simple to set up and priced right
Supports SSL
Unbelievably reliable
Performance is incredible

Cons

Not as feature rich as some other offerings, but at least you know what is on offer works well
Does not support custom SSL certificates for CNAME resolution of assets, so for SSL you have to use ugly cloudfront.net domains

Get in touch if your experience differs from mine. So far, I am very pleased with Cloudfront.

MaxCDN review - seriously unreliable

Mon, 28 Nov 2011 12:57:00 +0000

* Update 22 Feb 2012: Read my follow on post MaxCDN versus Amazon Cloudfront *

I know, this post will probably come across as a rant, but it’s not. I have recently launched my business (Easybacklog - http://easybacklog.com) into public beta and every customer and client I attract is valuable to me. So when my CDN goes down, and therefore my site is effectively unusable as all stylesheets, Javascript and images have not loaded, I get upset because I might lose customers.

To give you some background, recently I looked around for a CDN to help speed up the loading of my site’s assets. I looked into numerous options, and found MaxCDN to be my number one choice (second to Amazon CloudFront). And to be fair, setting up was reasonably painless (other than issues with payment which meant my account was locked), and I was up and running in no time at all.

And then yesterday it all went wrong, and it stayed like that. At around 11am UK time I found that the site’s asset hosting was no longer working. Firstly the assets were not being served, and secondly the SSL certificate being issued was the wrong one, meaning there was zero chance of my assets being served. Mistakes happen, and I can handle that.

What frustrates me, and has motivated me to post this blog post so as to warn others about MaxCDN’s terrible customer service, is that I could not get hold of anyone at MaxCDN. On their home page, they state 24/7/365 support (screenshotted below):

So as my site was down as this was pretty critical I endeavoured to get in touch with MaxCDN by:

Calling them (no answer, got a voicemail which disconnected and wouldn’t let me leave a message)
Emailed them (no response)
Tweeted them (no response)
Used their contact form (no response)
Used their live chat feature (no response)

Eventually, 10 hours later, I managed to get hold of someone using their live chat feature (still no response to any of my emails), and that was the icing on the cake. He told me two things in our live chat:

Sorry, but it’s the weekend, so we don’t have any engineers available. Someone will get back to you on Monday.
Sorry, but MaxCDN is lower priority than NetNDA (I assume their parent / provider), so we’ll get to your issue when we can.

So I warn you all, if you want a CDN that is reliable, do not even dream of using MaxCDN. I am absolutely flabbergasted that a company touting itself as a CDN would find it acceptable to not fix an issue until the following day. (They have fixed my issue now, 20 hours later, and without an apology)

So who would I recommend as an alternative?
Well I’m looking into that now and hope to post an update in here once I have had some experience using the competitors. The ones I am trying support pay per usage models (i.e. no fixed subscription) and are as follows:

Do tell me if you also experience bad support with MaxCDN so that hopefully others considering MaxCDN will not fall into the same trap that I did.

* Update 22 Feb 2012: Read my follow on post MaxCDN versus Amazon Cloudfront *

URL regular expression for links with or without the protocol

Tue, 22 Nov 2011 22:33:00 +0000

I’ve just come across a pretty common requirement to convert any text that looks like a link into a link within some HTML text. Strangely, after searching for a good 15 minutes for a regular expression, all I could find was either a regular expressions which detects URLs with a protocol such as http://mattheworiordan.com/, or a regular expression which detects URLs without such as www.mattheworiordan.com. Why the hell I could not find one which does both is beyond me, so here I go at posting a solution for anyone else to use.

Here is the holy grail:

/((([A-Za-z]{3,9}:(?:\/\/)?)(?:[-;:&=\+\$,\w]+@)?[A-Za-z0-9.-]+|(?:www.|[-;:&=\+\$,\w]+@)[A-Za-z0-9.-]+)((?:\/[\+~%\/.\w-_]*)?\??(?:[-\+=&;%@.\w_]*)#?(?:[.\!\/\\w]*))?)/

Here is a nice example of this regular expression in action http://jsbin.com/eqocuh/5/edit#source

Please feel free to modify this JSBin, add examples, and update this regular expression, and I will update within this blog post.

Here is an explanation of the regular expression for those who care

(
 ( # brackets covering match for protocol (optional) and domain
  ([A-Za-z]{3,9}:(?:\/\/)?)   # match protocol, allow in format http:// or mailto:
  (?:[\-;:&=\+\$,\w]+@)?   # allow something@ for email addresses
  [A-Za-z0-9\.\-]+   # anything looking at all like a domain, non-unicode domains
  | # or instead of above
  (?:www\.|[\-;:&=\+\$,\w]+@) # starting with something@ or www.
  [A-Za-z0-9\.\-]+   # anything looking at all like a domain
 )
 ( # brackets covering match for path, query string and anchor
  (?:\/[\+~%\/\.\w\-]*)  # allow optional /path
  ?\??(?:[\-\+=&;%@\.\w]*)  # allow optional query string starting with ? 
  #?(?:[\.\!\/\\\w]*) # allow optional anchor #anchor 
 )? # make URL suffix optional
)

Infinite scrolling tab system for JQuery

Tue, 11 Oct 2011 21:38:20 +0100

Google Docs Spreadsheets has a useful worksheet tab system whereby if there are more tabs than the space available, a scroller device automatically appears. Whilst building the new sprints functionality for my backlog management tool easyBacklog, I went on a search for a scrolling tab system and couldn’t find one.

So I’ve built a JQuery plugin inspired by the Google Docs Spreadsheet tab system, along with the rounded tabs solution by Chris Coyier.

Check out the plugin at https://github.com/mattheworiordan/jquery.infinite.tabs

Comments / feature requests / bug reports welcome.

Setting up a Jenkins (old Hudson) Continuous Integration Server

Thu, 01 Sep 2011 16:31:00 +0100

Following on from my recent full stack integration work on easyBacklog, the Agile Project Management app I am building, I have set up a Continuous Integration server which will run both unit tests and integration tests across the entire app each time I push a commit. In order to do this, I needed full support for Selenium and Capybara-Webkit, along with Cucumber and RSpec.

I looked at various CI solutions:

TeamCity - looked really good, but very .NET and Java focussed so that put me off
Integrity - looks like it’s no longer maintained
CruiseControl.rb - not much documentation for it, and not a huge amount of community interest it seems
Jenkins - split from Hudson when they became commercial, and seems to be the big daddy and very well supported in the community.

So I opted for Jenkins and went about setting up a CI server on my Mac Mini with OS X Lion.

As I couldn’t find any easy guides on what to do, I loosely documented the steps I followed to get my CI server up and running so that hopefully others trying to do the same thing will have a good starting point. Here is a simple chronological log of what I did:

Note: I am pulling my working directory from a local Git repository. If you want to use Github, then install the Github plugin (see below) and figure that part out yourself!

Log into your Mac with your admin account

Install Homebrew: /usr/bin/ruby -e “$(curl -fsSL https://raw.github.com/gist/323731)”

If you have Homebrew installed already, then I would advise you `brew update` to get the latest recipes

Install Qt (needed by Capybara-Webkit): brew install qt —build-from-source
Note, this will take over an hour to build, so make yourself a nice cuppa and wait.
Whilst this is happening, I would go ahead and install Firefox which is needed for the standard Selenium install.

Install Jenkins using the native installer, http://mirrors.jenkins-ci.org/

Server should be running, if not, reboot the machine as on Mac OS X a LaunchDaemon is installed by Jenkins.

Go to http://localhost:8080, Manage, Manage Plugins, and install Jenkins Git, Hudson Ruby and Jenkins ruby metrics

I would also advise you set up security for your Jenkins server. Go to Manage Jenkins, Configure System, choose “Jenkins’s own user database” under “Access Control” and the rest is reasonably self-explanatory.

If you want email notifications when builds succeed or fail, then you will either need an SMTP server you can use, or you can configure OS X to run postfix. See the following articles to get postfix running on Mac OS X:

Set up a user called jenkins in OSX (login jenkins).

Modify the file /Library/LaunchDaemon/org.jenkins-ci.plist, remove the GroupName and daemon value and then change UserName to jenkins

You will now need to change the permissions on a folder Jenkins uses, run the following:
sudo chown -R jenkins:wheel /Users/Shared/Jenkins

Now reboot, and make sure Jenkins is still up and running (http://localhost:8080)

Log into your OS X box with the jenkins account

Install RVM: bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

Install ruby: rvm install 1.9.2

Ensure 1.9.2 is default: rvm —default install 1.9.2

Reload your console, and type rvm list. Ruby 1.9.2 should be selected. If not, your default Ruby 1.9.2 is not working.

Now install the Bundler gem, `gem install bundler`

Now before you set up your first job in Jenkins, I would advise that you make sure the jenkins user account has everything needed to actually run Rake and RSpec. So to do this, simply go to the console, create a working folder and git clone git://your-git-respository-url.git to get a working version of the files onto the server.
Run `bundle install && RAILS_ENV=test bundle exec rake db:migrate && RAILS_ENV=test bundle exec rake`. Your Cucumber tests (and Unit tests if you have some) should run. If they don’t pass, figure out why not before you get Jenkins to start automatically building for you.

To set up your CI job, go to Jenkins home page (http://localhost:8080), click New Job, and set the following:

Name the job, and select “Build a free-style software project”, then click Next
I opted for “Discard Old Builds” and kept the last 30 builds
I chose Git and entered the URL (local path) to my Git repository. This could of course be a public Git repository, but make sure when you are logged in as jenkins on your Mac that the repository is accessible. Set up public key / private key authentication if necessary.
Under build triggers, choose Poll SCM. I entered * * * * which means I poll the SCM every minute. If this is a remote repository, then I would suggest you do something like 0,15,30,45 * * * to poll every 15 minutes.
Add a build step, “Execute Shell” and use something like this:

#!/bin/bash -eexport RAILS_ENV=test
source “$HOME/.rvm/scripts/rvm”
rvm use 1.9.2
bundle install —deployment
bundle exec rake db:migrate spec

Under email notifications I would put in your email address for notifications.
I have not yet looked into the Rails / Ruby specific plugins and how to use them, so for now leave these blank.
Save your job

You can then manually start the build by going to the home page, clicking on the title of your build, and clicking Build Now. You should see the status of the build in the Build History (bottom left), and if you click on the Build in progress, and then on Console Output, you can monitor the build as it happens.

Now that you have SCM polling, each time you push a change to your Git repository, a new build should automatically fire off.

Do give me feedback if I’ve missed out any steps. Unfortunately I wrote down a lot of these steps retrospectively.

A library for simulating a drag event with a JQuery UI Sortable widget

Wed, 24 Aug 2011 13:56:03 +0100

I have been working on thorough integration testing of my new Agile Project Management tool called easyBacklog, and came across a problem whereby it seems there is no easy way to test / simulate drag and drop events with a JQuery UI Sortable widget.

Whilst jquery.simulate.js provides functionality to simulate many JQuery and JQuery UI event, it is unable to simulate a drag event for a JQuery UI Sortable widget due to the intricate behaviour needed to make the JQuery UI Sortable widget fire the correct events.

So I’ve developed a library to solve this problem: it allows you to simulate Drag events for a JQuery UI Sortable, and supports a number of useful features.

Go to the Github repository jquery.simulate.drag-sortable.js for the code and the tests.

Testing :focus with JQuery and Selenium or Capybara-Webkit

Tue, 23 Aug 2011 23:37:00 +0100

Recently whilst trying to write integration tests for a JQuery heavy front end application, I came across a very strange issue when testing to see if an element has focus.

What I found is that if an element has focus, yet the actual browser or tab does not have focus, then using the pseudo selector :focus from JQuery (or Sizzle) will not find the focussed item. Putting it another way, the browser and web page must have the focus in the OS for the :focus selector to work.

A simple way to replicate this issue is to go to http://jsbin.com/abozas, open up your Javascript console, and type the following commands

$('#inpt').focus();
$('#inpt').is(':focus');

As you have just put the focus on #inpt, you would expect it to retain focus and thus pass a JQuery test of .is(‘:focus’); However, because the Javascript console has the focus, and not the web page, .is(‘:focus’) will always return false.

Now generally this is not an issue as your web page actually has focus, but unfortunately I discovered that in Rails when using Cucumber, Capaybara and Capybara-Webkit and/or Selenium and testing for :focus, all tests will fail because the browser window does not have the focus of the OS at the time it is run.

So firstly I logged this is an issue with Capybara-Webkit, and in the mean time I’ve worked up a workaround solution by poking around in JQuery and Sizzle (the CSS selector engine that powers not only JQuery, but also Prototype, Dojo, MochiKit, TinyMCE). I discovered that the issue is in fact not with the Sizzle selector logic, but resides with some optimizations that Sizzle uses for performance reasons. For the function matchesSelection, it tries to first use the native method webKitMatchesSelector() to match a selector, and if this fails, it falls back to the Sizzle selector logic. The problem is that the webKitMatchesSelector is OS aware and thus removes focus from an element when the browser does not have focus, whereas Sizzle is unaware of the browser’s focus status and thus matches successfully using the following code:

function( elem ) {
  return elem === elem.ownerDocument.activeElement;
}

So to fix this problem, I have written a small Javascript file which you should include when you are running a test or cucumber Rails environment. This script will then deactivate the native matches selector and query selector for your browser.

The code to fix the :focus issue can be found at https://gist.github.com/1166821 and simply does the following:

/* Prevent use of native find selector */
document.querySelectorAll = false;

/* Prevent use of native matches selector */
document.documentElement.matchesSelector = false;
document.documentElement.mozMatchesSelector = false;
document.documentElement.webkitMatchesSelector = false;
document.documentElement.msMatchesSelector = false;

If you would like to replicate this issue, review this Gist. An example set of tests where the issue is resolved can be seen in this Gist.

I hope others find this useful.

Cucumber and Capybara-Webkit automatic screenshots

Fri, 19 Aug 2011 13:34:00 +0100

This post has been superseded by a Gem I have written to do automatic screen shots for Capybara (not just Cucumber, also supports RSpec and Mini-test). Please go to https://github.com/mattheworiordan/capybara-screenshot

Writing Cucumber front end tests (AJAX & Javascript) can be pretty damned painful and slow, especially when a failure does not come with any useful error message. Fortunately I’ve been using Capybara-Webkit recently, which brings two very useful features for debugging issues:

Javascript console messages (console.log) are sent directly to the shell whilst running your tests. So you can effectively add debugging statements in your Javascript that are sent through to your terminal. See below.

Capybara allows you to save out PNG screenshots from the current page. So I’ve written two convenient pieces of code that you may find useful.

The first one allows you to write “Then take a snapshot” or “Then take a snapshot and show me the page” within your features, and a PNG and HTML file will be saved into tmp/capyabara.

And insert this into your shared_steps.rb (if you have one) under /features/step_definitions

Then /take a snapshot(| and show me the page)/ do |show_me|
  page.driver.render Rails.root.join("tmp/capybara/#{Time.now.strftime('%Y-%m-%d-%H-%M-%S')}.png")
  Then %{show me the page} if show_me.present?
end

The second one ensures that whenever a scenario fails, a screen shot and HTML version of the page is saved into tmp/capybara automatically.

Place this in your /features/support folder

def screen_shot_and_save_page
  require 'capybara/util/save_and_open_page'
  path = "/#{Time.now.strftime('%Y-%m-%d-%H-%M-%S')}"
  Capybara.save_page body, "#{path}.html"
  page.driver.render Rails.root.join "#{Capybara.save_and_open_page_path}" "#{path}.png"
end

begin
  After do |scenario|
    screen_shot_and_save_page if scenario.failed?
  end
rescue Exception => e
  puts "Snapshots not available for this environment.\n
    Have you got gem 'capybara-webkit' in your Gemfile and have you enabled the javascript driver?"
end

This code is up on a Gist at Github at so feel free to fork away.

Rate limiting Javascript requests

Tue, 26 Jul 2011 14:30:00 +0100

I recently needed to rate limit requests to a particular function in Javascript, and was surprised nothing obvious came up in Google searches so here is my contribution for others who need this.

Note that I am a big fan of Underscore.js and have thus simply extended the _ to include my new rate limiting function. Also, many people immediately think that _.throttle will achieve rate limiting, but in fact _.throttle is a destructive rate limiting function in that it will disregard all requests that arrive within the time threshold specified. If therefore you want to ensure that all calls are still executed, but are never executed more than once ever X milliseconds, then I suggest you use my method.

The code is uploaded to a Gist which you can fork at https://gist.github.com/1084831, and is embedded below.

// Rate limit ensures a function is never called more than every [rate]ms
// Unlike underscore's _.throttle function, function calls are queued so that
//   requests are never lost and simply deferred until some other time
//
// Parameters
// * func - function to rate limit
// * rate - minimum time to wait between function calls
// * async - if async is true, we won't wait (rate) for the function to complete
//           before queueing the next request
// 
// Example 
// function showStatus(i) { 
//   console.log(i); 
// } 
// var showStatusRateLimited = _.rateLimit(showStatus, 200); 
// for (var i = 0; i < 10; i++) { 
//   showStatusRateLimited(i); 
// } 
// 
// Dependencies 
// * underscore.js 
// 
_.rateLimit = function(func, rate, async) {
   var queue = [];
   var timeOutRef = false;
   var currentlyEmptyingQueue = false;
   
   var emptyQueue = function() {
     if (queue.length) {
       currentlyEmptyingQueue = true;
       _.delay(function() {
         if (async) {
           _.defer(function() { queue.shift().call(); });
         } else {
           queue.shift().call();
         }
         emptyQueue();
       }, rate);
     } else {
       currentlyEmptyingQueue = false;
     }
   };
   
   return function() {
     // get arguments into an array
     var args = _.map(arguments, function(e) { return e; });
     // call apply so we can pass in arguments as parameters as opposed to an array
     queue.push( _.bind.apply(this, [func, this].concat(args)) );
     if (!currentlyEmptyingQueue) { emptyQueue(); }
   };
};

You can see a working version at http://jsbin.com/upadif/8/edit#preview

Full stack integration testing with Rails 3, Cucumber, RSpec, QUnit and Capybara

Sun, 17 Apr 2011 23:48:00 +0100

I find it enlightening to be part of such an active Ruby and RoR community where Gems, code samples and responsive community support is available, however I also find it hard sometimes for that very same reason to figure out what stack to use for my new projects because there are just too many options.

One of the problems I’ve had for a while now is figuring out what stack to use for my testing, both for my back end code (unit & integration) as well as my front end code (unit & integration). So this blog post is about my journey in April 2011 to find the best full stack integration components for a Rails project and document it for others to use. I hope others find this useful.

Background

I use RSpec and Cucumber for all my testing, but my application’s front end is becoming increasingly complex and needs testing as well. I would ideally like a combination of both unit testing and integration testing on the Javascript and front end elements. And I want my tests to be as fast as possible because the longer it takes the less likely it is I’ll run them.

The objective of this exercise

Document the entire process collecting all useful resources together in one place so others can learn from this
Set up a basic Rails 3 project and push it to Github which has enough back end & front end code to write some meaningful tests. This must include some asynchronous client-server requests. And a Javascript framework such as jQuery should be used to make the stack and environment realistic.
Write RSpec & Cucumber tests for traditional unit testing and integration testing.
Explore options to unit test and integration test the front end code, documenting the pros & cons for each
Find the best stack for me and put that up on Github for others to use

My journey to find the ideal full stack integration suite for Rails 3 (as of now)

I installed a clean version of Rails 3.0.3 on Ruby 1.9.2. See commit.

I like to use RSpec for all unit tests, so I installed RSpec and removed all the old Test::Unit files in /test/. See commit. After you add the gems to your Gemfile, remember run `bundle` and also to run the RSpec generator `script/rails generate rspec:install`. For those of you that use scaffolding (I avoid it), I also modified config/application.rb to instruct Rails 3’s generators to always use RSpec rather than Test::Unit.

My favourite tool for integration testing is Cucumber. RSpec can be used for integration testing too, and I know there is a current view from DHH himself that both RSpec and Cucumber are unnecessary, however I like Cucumber and the readable user stories are great. So I went ahead and installed Cucumber that by default now uses Capybara (it used to default to Webrat). I followed the Cucumber Rails installation instructions, however to ensure Capybara is used for tests as opposed to Webrat I ran the following generator command `rails generate cucumber:install —capybara`. See commit.

In order to get the project ready for front end development, I cleared out the index.html file, installed a copy of jQuery and RoR JQuery UJS, removed all javascript files and added the UJS Javascript file), and added in a stripped down jQuery UI code base to allow for some more complex user interactions. See commit.

I then went ahead and installed SASS because writing traditional CSS just sucks. I normally use HAML as well instead of ERB, but for the purposes of this example I thought it best to keep things as understandable as possible and it’s quite possible HAML is less understood (and once again, DHH does not like it). See commit.

Building the Calculator app and tests

Following good TDD, I then went ahead and wrote the tests for a new Calculator model class that I am about to build. The calculator is intentionally very simple and only supports addition, subtraction, multiplication and division. I kept the tests simple and adequate for the purposes of this project. I then ran RSpec `rspec spec/models/*.rb` and got a load of errors as the Calculator class does not yet exist.

In order to satisfy the tests, I built a simple Calculator class that lives in the /app/models/ directory. I ran RSpec until everything was green, and voila, we now have a test suite for the model which passes. See commit.

I then built a very simple HTML based calculator and a controller called Calculator to manage all calculator operations. See commit. I pushed it up online using Heroku and posted it at http://full-stack-testing.heroku.com/calculator

Once finished, I added some Cucumber integration tests which will run using Capybara’s default driver rack_test as they do not need Javascript support. Ideally I should have written the features first, but for the purposes of this example I was very much discovering as I built, and had no spec to work from. Fortunately in most traditional situations I have a spec I can work from and can therefore follow TDD and build out the Cucumber features first.

The first test was extremely simple as follows:

I ran `rake cucumber` and ran into the following issues:

Firstly I received the error:

schema.rb doesn’t exist yet. Run “rake db:migrate” to create it then try again. If you do not intend to use a database, you should instead alter /Users/matthew/Projects/full_stack_integration_testing/config/application.rb to limit the frameworks that will be loaded

To fix that I simply ran rake db:migrate which created an empty schema.rb file.

I then re-ran `rake db:migrate` and ran `rake cucumber` again. Unfortunately this time I received a rather less intuitive error:

undefined local variable or method `node’ for #<Capybara::Driver::Node tag=”a” path=”/html/body/a”> (NameError)

After a little searching it seems this is a known error. I found this strange considering I thought I was on the latest stable versions of all gems, and my feature is extremely simple. However, after looking at a related closed issue in Capybara I see that this issue was corrected, so I clearly don’t have the latest gems. I had to modify my Gemfile to explicitly request cucumber-rails with a version greater than or equal to 0.4.0 to replace the 0.3.2 version that we was used by default.

I then re-ran `rake cucumber` and unfortunately hit another error as described at http://blog.firsthand.ca/2011/02/cucumber-rails-undefined-local-variable.html

I had to re-run `rails g cucumber:install` to regenerate the cucumber files, and then I was ready to run rake cucumber again. See commit.

So I finally ran `rake cucumber` and fortunately my very simple test passed as follows:

So I went on to writing some more expansive Cucumber scenarios to provide better integration testing of the calculator. And after building only one additional step, I managed to create a Cucumber feature which covered the bulk of the functionality offered including exceptions for the Calculator. See commit.

Test performance and Spork

So a recurring problem I have is that whilst Cucumber tests report completing quite quickly (in around 2.5s in this case), it took considerably longer than that in reality as the rails environment start up time is very slow. Whilst I appreciate testing is an absolute necessity, if it’s going to be painfully then that’s going inevitably deter me from testing. So I have heard about spork on a few of the podcasts I follow, and understand that spork is effectively a library which runs your test suite by forking a new copy of the Rails environment each time you run your tests. So I went ahead and followed the Spork installation instructions.

Specifically I added the following lines to my Gemfile

I then ran `spork —bootstrap` which modified the file spec/spec_helper.rb and added further installation instructions into that file. I looked through the instructions and read that out of the box spork works well with rails. So I left all settings as they were.

I then started up the spork server by simply running `spork` and got the following:

I then ran my first set of RSpec tests with `rspec —drb spec/**/*` and it was damn quick. Everything was completed in under a second, amazing.

So I then read that to run the cucumber tests using spork all I needed was to run `script/cucumber —drb` so I went ahead and did that but unfortunately was not quite so lucky. I got the response:

WARNING: No DRb server is running. Running features locally:

So I did a bit of searching and found an article on using spork with DRB on the cucumber Wiki. It turns out I need to run the generator again and include spork support. As I need capybara support too (see above), I ran the following:

rails generate cucumber:install —capybara —spork

It asked to overwrite my paths file, so I let it do that and just reinserted the custom path I had added earlier.

I then ran `script/cucumber —drb’ and was told again:

WARNING: No DRb server is running. Running features locally:

I did a bit of searching and discovered that this was because the spork DRB server was actually running for use with RSpec and it needs to be run for use with Cucumber. So I stopped the spork server and executed the spork server for cucumber as follows `spork cuc`

I then run `script/cucumber —drb’ and as expected the test ran immediately, considerably quicker than before.

As I want to run both Cucumber tests and RSpec tests as fast as possible, I wanted to find out how I can run spork for both cucumber an RSpec simultaneously. It turns out this is quite simple, all I need to do is run the following command:
`bundle exec spork cucumber & bundle exec spork rspec &`
which gets spork for cucumber and rspec running as background tasks, and on different ports namely 8989 for RSpec and 8990 for Cucumber. Note you can access these tasks using `jobs`, or `fg [job number]`. You can also kill these tasks using `kill %[job number]`

Spork for both environments is something I’m going to use often, so I set up a simple script script/spork which runs spork in the background for both RSpec and Cucumber. To make this an executable I ran the command chmod 777 script/spork in the root. I ran `script/spork` and both servers were started in the background.

In order to measure performance improvement of the tests using spork, I set up a basic benchmarking script residing in scripts. The code is extremely simple and runs the rspec tests and cucumber tests without DRB (that’s what spork uses), and then with DRB to see the difference. I noticed however that cucumber.yml was configured to run —drb as an option automatically, so I removed this and added -c for colour output. I then ran script/benchmark_tests and got the following results:

RSpec without spork took: 9.81s
RSpec with spork (DRB) took: 1.11s
Spork saved 8.70s when running RSpec tests
Cucumber without spork took: 13.69s
Cucumber with spork (DRB) took: 4.31s
Spork saved 9.38s when running Cucumber tests

So it looks like spork is doing a great job in saving on the start up time of the tests for both Cucumber and RSpec. Admittedly as my tests increase in size this start up time saving will become less relevant, but any speed improvement is welcome especially when running RSpec unit tests which are typically quite fast. See commit.

Automated tests and Watchr

And this is all great, but I don’t really like jumping into the console every time I need to run a test. What I want is to run tests automatically whenever I change the relevant files in my project. I’ve heard of two apps which achieve this, autotest and watchr. Watchr seems to be gaining more traction of late, and my previous experience with autotest has not been brilliant and have always had problems getting it running reliably. So without much further research, I decided to use Watchr quite simply because influential Rubyists have been talking about it recently so I trust their judgement. So firstly I added watchr gem to my Gemfile and ran `bundle`

I then found a Gist which seems to roughly achieve what I want in that it supports DRB, Cucumber and RSpec. I saved the Gist source code to script/watchr.rb and I tried running `watchr script/watchr.rb` but unfortunately was told that Growl was not installed. I added growl to the Gemfile, ran `bundle` and then reran `watchr script/watchr.rb` and it started up as expected. However, when changing an RSpec file in spec/models the tests did not run as expected, so I made a few changes to watchr.rb. Please take a look at my Gist to see the list of changes.

So once I made my changes, everything seemed to be working except that the growl notifications did not have an image displaying, and the output from RSpec was not colour coded. So firstly to address the Growl issue I downloaded some PNG status icons and saved them into /script/.watchr_images. In order to resolve the ANSI colors being lost for RSpec issue, I tried a number of alternatives using all methods described in the article “5 ways to run commands from ruby“ but unfortunately none of those solutions worked. So I posted a question on Stackoverflow and someone helped me find the solution. In order to have colour show up in RSpec you need to tell it you are outputting to tty so that colour is supported, so I modified the $spec_cmd in watchr.rb to the following:

Now any changes to features will trigger Cucumber, and any changes to models or unit tests will trigger RSpec and confine to the effected files where possible. See commit.

Front end unit and integration testing

So onto the more difficult parts, front end unit and integration testing. So I have gone ahead and added some trivial front end code to the calculator which presents a set of buttons for the calculator so users don’t have to use the text field when entering the number used in operations. I wanted to ensure that my tests had some AJAX functionality too as this is generally needed in my web applications. So in summary, the code I added provides us with the following to be tested:

Simple Javascript class methods that can be unit tested i.e. no DOM is required
DOM manipulation on load to add additional features to the calculator
AJAX communication with a server and dynamic updating of the DOM including handling of errors
Prompt dialog box which requires the user to interact with to continue when performing an operation on zero
Alert dialog box which prevents a user from dividing by zero
JQuery UI Dialog box which again needs user to interact with to continue
Drag & drop functionality (you can drag values from the calculator to a pretty pointless memory bank)
Use of persistent cookies to persist the values stored in the memory bank

I committed all those changes and you can see the calculator live at http://full-stack-testing.heroku.com/calculator. Please note that the calculator is not particularly intuitive so you need to enter the value first and then press the operation you wish to use! This was done so that most operations can be performed using simple HTML forms.

My search for the best front end testing libraries

The first library I found was the Harmony gem. Harmony uses Johnson to access the Mozilla Spidermonkey Javascript engine developed for Firefox. Harmony then uses env.js to simulate a browser DOM. Now if we’re just going to be doing unit tests at first, then arguably we don’t need a DOM, however considering most of the JavaScript relies on jQuery and in turn the DOM, I think it’s wise to ensure there is support for a DOM and thus we won’t need to mock or stub too much code to get the unit tests running. I also came across Holygrail which is a Harmony plugin specifically for Ruby on Rails. I added the following gems to my Gemfile:

I then run `bundle` but unfortunately I ran into issues installing Johnson. Looking through the errors (and there were loads), they were all related to the C build of this so not something I wanted to try and dig into and solve. I then took a look at the last commit dates for Johnson and it was worrying, March 2010, so over a year since anything was contributed. I did the same for Harmony and the last update was July 2010, so again, closing in on a year since the last update. I also found some posts indicating that Johnson does not support Ruby 1.9.2 which is the version of Ruby I am using. Encouragingly env.js was updated days ago, so that project is still very active. So I abandoned the Johnson / Harmony option and looked for a testing framework that has a more active community and was still being developed.

I found RubyRacer which is constantly being updated and arguably uses the best available Javascript engine as it uses Googles V8. Unfortunately I could not find any examples where The Ruby Racer had been used for RSpec or Cucumber testing other than a few discussions on the matter, some experiments and some basic examples:

Looks like Joseph Wilk has the right idea, but no activity since Sep 2010.
Looks like Cucumber can run tests with the V8 engine, but not sure this is quite what we are after. Also, I am looking to do Javascript Rspec testing too which this does not address.
I came across a discussion about Jasmine being integrated with Ruby Racer which allowed testing at, however again it looks like incomplete and not really active anymore.

So after some further research, I found that RSpec itself supports Capybara as well as Webrat. See the RSpec and Capybara Railscast and the following commit on RSpec Rails. There are also some examples on the Capybara Github home page. So I set out to get Capybara running with Rails Rspec to see if we could get it to run an RSpec test. The problem is that we need to decide which driver to use. The common options are:

Selenium (launches and automates browser, brilliant but slow)
Envjs (after trying to install this gem and capaybara-envjs this there are numerous problems getting this working with Ruby 1.9.2, probably because it relies on Johnson which is not Ruby 1.9.2 compatible, see http://groups.google.com/group/ruby-capybara/browse_thread/thread/1556c7a2b0fe0996)
Celerity (headless browser in Java, requires JRuby)
Culerity (headless browser that can be run from normal MRI ruby i.e. not JRuby, but does not appear to support Ruby 1.9.2 at present)
Akephalos which according to Thoughtbot back in Nov 2010 was the best option out there.
New (as of 26 Jul 11): Capybara Webkit is the new kid on the block, and supports a full high performance Webkit headless browser. Read at the bottom of the post for an update on how to get this working after following all the steps in this post.

So I installed Capybara and tried to run an RSpec test placing it in the requests folder (which automatically invokes Capybara). Unfortunately I wrote some tests which assume Javascript is running, but I could not get Capybara and Akephalos / Selenium to run with RSpec. I am told that Capybara is being used, but it refuses to use a Javascript driver for some reason even with my RSpec tests metadata set as follows:

So eventually I decided to focus for now on Cucumber tests to ensure that Capybara is set up correctly to use Selenium and Akephalos first. I wrote some basic tests with tags for @javascript and @selenium to test Javascript functionality. In my cucumber env.rb file I added the following so that Akephalos is the default Javascript driver and Selenium can be used where necessary:

And I ensured that the following gems were added to the Gemfile and bundle was executed:

Cucumber ran as expected, so the following two simple example tests passed and Firefox was instantiated for the test tagged with @selenium:

So now that I knew that Capybara is working correctly, I wanted to get RSpec working with Capybara. So instead of using RSpec metadata according to the instructions on the Capybara page under the section “Using Capybara with RSpec”, I manually set the driver in the RSpec tests as follows:

And this worked. So this led me to believe that for some reason RSpec is simply ignoring the metadata. I did a bit of searching and found a ticket about Driver swapping with RSpec 2 which pointed to a useful Gist. So I used the code form the Gist as a base and added the following to my spec_helper.rb file:

With this configuration in place I was able to run the following two simple tests, the first using the default Javascript driver Akephalos, and the second using the driver Selenium as it is explicitly required.

However, when I ran the tests from under watchr using —drb Akephalos failed with an error messages:

Failure/Error: within(“#calculator”) do
RangeError:
0x0000008184a6a8 is recycled object

I found an open ticket created around a month ago describing this issue. It seems that as both Cucumber and RSpec are failing Akephalos tests with spork, we’re going to unfortunately need to disable Spork for now. Whilst Spork helps speed up the start up time for tests, I have read it actually marginally slows down the tests themselves, and based on how long my test are taking to run now once we start up a headless browser and an actual browser, I don’t think the start up time is that significant anymore. ~~So until the spork and Akephalos issue is resolved,~~ (see http://spacevatican.org/2011/7/3/sporking-with-akephalos for a workaround) I’m going to opt for using watchr without spork.

Front end unit testing from Rails

Whilst for this calculator app there is not a specific Javascript unit testing requirement, for the purposes of this article I wanted to work how I will be unit testing my front end code in future, and what’s the best way for myself. On one hand, I can simply insert Javascript code into my RSpec tests using code such as:

however that doesn’t feel quite right to me. Firstly, if the Javascript test code got complex I’d have lots of Javascript code embedded in my Ruby code which is just messy and hard to debug. Also, I feel like I’d like a solution where the tests could also be run in my browser so that whilst developing my front end code I could manually initiate tests without having to invoke RSpec each time. I know there are lots of Javascript testing frameworks out there, but the one I hear of most often (and is used by the JQuery team) is QUnit. I also came across this QUnit plugin for Rails, however it’s very out of date (last update Dec 2009) so I expect no longer relevant. I then looked at JsTestDriver which allows you to have multiple browsers running as slaves and simultaneously run tests against all those browsers. Whilst this sounds useful, I don’t want to overcomplicate what I’m trying to achieve at this stage, so I decided to stick with QUnit for this article.

So first thing I did was download and add qunit.js and qunit.css to my project. I then wanted to keep my QUnit tests in the specs folder along with all my other tests files, but this was not possible out of the box as those files are not accessible from the browser. So I set up a simple route and controller for my Javascript tests which serves up the Javascript tests using a special QUnit HTML template I set up. The controller is simply:

Note that I store all my javascript test in /spec/javascript/ and suffix them all with _spec.js.erb. And I have also forbidden the running of these tests unless in test or development environments.

I then added a route:

And added a simple HTML template javascript_test.js.erb into layouts for use with QUnit tests.

I then modified my calculator.js class to be a bit more testable for the purposes of Unit tests (i.e. removed dependency on the DOM for various methods including the AJAX functions), and set up my first set of tests in a new file I named calculator_spec.js.erb. Here is an excerpt from the spec file which shows a normal Unit test as well as one which supports asynchronous AJAX requests:

So I now have QUnit running nicely in my browser when I visit http://localhost:3000/javascript_test/calculator on my dev machine.

However what I really want is for these QUnit tests to be run automatically as part of my RSpec tests. My rationale for this is that all my unit tests are run from RSpec whilst all integration tests are run from Cucumber. So I don’t want to now have to manually run QUnit tests as well, I want these tests to be run automatically whenever Watchr picks up changes or RSpec is invoked manually.

So I wrote a new RSpec test which simply iterates through the tests in the /spec/javascript folder, invokes the Akephalos headless browser, and renders the HTML and Javascript for the page. It then sleeps until the QUnit tests have completed (with a 60 second limit per suite of tests), and once the tests have completed it checks that each QUnit test passed using Nokogiri which parses the HTML. As QUnit generates an HTML based microformat it’s easy to iterate through each passed/failed test. As you can see in my commit, test 5 “Wait up to 5 seconds but fail with timeout after 3.5 seconds” fails as expected, whereas all the other tests pass. To ensure the Javascript tests run automatically, I modified watchr to include the Javascript tests too. See corrected commit here.

More complex user interactions for the Cucumber integration tests

The last thing I want to do now is have some more complex integration test examples working so that we can simulate a more extensive set of user interactions. Our calculator already supports some more complex interactions as follows:

Drag and drop - you can drag the calculator current value into the history area
Use of cookies - a user’s history of drag and drop should be retained across sessions
Handle a confirm dialog box which requires user input to continue - I have added a confirm when performing an operation on a zero value.
Handle an alert dialog box which should appear when dividing by zero.

Confirm dialog boxes are supported natively by Ruby Selenium Webdriver, however this method only works with Firefox and would also then fail for any Akephalos tests. My feeling is that unless Capybara has a generic way of dealing with alert / confirm dialog boxes, then it would be better to stub out this functionality using Javascript. I found a an article on this, “How to test a confirm dialog with Cucumber” which led me to the this Gist which I forked to support confirm & alert dialog tests with cucumber.

So I got confirmation and alert dialog boxes working, along with drag and drop testing in this commit. It was worth noting that Akephalos & HTMLUnit failed the drag and drop test in that no error was raised when calling drag_to, however I found the drag & drop event was never fired for some reason. So in order to get all my features passing I have used Selenium for the drag & drop tests, and Akephalos for all other tests.

Finally, I realised that although my application has no database, most applications would so I needed to configure RSpec2 to not use transactional fixtures when running Javascript tests. See commit.

Updates to this post as of 26 July 2011 - Capybara Webkit

Since I wrote this article, the great folk over at Thoughtbot have indeed come up with a solution called Capybara-Webkit that seems to be superior to both Akephalos and Selenium in many regards. They have integrated QtWebKit, which appears to be a library to enable cross platform development using the popular WebKit which powers Chrome, Safari and most mobile browsers. The beauty of the solution is that it seems to be really fast, it supports advanced features like drag & drop which were not possible with Akephalos, yet this can all be achieved without opening up a browser which was required previously through the use of Selenium. I’ve updated the Git repository with the changes so you can see how little you need to do to get Capybara-Webkit to replace both web drivers.

That’s it

I’ve got a test suite I’m pretty happy with now that reliably tests both back end and front end code with both unit and integration tests. It works well with the latest version of Ruby 1.9.2 and Rails 3.0. Thanks certainly goes to Capybara, RSpec2, Cucumber, Selenium, Akephalos (and now Capybara-Webkit), Watchr and Spork. The only thing I’m not entirely convinced with at the moment is the use of Watchr to automatically invoke Cucumber as the Cucumber tests can be particularly slow, but I’m sure each person will have their preference.

You can see the full source code at https://github.com/mattheworiordan/Full-stack-testing/

The live (and pretty pointless) demo project is up at http://full-stack-testing.heroku.com/

Useful links I came across whilst researching this article

Harmony - DOM + Javascript testing from within Ruby test frameworks such as mini test, unit test, RSpec. Please note that this project appears to be outdated and does not support Ruby 1.9.2. This library depends on http://env-js.appspot.com/, which has been superseded by https://github.com/thatcher/env-js
Selenium Ruby - looks outdated, last news was 2007
WebDriver Selenium project page
Culerity - Celerity without the need to run your Ruby code in JRuby
Watir - browser automation in Ruby, similar to Selenium, not clear on why Watir over Selenium, other than lack of Java. Good article on the differences between Watir and Selenium
Capybara and Envjs (ruby 1.8.*, no ruby 1.9.2 support)
JsTestDriver - run multiple unit tests across different browsers simultaneously
Capybara - testing framework supporting numerous headless and headed browsers
RSpec rails extension
Dealing with aSynchronous requests when testing

What next?

I would like to get this all working with a continuous build server which not only runs the full suite of tests whenever commits are pushed, but also runs the front end tests across multiple browsers including multiple operating systems. I think I’ll be using Selenium and possibly JsTestDriver with QUnitTestRunnerPlugin, but that’s for the next article. I’ll be working on that as soon as I can, and I’ll post a new blog post when it’s ready.

Follow me on @mattheworiordan

If you have any questions, suggestions, or corrections, please do get in touch Matthew O’Riordan

Do it, donate to Wikipedia, they need it and you can afford it

Thu, 24 Dec 2009 15:55:26 +0000

I decided to donate to Wikipedia today not because I feel especially altruistic today, but because of the following:

It’s genuinely a brilliant website, and it costs money, lots of money to keep it going. I have and continue to gain a lot of benefit from its existence, so I want to ensure it continues to live on the web.
As I continue to embrace open source software and the open source community, I realise how powerful it is not just for software development, but for the mankind and the planet as a whole to collaborate and contribute to a greater good. Wikipedia plays a valuable role in education globally, and provides this freely.
I don’t want advertising on Wikipedia, or for them to succumb to financial incentives which will inevitably result in the site being directed away from simply presenting information from as many perspectives as possible (which is fortunately completely different to being objective, and is arguably one the reasons why Wikipedia is brilliant and survives)
Wikipedia is suffering at the moment because editors and moderators are dropping like flies. Whilst I am not yet ready to take on that responsibility, I do hope that my contribution at least helps them to find ways to solve that problem.
It’s Christmas dammit, they deserver a present.

Come on, join me and donate to Wikipedia.

Matt

Why I chose tumblr over Posterous

Tue, 22 Dec 2009 17:48:49 +0000

I recently set up my personal blog as I am desperately looking for exciting new ways to waste my time expressing myself online. As I am spending quite a bit of time learning Ruby on Rails recently, I thought my best bet would be build myself a little blog in RoR and that would give me another excuse to use my new skills. However, I soon realised that building my own blog would only result in the following:

Extreme time wastage reinventing the wheel. Considering the time I am about to waste posting stuff into my blog, I thought it rather excessive wasting even more time building my own blog.
Blog software is surprisingly sophisticated now, getting my blog to support any of the cool features that I’ll probably soon want is going to take a long time.
Design and interface will suffer - I am not a designer and although I think I have a keen eye for good design, I will only disappoint myself trying to build something I actually think looks good.

So I then thought, heck, WordPress is probably the most infamous blogging platform out there, why not just use that. It has loads of free and paid for themes/skins so it could look good, it has 1,000s of plugins (one of which I recently contributed, albeit a simple and rather shoddy one) so bound to have everything I need. But again, I decided against this because:

I really cannot be arsed to waste my time pissing about with setting up a database, WordPress themes, templates, tweaking things I don’t like, removing all the fat in the features by turning them off etc.
PHP sucks and WordPress is a good example of what PHP is - a mess of code that is held together with cheap masking tape. Don’t get me wrong, PHP can be good, but it doesn’t encourage good practices so larger applciations just become a big mess. After recently doing two WordPress websites for a bit of fun, I don’t particularly feel like working in WordPress again. Please note, that I do still think WordPress is genius if you want to get something up quickly and will largely just do the job, but personally I don’t want to look at the code base ever again or have to tinker with that big smelly pile of code.

So my next 2 options were Tumblr or Posterous. Both of them I believe are superb blogging platforms because they’re simple, they’re pretty feature rich but the features are solely focussed on blogging and or micro-blogging to some extent.

Posterous seems to be the favourite generally at the moment as it has some really nice features (or possibly gimmicks) that I suspect attract people. It allows you to start your blog without an account by just simply emailing Posterous. Its great for group publishing i.e. managing multiple blog authors. It allows automatic republishing of content into other blogs (although I am not quite sure why one would want to do this unless you are simply creating links back to your content which kinds of defeats the purpose of having a blog if all it is is links… why not just use Twitter). I set up my blog, pointed a sub-domain of mattheworiordan.com to it, and everything was up in around 15 minutes. Pretty impressive, but I wanted to give Tumblr a try to.

I got set up with Tumblr equally quickly. The only gripe I had with Tumblr was to do with their WYSIWYG support for Chrome on Mac, but considering it was only out for a few days I can’t really complain about that. What I loved about Tumblr was their wide array of themes I could choose from which were generally very nice, and the sensible set of features. I don’t think there is anything in Tumblr that I need that is not already there. I did have one small issue with Tumblr, I emailed their support and got an email back within minutes - awesome, considering it’s a free service.

So I have chosen Tumblr as my blogging platform because:

It’s more me in terms of design, interface, general usability. It looks good and that’s important whenever people interact with my content.
It has a groovy Mac Widget which allows me to post quickly to my blog.
I can use email or even phone in and post something to my blog, yup, I can phone in (although it’s an American number)
It allows me to post links back to my posts into my Twitter account (a place where reposting seems sensible)

Google Chrome billboards literally outside my home

Mon, 14 Dec 2009 18:07:00 +0000

I’m not quite sure what to make of it. Madness, or genius? I am in some way confused though as to why Google chose to develop their own Chrome browser when they could have just supported Mozilla in their efforts to build their own browser (which I love bar the speed and lack of page tabs at the top).

Although Chrome is open source, I do wonder how many people actually contribute to Chrome other than Google employees..

Even Google employs idiots

Mon, 14 Dec 2009 17:19:00 +0000

After setting up the fantastic Google Apps Enterprise Edition for my email on my domain, I ran into some issues when I tried to migrate my 5GB of email from my Exchange 2007 server to Google Apps.

So I contacted their support service which is a commercial service they offer because I pay for the Enterprise Edition, and I was kindly taken care of by a technical support idiot who had no understanding of the issue, and was entirely unable to respond to the problem.

See the thread of an idiot at Google’s incompetence here which provides some insight into what happened.

To be fair, it’s not fair to think that Google could only employ geniuses and run a slick operation, but I think it demonstrates how a software/internet business which once required very little human interaction cannot be sustained when it needs people to scale.

I do hope my next venture requires few people and scales well. Although I’d of course be happy if it turned out to be a Google even if it did employ a few idiots amongst geniuses.

"Life is short, go large"

Mon, 14 Dec 2009 02:08:44 +0000

“Life is short, go large”

- Eli Thompson

Lemon Studios website is now up on Wordpress

Mon, 14 Dec 2009 01:56:49 +0000

Lemon Studios website is now up on Wordpress

I've decided to have a blog

Mon, 14 Dec 2009 01:55:33 +0000

Because everyone else seems to think it’s a good idea and there’s no point me bucking that trend. I’ve been told that Posterous and Tumblr are both fantastic to get up and running quickly, so this is my exploration into which one is right for me. What I’m looking for is a beautiful yet simple interface for users, and stupidly quick and simple for me to get content up. http://mattheworiordan.com/ @mattheworiordan