Bamboo Blog - Home

Stakeholder availability is a problem for any methodology, but Agile gets hit particularly hard. The User Stories Workshop is most vulnerable; although the rest of the process uses a single Product Owner, the USW needs representatives from all over the client organisation. This, combined with a typical half-day duration, makes it very hard to schedule.

Above a certain level in the management hierarchy, nobody's ever really 100% present in any case. At our last workshop, both high-level stakeholders had mobiles out, dealing with incoming emails that are more polite, but not much less disruptive, than incoming calls.

So we extended the coffee break to a fifteen-minute Blackberry Break. Those with urgent issues dealt with them. Everyone else had an out-of-band chat about how the process was going, and dealt with other todo items like external dependencies and scheduling.

We returned to the meeting with everybody focused. Going forward, we might even make this explicit: "there'll be a Blackberry Break every half hour". For product demos, we could go as far as "this is a fifteen-minute demo. Phones off, please".

Singletasking: it's the future.

A few months ago we transitioned from hosting all of our apps in development on two Slicehost slices to individual Amazon EC2 instances for each. As much you want vendor frozen Rails and Merb to work (including all your app's dependencies) it's always plagued with issues. Having completely separate environment for each app clears up a lot of issues, including the all to often mistake of upgrading Rails only to find some old project isn't able to co-exist with it. Our EC2 setup involved creating a base image with the basic Rails environment, and whenever a new staging server was needed we would clone the Base image and modify it specifically for the app. Being able to startup new environments with a single command is wonderfully easy, but a dozen staging servers later, the cost certainly starts to add up.

So for the cost of 1 month of our EC2 instances, we purchased a Dell box with 8Gb to run as an internal staging server and host all of our virtual servers. We chose Ubuntu as both the host and guest operating system. We started out trying KVM which is officially supported by Ubuntu. Everything went to plan and we had a great setup except for one major problem: slices would randomly lock up. After no success finding a remedy we decided to give Xen a go and with a little work got it setup and running along nicely.

Most of the instructions here have come from a combination of the Ubuntu and Virtuatopia wikis.

We have just released our most recent site, Five Fwd, for Channel 5. We have been working closely with Five for the past few months, developing the new site from scratch, which is the new online home of their highly popular Gadget Show.

Five wanted to create a site where people could catch recent broadcasts from the TV show, get more details about the features on the show, and explore a large collection of custom video content, produced exclusively for the web. As peoples' viewing habits change, traditional TV stations are going to be judged as much by their online presence as by their more traditional broadcasts. The goal was to extend the boundaries of the TV show into the web sphere. But the site goes beyond just putting videos online. It also contains a rich selection of the best gadgets featured in the TV show, allowing visitors to select the ones they want, and to get an up to date comparisons of the cheapest prices for any of the coolest gadgets on the show.

The project relied heavily on the our agile approach using Scrum and XP techniques, which we followed right the way though the development. We were able to send a team to work side by side with the producers at Five. With the client fully on board with the agile process, it really started to shine! Like any projects there were some hiccups along the way, causing scope to be modified, redesigns to take place, but our approach helped us handle all of these.

We sat down with our product owner, discussed the changes to the stories we had, presented options, and made the choices and tradeoffs available to him clear. Having such clear options presented to him made the choices almost easy to make. The result was, if we do say so ourselves, really cool. The site launched on time, ready to be promoted on the TV broadcast, and comfortably surfed over the corresponding wave of traffic. Of course we also maintained our commitment to the highest possible standards of coding, and even with the expected myriad of changes coming though in the few days before launch, we maintained 100% test coverage.

It couldn't have been better timed. Amazon has announced their forthcoming CDN service on the AWS Blog. Once released this service will be a perfect complement to Panda, allowing videos to be streamed to users even more efficiently!

You will start by storing your content in an Amazon S3 bucket and then marking the content as publicly readable. Next you'll make a single API call to register the bucket. The call will return a domain name that you'll use to refer to your content in your web page or application. When clients request the object via the returned domain name they'll be routed to the nearest edge location, for high performance delivery.

Earlier in the year, we found many of our clients were looking for a solution to allow users to easily upload their own videos. We had a good look into the numerous existing services already available, but ultimately found none quite fit the bill; either being too expensive, closed source or overly complex. Panda was conceived to be a simple open source solution for handling online video. Essentially, Panda allows you seamlessly integrate video uploading, encoding and streaming into your web application, while utilizing the power of the cloud.

We took inspiration from what was already out there and distilled it down to its core. Panda runs entirely on Amazon's Web Services utilising EC2, S3 and SimpleDB and the Merb framework. Once you've started up the application on an EC2 instance you can access the API and administration backend. The REST API allow painless integration with your own web application. Videos are uploaded directly to Panda and transcoded using FFmpeg to any format you desire. Video can be streamed using any Flash video player such as the JW FLV Media Player, and with the addition of h264 support in Flash 0.9.115 users can enjoy full HD quality.

So, visit the Panda site, grab the code from Github, read the Getting Started Guide and don't forget to join the Google Group if you have any questions or success stories to talk about!

We don't normally use the blog to announce the launch of client websites, but it might interest some people to check out some of the work we actually do. With this in mind, I'd like to introduce you to Protect The Human, a recently launched project for Amnesty International UK that we have been working on for the past few months in collaboration with our design peeps up in London Made by Many. Though it has been publicly available for a couple of weeks now, I wanted to let the dust settle a bit before posting.

So what is it? It is different things depending on who you talk to, but for me it is an assortment of different parts, playing a key role in Amnesty's online strategy to increasing activism (catchy, eh?). It is firstly a content publishing platform; populated by fantastic Amnesty videos, galleries of images about the work they do, and masses of bookmarks to fascinating and current content all over the interweb. But this is not purely a top down publishing platform, users of the site are actively encouraged to create their own content, and rate comment and discuss any item that is on there.

This is a site about activism: the small things people can do which collectively create interest for others and effect change on important issues. Users are prompted to take various actions at every opportunity, depending on the time they have available. Examples of actions can be signing online petitions for the victims of Human rights abuses, attending protests, or simply sending great content to friends to help spread the word. This aspect of the site has been a real eye opener, I have never before seen so many opportunities to take proactive action on such important issues in such an accessible way.

There are other parts to this site, such as hubs for the various campaigns Amnesty UK are engaged in, and rather than me describe it all, the best thing to do is to have a poke around. Because we worked in an Agile way, there are still more features which are going to be rolling out soon, and we hope to be iteratively improving the site over the next few months.

We will probably be talking about how the Agile process worked in practice on this at some point, but that will have to wait a bit. For the moment, have a look, sign up, and take some actions. There really isn't any excuse not to, the site makes it so easy to make a contribution, whatever that might be.

APML is an XML file format used to store the attention profiles of users, and it's got the potential to become a revenue stream for web 2.0.

A lot goes on here, believe me. Not everything is worthy of a full post though, so we've been running a tumblog internally for a little while. Bambinos is where all those useful code snippets and links go. If you enjoy the posts here I'd certainly recommend you subscribe.

Some recent picks from the archive:

• HangUp Meetings - An improvement on XP’s stand-up meeting...
• Picture Bong - Graphs for the Bong benchmarking tool...
• ManEC2 - Keep tracks of all those EC2 instances you have...

I've written a little lib which piggy backs on the standard merb form helpers to give you nicer looking forms with help from Andy and Martyn.

We weren't happy with our forms and we were writing too much code. So we wrapped some goodies around the merb form helpers. We've got notes, inline errors, required / not required formatting, and cancel buttons. The API is so minimal you won't be able to resist.

Here's an example of the ruby code, the generated HTML, and what it looks like with some sexy CSS:

Example Code

Ruby

<%= field(:text, :nickname, :required => true, :note => "You'll be known by this on the site. Don't use your real name. It has to be unique, and you can't change it later.")%>
<%= field(:password, :password, :required => true)%>
<%= field(:password, :password_confirmation, :required => true, :note => "Type the same password again, just to make sure we've got it right" )%>
<%= field(:checkbox, :terms_and_conditions, :required => true, :value => true, :label=>"I accept the terms and conditions")%>
<%= form_submit "Sign Up!" %>

Html Generated

<div class="field password required">
<label for="user_password">Password</label><input type="password" class="password" name=
"user[password]" id="user_password" />
</div>

<div class="field password required">
<label for="user_password_confirmation">Password confirmation</label><input type=
"password" class="password" name="user[password_confirmation]" id=
"user_password_confirmation" />

<p class="note">Type the same password again, just to make sure we've got it right
<abbr title="smile">: )</abbr></p>
</div>

<div class="field checkbox required">
<input type="hidden" class="hidden" name="user[terms_and_conditions]" value=
"0" /><input type="checkbox" class="checkbox" name="user[terms_and_conditions]" value="1"
id="user_terms_and_conditions" /><label for="user_terms_and_conditions">I accept the
terms and conditions</label>
</div>

<div class="field controls">
<button type="submit" class="positive">Sign Up!</button>
</div>

Html with Errors

<div class="field error password required">
<label for="user_password">Password</label><input type="password" class="error password"
name="user[password]" id="user_password" />

<p class="error">can't be blank and is too short (minimum is 4 characters)</p>
</div>

<div class="field error password required">
<label for="user_password_confirmation">Password confirmation</label><input type=
"password" class="error password" name="user[password_confirmation]" id=
"user_password_confirmation" />

<p class="note">Type the same password again, just to make sure we've got it right
<abbr title="smile">: )</abbr></p>

<p class="error">can't be blank</p>
</div>

<div class="field error checkbox required">
<input type="hidden" class="hidden" name="user[terms_and_conditions]" value=
"0" /><input type="checkbox" class="error checkbox" name="user[terms_and_conditions]"
value="1" id="user_terms_and_conditions" /><label for="user_terms_and_conditions">I
accept the terms and conditions</label>

<p class="error">must be accepted</p>
</div>

Sexy Form

You can be the judge of the sexyness of these forms, but they are much nicer than the default error message in my opinion.

Code/CSS

Code is licensed under the MIT License, and under no guarantee that it will not break if they change the external API for form helpers.

• For Merb and AR

• For Merb and DM

• The CSS

Install

Add the file in your lib folder, add the dependency in your init.rb and then include in your global helpers.

Enjoy!

You want to use the Twitter API but you don't want to die? I have the solution to API caching:

APICache.get("http://twitter.com/statuses/public_timeline.rss")

You get the following functionality for free:

• New data every 10 minutes
• If the twitter API dies then keep using the last data received for a day. Then assume it's invalid and announce that Twitter has FAILED (optional).
• Don't hit the rate limit (70 requests per 60 minutes)

So what exactly does APICache do? Given cached data less than 10 minutes old, it returns that. Otherwise, assuming it didn't try to request the URL within the last minute (to avoid the rate limit), it makes a get request to the Twitter API. If the Twitter API timeouts or doesn't return a 2xx code (very likely) we're still fine: it just returns the last data fetched (as long as it's less than a day old). In the exceptional case that all is lost and no data can be returned, it raises an APICache::NotAvailableError exception. You're responsible for catching this exception and complaining bitterly to the internet.

All very simple. What if you need to do something more complicated? Say you need authentication or the silly API you're using doesn't follow a nice convention of returning 2xx for success. Then you need a block:

APICache.get('twitter_replies', :cache => 3600) do
  Net::HTTP.start('twitter.com') do |http|
    req = Net::HTTP::Get.new('/statuses/replies.xml')
    req.basic_auth 'username', 'password'
    response = http.request(req)
    case response
    when Net::HTTPSuccess
      # 2xx response code
      response.body
    else
      raise APICache::Invalid
    end
  end
end

All the caching is still handled for you. If you supply a block then the first argument to APICache.get is assumed to be a unique key rather than a URL. Throwing APICache::Invalid signals to APICache that the request was not successful.

You can send any of the following options to APICache.get(url, options = {}, &block). These are the default values (times are all in seconds):

{
  :cache => 600,    # 10 minutes  After this time fetch new data
  :valid => 86400,  # 1 day       Maximum time to use old data
                    #             :forever is a valid option
  :period => 60,    # 1 minute    Maximum frequency to call API
  :timeout => 5     # 5 seconds   API response timeout
}

Before using the APICache you need to initialize the caches. In merb, for example, put this in your init.rb:

APICache.start

Currently there are two stores available: MemcacheStore and MemoryStore. MemcacheStore is the default but if you'd like to use MemoryStore, or another store - see AbstractStore, just supply it to the start method:

APICache.start(APICache::MemoryStore)

I suppose you'll want to get your hands on this magic. For now just grab the source from http://github.com/mloughran/api_cache/tree/master and rake install. I'll get a gem sorted soon.

This is the irb quickstart so that you don't have to re-parse the above:

require 'rubygems'
require 'api_cache'
APICache.start(APICache::MemoryStore)
APICache.get("http://twitter.com/statuses/public_timeline.rss")

Please send feedback if you think of any other functionality that would be handy.

RSpec stories are a way of doing integration and acceptance testing using plaintext executable tests. You can use them in Merb as well as Rails. Here's how.

Setting up

Install edge Merb; the latest gem (0.9.2) will not work. You need merb-core, merb-more, and merb-plugins.

Merb-plugins gives you the merb_stories gem, so you don't need to install that separately.

Add this line to your app's config/environments/test.rb:

dependencies "merb_stories", "webrat"

(Note that merb_stories' README file is wrong about this, for now - and the generator will create a dependency on merb-rspec, which no longer exists. My fork fixes this.)

Now generate your story:

merb-gen story mystory

Now run your story:

rake story\[mystory\]

Yes, you must include the square brackets, and you have to escape them.

Writing stories

Now fill out your story. There are some differences to Rails' versions. The best places to look for help are in the Merb code itself:

• spec/public/test/controller _matchers _spec.rb
• lib/merb-core/test/helpers
• lib/merb-core/test/matchers

To start you off, here are the steps for a simple integration test:

steps_for(:homepage) do
  When("I visit the root") do
    @mycontroller = get("/")
  end
  Then("I should see the home page") do
    @mycontroller.should respond_successfully
    @mycontroller.body.should contain("Hello") 
  end    
end

As you write your tests, don't trust Merb absolutely. Some things are wrong, don't work, or aren't meant to work [yet]. As part of debugging, look at the Merb source, and fork it and fix it if needed.

Adding Webrat

Webrat lets you write integration tests that are even closer to natural language. You can say things like:

visits '/auth/login'
fills_in 'username',:with=>"bob"
fills_in 'password',:with=>"hunter2"
clicks_button "login"
response.should be_successful
...

I've forked Webrat to add Merb support.

To get started, clone it, build the gem and install it.

You'll also need to make sure you're using the memory session store (for testing), or your sessions won't be preserved:

Merb::Config.use do |c|
  c[:session_store] = 'memory' 
end

Continuous integration

Autotest will run your specs continuously, but won't run your stories. You can fix this, but stories run very slowly, since they use the full stack. At the very least, though, you want to run your stories after you deploy, in case you have "works on dev machine, dies on production" problems.

Add this to deploy.rb:

namespace :deploy do
  task :after_deploy do
    run_remote_tests
  end
  desc "Run tests on remote server"
  task :run_remote_tests do
    run "cd #{deploy_to}/current && rake spec"
    run "cd #{deploy_to}/current && rake story[all] MERB_ENV=test"
  end 
end

This will fail if you're using Vendor Everything like us - it won't be able to find merb-core. To fix it, replace stories/stories/all.rb:

env = ENV['MERB_ENV'] || 'test'
require 'rubygems'
Gem.clear_paths
Gem.path.unshift(File.join(File.dirname(__FILE__), "..","..","gems"))

require 'merb-core'
Merb.load_dependencies(:environment => env)

require 'spec'
Merb.start_environment(:testing => true, :adapter => 'runner', :environment => env)

dir = File.dirname(__FILE__)
Dir[File.expand_path("#{dir}/**/*.rb")].uniq.each do |file|
  require file
end

You'll have the same problem with specs. Edit spec/spec_helper.rb:

Gem.clear_paths
Gem.path.unshift(File.join(File.dirname(__FILE__), "..","gems"))

Now you have natural-language integration tests that run automatically on deployment. Sweet.

Here we are going cover the install instructions for Merb, RSpec, and DataMapper (0.9) and how to create a bare application. The next post will cover aspects of the framework and introduce the example application we will be building.

Here is the next part of the Merb, DataMapper, RSpec book. I was planning on releasing more today but I am rewriting the examples for DataMapper 0.9. There is a example app for DataMapper 0.3 in the git repository of the book, if you can't wait to play with it.

When I started learning Merb and DataMapper I kept a collection of notes to help me keep up with these projects. These grew to the point that they couldn't fit nicely into a single text file. With contributions from others I started to put together a small book on developing web apps with Merb and DataMapper. I have decided to release it as we go. There is also a project in GitHub if you want to check it out and contribute.

So here we go (corrections and comments welcome):

Life On The Edge With Merb, DataMapper & RSpec

We're at Euroko in Prague, with some of the guys at New Bamboo. It's been pretty good so far (crippling flu aside) and Prague is a lovely city.

I gave a talk today on Aspect Orientated Programming in Ruby using Aquarium, it quickly covered what I mentioned in my last post on AOP, and went into more detail with actual code examples this time!

The slides are on slideshare, the demo code is currently in a git repo . The demo app is Merb/Datamapper and I'm currently trying think of more examples of cool aspects to put in there, feel free to add some.

If anyone saw my talk, I'd love to hear what you thought of it so drop me a line matt@new-bamboo.co.uk or leave a comment on this post.

<object height="355" width="425"><param /><param /><param /><embed src="http://static.slideshare.net/swf/ssplayer2.swf?doc=aspect-orientated-programming-in-ruby-1206886984742805-4" height="355" width="425"></embed></object>

| View | Upload your own