Relevance, Inc. | Agile Software Design & Development

Rethinking PDF Creation in Ruby

Every now and then, a requirement will come up in a project, that will make me second guess my career choice as a programmer. It usually involves making me go through tedious exercises, never knowing if I'll end up where I want to be along the way.

This happened a couple of weeks ago when one of our projects called for generating PDF reports. The reports needed many stylized elements, layouts, and dynamic graphs. If you've ever generated PDFs in Ruby before, you know that it can be both tedious and difficult using the standard go-to PDF libraries out there. Let's face it, we're web developers. Coming from HTML+CSS-based layouts, writing Ruby code for that stuff is a major pain.

To give you an idea of how heavy it can get, here's an example taken from Prawn. The example was ironically called simple_table.rb.

  Prawn::Document.generate("simple_table.pdf") do 

  table([["foo", "bar " * 15, "baz"], 
         ["baz", "bar", "foo " * 15]], :cell_style => {:padding => 12}) do
    cells.borders = []

    # Use the row() and style() methods to select and style a row.
    style row(0), :border_width => 2, :borders => [:bottom]

    # The style method can take a block, allowing you to customize 
    # properties per-cell.
    style(columns(0..1)) { |cell| cell.borders |= [:right] }
  end

  move_down 12

  table([%w[foo bar bazbaz], %w[baz bar foofoo]], 
        :cell_style => { :padding => 12 }, :width => bounds.width)

end

If you're scratching your head at this point, the code above generates a PDF with two simply styled tables. That's it. If you asked me to implement this in an app, I might have something half-way presentable in an hour. But, I could get a monkey, who just drank a whole bottle of scotch—don't ask about his drinking problem—to write two tables using HTML in less than 5 minutes.

A New Hope

Now some of you may be familiar with PrinceXML, which is a command line utility that will take HTML+CSS and give you back a beautiful PDF. It's even CSS2 compatible and passes the ACID2 test. Awesome. The only problem is that a single server license will set you back $3,800—which is prohibitively not awesome.

Being the open source zealots we are here at Relevance, we set out to find another solution. Tucked away in the internets, we stumbled across wkhtmltopdf. I know what you're thinking; awesome name, huh? wkhtmltopdf uses a WebKit rendering engine to make pretty PDFs out of HTML+CSS. Since it's leveraging WebKit, you get all the tasty CSS3 properties it supports. Ugly PDFs are suddenly a thing of the past.

Goodbye Prawn, Hello PDFKit

We were surprised that none of us had ever heard of wkhtmltopdf, considering how useful it is. When we looked for a Ruby library that leveraged it, we realized it didn't exist. Apparently not a whole lot of other people had heard of it either. That couldn't stand. A couple of open-source Fridays and several gallons of Mountain Dew later, we're excited to announce PDFKit, an open source library that makes working with wkhtmltopdf a snap.

Usage

Inline HTML+CSS => PDF

  kit = PDFKit.new("<h1>Oh Hai!</h1>")
  kit.stylesheets << '/path/to/pdf.css'
  kit.to_pdf # inline PDF
  kit.to_file('/path/to/save/pdf')

HTML file => PDF

  html_file = File.new('/path/to/html')
  kit = PDFKit.new(html_file)
  kit.to_pdf # inline PDF

Remote HTML => PDF

  kit = PDFKit.new("http://google.com")
  kit.to_pdf # inline PDF

What's the big deal?

If this hasn't sunk in yet, let's go over a quick list of wins this buys us:

1. HTML+CSS - Assuming you're a web developer, there's a good chance that you already know HTML and can work with it efficiently.
2. CSS3 - We get WebKit's CSS3 support for free. This means effects like drop shadows, rounded borders, transformations and others are super-easy. (Note: effects requiring blur radius do not work.)
3. Testing - We have tools built into our normal workflow for testing HTML. You can even use Cucumber to drive the development of a PDF with PDFKit.

To give you an idea of how well this fits into our normal workflow here at Relevance, this is how we built out our PDF reports:

1. Our designer mocked up a sample PDF and converted it to HTML+CSS.
2. Using Cucumber to drive development, we created a controller action to generate this HTML view of the PDF. (It was just another URL in our app.)
3. We added a screen-only stylesheet to the HTML that mimics the look of a PDF reader. This allowed us to get a feel of how it would look as a PDF.
4. Using a bit of Rack Middleware that ships with PDFKit, we can get the PDF version of that web page by simply appending '.pdf' to the url.
5. We're done. No crazy extra class to handle PDF rendering. No need to spend all day reading through docs to learn the obscure code and magic incantations required to generate your PDF.

Samples

• PDF of google.com - PDF rendered from http://google.com
• CSS3 Examples - Sample rendering of common CSS3 effects including border-radius, text-shadow, box-shadow, and border-image. Notice the lack of a blur radius on text-shadow and box-shadow.
• Sample HTML page with PDF viewer CSS - Example of using a single HTML source to render both a screen version and a PDF version. Uses a media="screen" and media="all" to mark relevant CSS.
• PDF generated from PDF viewer HTML - PDF generated from sample HTML above. You must tell PDFKit to only use print stylesheets in order to achieve this effect (PDFKit.new(html, :print_media_type => true)).

Go Forth and PDF

I encourage you to take PDFKit for a spin, let us know what you think, and even submit some patches.

If you are pumped about the possibility of using PDFKit on a future project, then I've achieved my goal. If not, I'd ask you to think about what is missing, find out if it's already out there, and let us know how to make PDFKit even better.

10 Must-Have Rails Plugins and Gems (2009 Edition)

When Paul Graham wrote that the "list of n things" is a degenerate case of the essay, our first thought was "Wow! That's for us!" And we're going one step farther: we're recycling an old "list of n things" essay from last year.

Seriously, we've been thinking of revisiting 10 must-have Rails plugins for a while now. There is a place for lists like that, and the Rails plugin and add-on space has been moving quickly. We are always looking for better ways to do things, so we try out a lot of the plugins that come along. Our list of favorites---the ones that we use on almost every project---is almost completely different than last year's model.

There's one important change in focus: the plugins and gems that are solely related to testing are gone from this list. Of course, that doesn't mean we're down on testing. On the contrary, we built RunCodeRun because we think testing is so vital. We're saving the testing tools for the RunCodeRun blog; we'll be writing another degenerate essay there as a counterpart to this one.

There are numerous other plugins we use for special needs, such as PDF generation or attachment handling. But our favorites are the ones that we use on almost every project. So here they are, along with brief comments explaining why you want to check them out:

• Inherited Resources: eliminates most of the boilerplate code from our controllers. (The new controller responder feature in Rails 3 is similar in intent.)
• Formtastic: takes most of the pain out of writing the markup for HTML forms. (Together, Inherited Resources and Formtastic make a nice alternative to scaffolding frameworks like Streamlined and ActiveScaffold.)
• CapGun: provides easy build notifications (see this previous post for more info).
• Faker: helps us generate fake data. We use it for testing, but mostly for providing demo data for development and staging environments.
• Clearance: feature-rich authentication and signup.
• Safe ERB: helps ensure that our apps are not vulnerable to cross-site scripting attacks. (We look forward to similar functionality being baked into Rails 3.)
• RedHill on Rails Core: we use this primarily to declare foreign key references in our database schemas. Telling the database about table relationships adds a small cost to our projects, but we've found that the benefits outweigh that cost. (It's unclear where this plugin lives at the moment, but there are numerous forks of it on GitHub.)
• RPM: Rails Performance Management from New Relic; wonderful for discovering and diagnosing performance problems.
• will_paginate: the nicest, easiest pagination plugin we've seen.
• hoptoad: great, customer-friendly notifications about exceptions that happen in the app.

Don't reinvent the wheel! Use these plugins (or others like them), and definitely consider contributing to them if they fall short of what you need!

Quick and Easy Logging with LogBuddy

Good logging is crucial for effective problem diagnosis in production. Plus, easy logging remains a terrific debugging technique during development. As helpful as "real debuggers" are, sometimes a debugging log statement is exactly what you need to find a problem quickly.

LogBuddy is a little gem that makes good logging easier. It helps even in Rails, which already has good logging support, and it's a bigger help when building gems and standalone Ruby apps, where you have to start from scratch with logging.

What does LogBuddy do for you that Rails doesn't already? There are numerous small features, but here are the big ones:

1. It ensures that logger is available everywhere, not just in classes that extend parts of the framework.
2. It makes it extremely easy to add informative debugging messages with annotated output and full exception info.

Installing and Adding LogBuddy to Your Project

To install the latest release of LogBuddy, just install the gem:

$ gem install relevance-log_buddy --source http://gems.github.com/

Then add a require 'log_buddy' statement to your app. If it's a Rails app, it's best to add this to environment.rb:

  config.gem 'relevance-log_buddy',
             :source => "http://gems.github.com/", 
             :lib => "log_buddy"

Finally, initialize LogBuddy. In Rails apps, we usually put this in config/initializers/logging.rb:

  LogBuddy.init

(You can pass some options to LogBuddy.init; we'll get back to that in a bit.)

LogBuddy creates and initializes a logger for the app to use. In Rails apps, it simply uses RAILS_DEFAULT_LOGGER unless you tell it differently.

Using LogBuddy

LogBuddy mixes a couple of methods into every object in the system: logger and d. Here's how they work.

The logger method is no surprise at all. It simply returns the Logger instance, and you can log by calling debug, info, warn, error, or log methods on it. LogBuddy's logger doesn't usually do anything special; the benefit is that, since it's mixed into Object, it's available everywhere, automatically. (In a typical Rails app, there are numerous contexts where logger doesn't work, and you have to explicitly use RAILS_DEFAULT_LOGGER.)

My personal favorite LogBuddy feature is the d method. Like logger, it's available everywhere. But the d method is designed just for debugging messages. You can call it with an explicit string, or with some object you want to see the value of:

  d "FINISHED PARSING"
  d some_exception
  d result

Strings are logged the same way logger.debug would do it. Exceptions are logged with all of the information you might want: the message, exception class name, and backtrace. Finally, if you pass any other object, d calls that object's inspect method and logs the resulting string.

Where d really shines is when you want to log several values at once, with annotations to distinguish them. Just pass a single-line block to d, like this:

  d { first; current; last}

That produces these three log lines:

  first = "foo"
  current = "bar"
  last = "baz"

The values you log can be any Ruby expression:

  d { 3; name; @model; RAILS_ENV; options[:limit] }

and you'll get just what you want out of that:

  3 = "3"
  name = "primary"
  @model = #<Contact id: 14, name: "Joe">
  RAILS_ENV = "development"
  options[:limit] = 5

There are some restrictions if you use this feature. The entire call to d must fit on one line, and you must use the curly-brace style of block, rather than the do/end style. Finally, if you want to log multiple values, separate them with semicolons, not commas.

(You may be wondering how LogBuddy accomplishes that trick. The answer is left as an exercise for the reader ... especially since there are some hints in the restrictions just mentioned. Of course, you can always read the source.)

LogBuddy Initialization Options

The LogBuddy.init method takes an options hash. Here are the permissible options:

• :logger -- you can supply a logger instance for LogBuddy to use. If you don't supply one, LogBuddy uses RAILS_DEFAULT_LOGGER if it's defined; otherwise, it creates a new logger that writes to standard output.
• :log_to_stdout -- by default, messages from the d method are logged (using logger.debug) and also written to standard output. Set this option to false to only use the logger.
• :disabled -- set this option to true to turn off the output from the d method. It's common to set it this way:
  LogBuddy.init :disabled => Rails.env.production?
• :log_gems -- if you set this option to true, LogBuddy watches gem activation and logs information about each gem. This can be useful for tracking down gem activation errors

Try It Out!

LogBuddy is really easy to set up, and then it's there when you need it most: when you're focused on a problem and just need to get the details quickly. Please try it and let us know what you think!

Easy Build Notifications with CapGun

Most of us get notified about lots of little events on our projects: Commits, build failures (and fixed builds), and runtime exceptions, for example. But deployment is where the rubber meets the road on web development projects. Deployment to staging means new functionality to try out and test, and of course deployment to production is even more important.

CapGun is a gem we use to send email notifications whenever we deploy one of our projects. It works with Capistrano and uses ActionMailer to let every interested party know when a deployment happens.

Here's how to work with CapGun in a Rails project. Add this to config/environment.rb:

  config.gem 'relevance-cap_gun', 
             :source => "http://gems.github.com/", 
             :lib => "cap_gun"

Then install and unpack the gem:

  $ rake gems:install
  $ rake gems:unpack

Finally, edit your deployment script (usually config/deploy.rb) and add this:

  require File.join(RAILS_ROOT,
                    Dir["vendor/gems/relevance-cap_gun-*/lib"].last,
                    'cap_gun')

  set :cap_gun_action_mailer_config, {
    :address => "smtp.gmail.com",
    :port => 587,
    :user_name => "grey-goo@example.com", # deploy bot email address
    :password => "outtacontrol",  # deploy bot email password
    :authentication => :plain 
  }

  set :cap_gun_email_envelope, {
    :recipients => %w[fooproj-devs@example.com fooproj-mgr@example.com], 
    :from => "Foo Project Deployment Bot <grey-goo@example.com>"
  }

  after "deploy:restart", "cap_gun:email"

We usually use GMail as our MTA for this purpose, and CapGun includes support for secure, authenticated email connections so that you can do the same. It's best to create a special-purpose, throwaway email account just for things like this; that way there's little risk in putting the email password in the project deployment script. (But there's also nothing stopping you from reading it from another file that's not checked into source control, just like you would do with your production database passwords. The deploy.rb file is just Ruby code, after all. And if the source is going to be in a public repository, you should definitely do that.)

The deployment email has a brief summary at the top (in fact, the subject often tells you all you need to know). But there are useful details in the body. Here's an example of what you might see from the configuration above:

From: Foo Project Deployment Bot <grey-goo@example.com>
To: fooproj-devs@example.com, fooproj-mgr@example.com
Date: Tue, 15 Sep 2009 17:27:57 -0400
Subject: [DEPLOY] fooproj deployed to production

fooproj was deployed to production by george at September 15th, 2009 5:27 PM EDT.

Nerd details
============
Release: /var/apps/fooproj/releases/20090915212721
Release Time: September 15th, 2009 5:27 PM EDT
Release Revision: 69246739f2a1cbcef5ca76f1842c21849db7778a

Previous Release: /var/apps/fooproj/releases/20090915194707
Previous Release Time: September 15th, 2009 3:47 PM EDT
Previous Release Revision: 9ceb19b1777470d1d5133f433fdd5d1873c7a4c0

Repository: git@github.com:foocorp/fooproj.git
Deploy path: /var/apps/fooproj
Domain: fooproj.example.com
Branch: main

Commits since last release
====================
d37a15c:who put a blink tag in here?
f1b603e:Added favicon

Setting up CapGun only takes a few minutes, and you'll love the increased visibility into your team's deployments. (And even if you're a team of one, you'll appreciate having the emails as a record of your deployments.) Try it out, and let us know what you think!

Working With Multiple Ruby Versions Has Never Been This Easy

The Ruby VM market is thriving, and we increasingly find ourselves spending the morning in JRuby and the afternoon in Ruby Enterprise Edition. Or Thursday using Leopard Ruby and Friday using Ruby 1.9. Or ... well, you get the idea. So we needed a lightweight tool that eliminates the cost (in both time and frustration) of switching between Ruby versions. We wanted something drop-dead simple. Meet the Ruby Switcher.

Hello, Ruby Switcher

The Ruby Switcher debuted earlier this year as a means for toggling between a handful of pre-installed Ruby versions. And once you bit the bullet and manually installed those VMs, and assuming you installed them in the "right" location, the Ruby Switcher offered crazy-fast switching between those few Ruby versions. But we're passionate about continuous improvement and maintaining a sharp set of tools, so we recognized that switching Ruby versions was Step 2; Step 1 (i.e., installing those Ruby versions in the first place) needed a healthy dose of automation as well. Today's Ruby Switcher makes both steps insanely simple, and it adds support for more (and newer) Ruby versions as well.

Pick a Ruby, Any Ruby

Enough talk! Let's download the Ruby Switcher and get this party started.

  cd
  curl -O -L http://github.com/relevance/etc/raw/26ae85c2f6c7d2640a3c75d619ad7ab8fc1cc570/bash/ruby_switcher.sh
  echo "source ~/ruby_switcher.sh" >> .bash_profile
  source .bash_profile

That's it. Let's quickly verify that you have your platform's core developer tools installed, and then you'll be ready to put the Ruby Switcher to work.

Prerequisites - Compilers, Libraries, and Whatnot

If you're on OS X, you'll need to download and install Xcode. (If you just want to switch between Leopard Ruby and JRuby, you can skip this step. If you want to use any other versions of Ruby, you'll need Xcode.)

If you're using Ubuntu, use apt-get to grab a few essential packages:

  sudo apt-get update
  sudo apt-get install build-essential zlib1g-dev libreadline5-dev libssl-dev

For other *nix variants ... well, clearly you enjoy figuring this stuff out.

Ruby 1.9

Yehuda Katz recently asked the Ruby community what we need in order adopt Ruby 1.9. Having an easy way to experiment with Ruby 1.9 is surely a good place to start. What could be easier than a single command?

  install_ruby_191

The installation will take a few minutes. In the meantime, why not check out Bruce Williams' slides on Ruby 1.9 from last year's Lone Star Ruby Conference — Ruby 1.9: What's New and Why it Matters?

  ...
  checking build system type... i686-pc-linux-gnu
  checking host system type... i686-pc-linux-gnu
  checking target system type... i686-pc-linux-gnu
  checking for gcc... gcc
  ...
  installing rdoc
  Using ruby 1.9.1p129 (2009-05-12 revision 23412) [i686-linux]
  Successfully installed rake-0.8.7
  1 gem installed

By the time you're done reading through the Ruby 1.9 highlights, you'll be ready to take it for a spin. Type ruby -v to verify that you're rockin' with 1.9.1.

  ruby 1.9.1p129 (2009-05-12 revision 23412) [i686-linux]

JRuby

Installing and using JRuby is just as easy.

First, be sure that you have the JDK installed. If you're using OS X, you're all set. If you're on Ubuntu, you can install it with apt-get. (For other Linux distros, you can ask your native package manager for the JDK bits.)

  sudo apt-get update
  sudo apt-get install sun-java6-jre sun-java6-jdk

With the JDK in place, installing JRuby is a cinch.

  install_jruby

Verify the results with ruby -v.

  jruby 1.3.1 (ruby 1.8.6p287) (2009-06-15 2fd6c3d) (Java HotSpot(TM) Client VM 1.6.0_10) [i386-java]

Wait a minute: shouldn't we have typed jruby -v instead of ruby -v? Sure you can do that if you really want to. But we're not big on having to remember to type jirb instead of irb, or jgem instead of just gem, etc. Once we tell the shell to use JRuby, we want the normal Ruby commands to just work. The Ruby Switcher ensures that they do.

Ruby.*

Looking for yet another Ruby version? See what else the Ruby Switcher has to offer:

• install_ree_186
• install_ruby_186
• install_ruby_187
• install_jruby_120

And if you find yourself wanting some other Ruby variant, be sure to check out the Ruby Switcher's internals; you can likely adapt one of the existing installation functions to meet your exact need.

Willy-nilly Switching

Once you have the desired Ruby versions installed, switching between them couldn't be easier. Each of the install commands comes with a corresponding use command to instruct your shell to switch to the specified Ruby version:

And these commands are shell-specific! So while one terminal window is using Ruby 1.8.7 to run your front-end Rails app, you can have another terminal using JRuby to run your back-end messaging code.

Ruby 1.9, the Community, and You

Want to see whether your app runs on Ruby 1.9? Flip your shell to use_ruby_191 and try it out. It's that kind of experimentation that will allow the Ruby community to make the migration to Ruby 1.9.

See a problem with a certain gem on Ruby 1.9? Let the gem author know about it and chime in at isitruby19.com. Better yet, write a test that proves the bug and pass that test along to the author. Better still, fix the issue and submit a patch. Any one of these steps moves us forward, and all the while you've still got your default Ruby installation standing safely by for your day job.

When switching between Ruby versions is this seamless, there's no reason not to experiment.

Notes

[1] Ruby is installed by default on OS X Leopard, so there's no need for an install_leopard_ruby command.

Image courtesy of bdu (flickr.com/bdu). [Creative Commons License]

Micronaut: Innovation Under the Hood

Last week, Rob announced Micronaut, our new BDD framework written by Chad Humphries. I’ve been itching to write about Micronaut as well, and my take on it is similar to Rob’s, although I start with a very different perspective.

It’s a little surprising that I’m excited about Micronaut. I’m sort of a Ruby old-timer, and I’ve never been that excited about RSpec or any of the other Ruby test frameworks that have appeared over the last few years. Yes, they offer some advantages in some cases, but I’ve always seen the improvements as incremental rather than revolutionary. I like RSpec’s facilities for structuring tests, but should never struck me as a radical improvement in expressiveness. In fact, I frequently run into situations where I think a well-crafted assertion is much clearer than a should expression.

I just never got the BDD religion, in other words.

What I like about Micronaut is that its innovation is mostly under the hood, rather than on the surface. Rather than designing some new variation on how we express our tests, Chad opted for RSpec compatibility and built a really nice engine for loading and running RSpec-style tests. Architecture matters, and Micronaut’s architecture is a joy.

First of all, as Rob noted, Micronaut is small, easy to understand, and fast. That makes it fun to hack on, and great to use on projects. It also gives me confidence; simple tools tend to be more robust, and I want that in my testing tool.

Second, Micronaut’s architecture provides a lot of flexibility. To illustrate that, while attending talks at QCon a couple weeks ago, I added test/unit compatibility to Micronaut. Here’s a short example that mirrors Rob’s example from last week, but in a test/unit style:

  require 'micronaut/unit'

  Micronaut.configure do |config|
    config.filter_run :focused => true
  end

  class MicronautMetadataTest < Micronaut::Unit::TestCase

    # 'testinfo' allows declaring metadata for a test, and works
    # like Rake's 'desc' method: it applies to the next test.

    testinfo :focused => true
    def test_this_will_definitely_run    
      assert self.running_example.metadata[:focused]
    end

    def test_this_never_runs
      flunk "shouldn't run while other specs are focused"
    end

    testinfo :pending => true
    def test_this_is_pending
    end

    testinfo :speed => 'slow'
    def test_too_slow_to_run_all_the_time
      sleep(10000)
      assert_equal 4, 2+2
    end
  end

I don’t know how interesting test/unit compatibility is in its own right. I certainly wouldn’t claim that the example above is superior to the RSpec equivalent. But it was a fun exercise to really prove Micronaut’s design. It only took a few hours (of partial attention) and it clocks in at under 200 lines of code (plus `assertions.rb` from the test/unit codebase). I love the idea of the testing API and the test execution engine not being so tightly coupled. And it might be very useful if you have existing tests written with test/unit and you’d like to gain the benefit of Micronaut’s metadata support.

Which brings me to Micronaut’s best feature. Almost all unit-testing frameworks incorporate some support for test suites, but suites have never lived up to their promise. It would be nice to have tests organized in suites for distinct purposes, but the setup and maintenance costs associated with suites have meant that very few teams make good use of them. Micronaut’s metadata is a different—and much better—solution to the same problem. As some have noted, the idea is similar to Cucumber’s support for tags. Just attach metadata of your choice to Micronaut tests or examples, and then you can use that metadata to select what tests are run in different situations.

At the moment, test/unit compatibility is only available in my fork of Micronaut. It allows using RSpec-style describe and it blocks alongside test/unit-style test methods, and supports test blocks à la Context and ActiveSupport. I don’t think we’ll pull this completely into Micronaut; I think it would work better as a separate gem. But I’d love to get your feedback about the whole idea.

RunCodeRun at the Professional Ruby Conference

I will be bringing RunCodeRun Beta invites to the Professional Ruby Conference in Boston next week. If you are attending the conference, and want to get your open source Ruby project configured for free Continuous Integration on RunCodeRun, come and see me.

If your build goes green, I will have a T-shirt for you (while supplies last).

Once you go green, you don't go back.

Building the Enterprise Hammer, Part 3

In this installment we are going to dig further into Idea #3 from Building the Enterprise Hammer:

The transition from EJBs to POJOs is only a halfway point. We now need to leave Plain Old Static languages (POSs) behind.

Developers have already rejected "special hammers" (past versions of EJB), in favor of plain old hammers. This is a great step towards more flexible applications. However, they still typically mount their hammers in some kind of frame (e.g. Spring). What happens when this frame is not perfectly adapted for the task at hand, or when the task at hand changes?

Let's say, for example, that we wanted to repurpose our Enterprise Hammer as an Enterprise Window Washer, by installing different attachments. We believe that this should work because:

• Windows are regularly spaced on the side of the building.
• Window spacing has a known relationship to the spacing of nails hammered by the Enterprise Hammer.

This is an example of refactoring to generality: finding similarities so that you can cheaply provide a new function (washing windows) through the same mechanism that provided the original function (driving nails).

Java makes this kind of generalization difficult. Java encourages up-front rigor about the type of an object. If you decide later than the "Hammer" type can be generalized to "Widget", you may have to touch hundreds or thousands of lines of code to enact that change. Duck-typed languages, such as Ruby and Python, encourage less assumptions. And, the assumptions are easier to change later. This is one place where the 10x hype is real.. Duck-typed languages can be an order of magnitude more productive, in both developer time and lines of code, when designing for (or refactoring to) generality.

There's another issue here, as well. Our Enterprise Window Washer may require changes to the frame itself, since windows are spaced differently than nails, and washing requires a different motion than hammering. But, we didn't build the frame. What are our options? Many languages provide a simple solution through open classes. With open classes, our own application code can make spot modifications to the frame to suit our specific needs. Conversely, Java discourages this flexibility in two ways:

• deliberately, by closing classes
• inadvertently, through its complex deployment model

As a result, Java developers may learn to not even see opportunities for reuse. Elegant solutions are not only not chosen, they are not even considered.

But More Flexible Languages are Dangerous!

Yes. There are many good reasons that Java became dominant, instead of e.g. Smalltalk. Understand those reasons, and you'll also know why it's time for a change. Stay tuned for the next installment.

Some Numbers at Last

So, a few weeks ago I made an offhanded post here about my new-found love for Rails. I'd been skipping off the surface of Ruby for a while, trying to decide if it, or Python, or Groovy, or something else, ought to fill out the empty slot in my tool belt. (I'll save the "why LISP isn't on this list" post for another time.) Rails seemed like an excellent way to put Ruby through a workout, and I had the right sized project to try it out with.

The project itself is not open-source; the client is now and shall remain anonymous. But they are paying me my going rate to do this work, which makes it a commercial Rails project, and it will in the future be installed into some rather large organizations. I can't really say much about what the application's domain is, but I can lay out the general principles of the app.

The application must support multiple users, in the order of 10-100 concurrently. The load isn't particularly high, but the application will be treated like a desktop app (more specifically, a spreadsheet replacement) so it has to be responsive. The application is for managing pieces of a large machine process. There are lots of types of components to be managed, and the relationships between them can be quite complicated. There are no one-to-one relationships in the model, but plenty of one-to-many and many-to-many. In addition to managing the components, the application has to allow for the addition of entirely new categories of components as well as a variety of customizable fields. Finally, the authorization rules call for a breadth of user roles with a complex mapping of permissions to those roles.

I've finally gotten around to running the profiling numbers and doing some comparison between the two systems. I won't spoil the suspense by listing my conclusions up front -- you'll have to scroll through the rest of the post to see them. But, first, let me set the stage: the original application is based on the Java/JSTL/Spring/Hibernate/MySQL stack. I used ACEGI for the security, SpringMVC for the web controller, and was using Hibernate 2.x for the O/RM. To increase performance, I was using the default output caching from SpringMVC and data caching in Hibernate. The re-implementation is in Rails on top of MySQL. The authorization is done through a custom observer and a custom authorization class, which uses a declarative rules file for permission mapping. For performance, I'm using a combination of page caching, action caching, and cache sweepers (observers whose job it is to expire cached pages).

Now, for the comparisons:

Time to Implement I made a comment about this in the previous posts on the topic, and that comment has been quoted widely out in the wide blogosphere as a classic example of Rails hype. So, let me make it plain: any time you re-write an application, it will go almost infinitely faster because you already have a firm grasp on the problem domain. Such was the case for me in my re-write; we'd spent so much time on the domain model and the database schema that the second time through the application, everything already made perfect sense to me. Any comparison of how long it took to implement one or the other is bogus, since the only fair comparison would be to implement two roughly functionally equivalent projects in the two different stacks and measure the competitive results. Since I have not done that, making statements about how it only took 5 days to re-implement the site are almost meaningless. What I can say is that I had more fun implementing it the second time, but that's just personal preference.

Lines of Code This one is a lot more interesting. Folks will tell you all the time that there is a running debate about the meaningfulness of LOC comparisons. They're right; there is a running debate. I just think it's moot. For my money, the fewer lines of code I have to write to get a piece of functionality, *as long as those lines are clear in meaning*, the better off I am. Obfuscated Perl programs don't make the grade; I can write some really concise Perl code and not have any idea, three months later, what the heck I was doing. But if the code is legible, its intent obvious, then more concise is better, pure and simple.

Bear in mind that this comparison is somewhat unfair. The Rails version of the app has been in development for an extra month (meaning a month's worth of new features added) while the Java version has been stagnant since the switch. Since the Rails version started out as an experiment, I don't have a clear history of the codebase to be able to produce a version that is identical in features to the Java version. Therefore, I've made the comparison based on the current state of the app vs. the abandoned Java version.

The LOC counts used here do not include comments or unit tests, nor do they include simple punctuation lines (lines with just a "}" or "end").

Without further ado:

• Lines of Code
• Rails: 1164
• Java: 3293

Number of Classes:

• Rails: 55
• Java: 62


Number of Methods:
• Rails: 126
• Java: 549


Configuration Lines
• Rails: 113
• Java: 1161

Those numbers are beyond striking. The configuration count alone is enough to make me think long and hard about what I've been doing lately. Let me forestall any criticism about my "agenda", by the way. Somebody recently said that I must have an "anti-Java" or an "anti-Spring" agenda. That is far from the truth, since my Spring: A Developer's Notebook hits shelves any day now. I don't want people to stop using Spring, or Java. In fact, I want lots and lots of Java developers to use Spring and lots and lots of non-Java developers to start using Java so they can start using Spring. But one of the things I like most about Spring (its concise configuration) is still, well, huge, compared to the same app in Rails.

Performance This is where everybody really wants to see the numbers. So, for the sake of total specificity, the following numbers were generated on a 1.5GhZ Mac OSX (10.3.7) PowerBook with a 4200rpm hard drive and 1GB of RAM. The Java app is running on Jakarta Tomcat v 5.0.28, while the Rails app is running in Lighttp with FastCGI. The setups are standard for each application stack.

Since both stacks have different caching mechanisms, with different degrees of difficulty to manage them, I'll start the performance comparison with a walk through the application without using the caching systems. To generate the first number, I walked through every screen in the application (using the screens available in the Java version to form a subset of the screens available in the newer Rails version) hitting pages that have not been cached yet. This starts from logging in, then hitting every available piece of functionality once. These numbers do not include the time it took me (the user) to navigate to the next request, only the time to process the requests. Both applications had roughly equivalent logging turned on (obviously, it can't be exact, but without the logging, I couldn't provide measurements). Here's what I found out.

To walk the entire application feature set, once, in Rails, without caching, took 41.801s. To walk the exact same feature set in the Java app took 58.369s. These numbers are averages over five attempts with each app, with full restart between to give the cleanest runs possible. Those summary numbers are deceiving, though. What I found was that, the less complex the feature, the faster the Java app served it relative to the Rails app. The more complex the feature, the slower the Java app served it relative to the Rails app. Some of that difference might be changes to the model during the re-implementation based on a better understanding of the domain. Regardless, that difference comes out to show the Rails app at ~30% faster than the Java version when caching isn't taken into account. If I am generous and say that half of the difference is due to optimizations in the model, that still leaves us a 15% better performance in Rails.

The caching story is interesting. The default caching mechanism for Rails is either page caching (caching the output of the rendering to a physical html file and letting the server serve it directly instead of invoking Rails the next time it is requested) or action caching (similar, except the output is cached in memory or another kind of store and Rails is invoked to process the request). On the Java side, there is JSP pre-compilation, Spring output caching and of course Hibernate data caching. I used a simple load tester to test a couple of pieces of functionality in the application. For each URL, I ran the test without a pre-existing cache of the response, then again with it, 100 times each, and then determined the number of requests per second.

Functionality 1: Rails, 100 runs, no pre-existing cache: 75.59 request/second. Rails, 100 runs, pre-existing cache: 1754.39 requests/second. Java, 100 runs, no pre-existing cache: 71.89 requests/second. Java, 100 runs, pre-existing cache: 80.06 requests/second.

Functionality 2: Rails, 100 runs, no pre-existing cache: 62.50 r/s. Rails, 100 runs, pre-existing cache: 1785.15 r/s. Java, 100 runs, no pre-existing cache: 80.06 r/s. Java, 100 runs, pre-existing cache: 88.97 r/s.

These numbers are not a great comparison, because there is tons more I can do to increase the cache performance of the Java application. No doubt whatsoever (I just hadn't gotten around to it by the time I abandoned it). What *is* interesting, though, is that the Rails page caching maxes out . The server can't serve the pages any faster than that. And with the Rails cache_sweeper observer pattern, it is dead-simple to use this hyper-fast caching whenever possible (obviously, highly dynamic pages can't be cached, but that's the case with that kind of page in any application stack). If you switch to the action caching (which allows all the filters to be executed) the Rails app still ends up in the 800-1000 requests/second range.

Conclusions So what do I think? I think that the application I'm working on is perfectly suited for Rails and Rails is perfectly suited for it. I think that I have had more fun working on the Rails app than the Java version. However, I think that the Java version is just as capable, and could be just as performant, as the Rails app. To me, the eye-opening revelation isn't "Rails is faster than Java/Spring/Hibernate". It's "Rails can be very fast". I think that there is a lot to be said for Rails, and it deserves much of the press it is getting. However, I don't think its a Java-killer. I think there are plenty of applications, and development teams, that are better suited to Java and its immense universe of available support libraries. I certainly am not going to stop developing in and learning about Java just because I've discovered Rails. On the other hand, I am going to spend more of my time trying to find projects that I can use Rails on.

I'm extremely interested in how these numbers strike folks, and whether there are other comparisons that you would find useful. So interested, in fact, that I'm risking blog-spamming by turning comments back on. So, if you have some other comparisons you would find useful, or some insight onto these numbers, or even some things you think I ought to try to make the comparisons better, let me know. I can't promise I'll tackle them all, but I think it will be interesting.