Jul 28 2009Comments
As you've probably gathered by now, we're passionate about testing. Over the course of the past year, we've implemented various improvements to rcov — the Ruby code coverage analysis tool — to help satisfy our testing appetite.
To date, we've squashed numerous bugs, simplified the report-generation logic (making way for better template support in the future), and achieved experimental support for Ruby 1.9. While working out bugs reported by developers using our fork of rcov (relevance-rcov), one thing has become painfully obvious to us: the bug reports lack the context we need to properly troubleshoot the issues. Fortunately, rcov is bundled with a handy tool that can help mitigate this problem, and we'd like to take a moment to share the details.
Let's say you're having an issue with rcov and you want to report a bug.
First, simply locate the test that produces the bug.
Then, use rcov's
report-cov-bug flag to produce the bug report.
From your console run:
rcov --report-cov-bug='Class#method' path/to/testfile
As a practical example,
let's say that we have a test for a Rails ActiveRecord class called
Project and an offending method called
This class has Micronaut-based tests located in
In order to produce an rcov bug report you would run:
rcov --rails --report-cov-bug='Project#archived?' examples/models/project_example.rb
This will produce a nicely-formatted bug report that tells us what version of Ruby you are running as well as your version of rcov, the arguments you passed in, and the method in question. Along with this information, it will provide additional details about how the code was analyzed. The end result will look something like the following:
Bug report generated on Wed Jul 22 13:17:35 -0400 2009 Ruby version: 1.8.6 (2008-08-11) Platform: universal-darwin9.0 rcov version: 0.8.3.4 rcovrt loaded? true using RubyGems? true Command-line arguments: ["--rails", "--report-cov-bug=Project#archived?", "examples/models/project_example.rb"] Coverage analysis bug in: Project#archived? Line(s) 46 should be green. Raw coverage information (feel free to remove useless data, but please leave some context around the faulty lines): 3: 45: def archived? 2: 46: archived_at 2: 47: end 0: 48:
Once you have that information, you're ready to log the issue at github.com/relevance/rcov/issues.
Known Issues with report-cov-bug
As helpful as the
report-cov-bug tool is, it does fall down in a few places:
- ActiveRecord "declarative" methods (e.g.,
default_scope) report the actual ActiveRecord code, not the model code.
- The C version of
#callsitedoes not accurately report executed methods for Ruby 1.9.
We use rcov on Linux and OS X for all our Ruby 1.8.x projects. If you run into an issue using rcov in this environment, please let us know.
What about other runtimes and platforms?
- For JRuby projects, be sure to check out jruby-rcov. (relevance-rcov does not currently support JRuby.)
- The Ruby 1.9 support is not ready for prime time. We consider it an experiment at this point.
- We are not Windows users day-to-day, so Windows is not currently supported. However, you're welcome to fork the repository if you have the urge to scratch that itch.
You Can Help Build a Better rcov
report-cov-bug output with your bug report gives us a much better chance of tracking down the root cause of the issue.
Take relevance-rcov for a spin on your project, and if you see any hiccups, let us know.