Talk Like A Duck

edit

Why I Don't Mind Using RSpec - In Fact I've Come to Love It.

Posted by Rick DeNatale Tue, 29 Jan 2008 21:24:00 GMT

About a year ago when I first encountered RSpec, I thought that the idea sounded good, but I was concerned about how much the implementation at that time pushed new methods into Object and Kernel. It felt to me as though it could interfere with the code being specified/tested. Indeed back then there were some problems when RSpec and Rails bumped heads over the use of certain Ruby metaprogramming techniques. I’d been a TDD user advocate for many years, heck I was there right after Kent Beck, “test-infected” Erich Gamma and sat in on some of their early pairing sessions during an annual OTI company technical conference at Montebello in Quebec, when JUnit was in it’s infancy. The cleaner language of RSpec did have it’s attractions, particularly in trying to get across the idea to newcomers that TDD was really writing mini-specifications rather than tests, which helps put them in the right mindset, but for those of us who had already crossed that paradigm shift, or been born on the right side of it, it didn’t seem so important.

Since then the RSpec implementation has matured, and after talking to David Chelimsky and Dave Astels at RubyConf, I decided to give it another look, and, armed with a new perspective on the use of mocks and stubs to isolate specifications and tests in BDD/TDD, I quickly became a convert. I still use other frameworks as external requirements dictate, but RSpec has become my first choice.

These days, though the choice of testing/specification framework seems to have become one of those religious issues which divide the community, almost as much as emacs vs. vim vs. textmate. I run into people all the time who reject RSpec because it’s “too magical!” Although I’ve never been able to get them to expand on that thought. Perhaps it’s based on the kind of concerns I had about it at first, maybe it’s something else. I’d love to have it explained.

And like advocates do, other arguments get expressed, some of which don’t get the scrutiny they deserve.

For example, a few days ago Rob Sanheim, someone I know and respect, wrote about why he eschews RSpec for Test::Spec. His basic, quite brief argument is that because RSpec is 10 times bigger in terms of lines of code than Test::Spec it must be worse.

How to Lie with Statistics

Nice statistics, but back when I was in college, one of the must-read books was “How to Lie with Statistics”. Given the right pick of statistics you can prove anything.

First off I think that Rob was unfair to lump the lib and spec or test directories together. Even if you might believe that code bulk is bad, I think that it’s reasonable to think of the code bulk in the implementation to be like ‘bad cholesterol’, and test-code which verifies that implementation as ‘good cholesterol.’ Most of you are probably too young to be concerned about it yet, but my doctors tell me that a high HDL/LDL ratio is more important than an absolute HDL+LDL. The theory at least is that the HDL helps sweep the LDL out of the vessels, kind of like a good test suite keeps code under control.

Here are the numbers I get running sloccount on the latest RSpec and test/unit gems:

Framework	lib	spec/test	test/code ratio
Rspec	5473	10108	1.8
Test::Spec	275	879	3.2

So at first blush it looks bad for RSpec, Test::Spec even tests itself more thoroughly!

The real problem with Rob’s post is that he’s comparing Apple seeds with Apples. Of course Test::Spec is considerably smaller than RSpec, it’s a bump on Test::Unit to add the new assertion style. So we need to at least add Test::Unit’s numbers

Framework	lib	spec/test	test/code ratio
Test:Unit	2741	0	0

Now I’m sure that Nathaniel “ate his own dog food” when he developed Test::Unit, and Ryan is continuing to dine on it, now that he’s taken over its maintenance, but that dog food apparently doesn’t leave home. Now if we look at the combination of Test::Unit and Test::Spec the comparison looks like this:

Framework	lib	spec/test	test/code ratio
Rspec	5473	10108	1.8
TU+TS	2713	879	0.3

So RSpec looks a little better, particularly when it comes to eating it’s own dog food.

But, as that Billy Mays guy on the Infomercials is wont to say, “Wait! There’s more!”.

RSpec also includes a built-in mocking framework, now with the ‘traditional’ choice you need to throw in your choice of mock framework, here’s are the stats for Flexmock and Mocha

Framework	lib	spec/test	test/code ratio
Flexmock	1034	2743	2.6
Mocha	1218	3622	2.8

So now we’ve got two sub-variants to look at:

Framework	lib	spec/test	test/code ratio
Rspec	5473	10108	1.8
TU+TS+FM	3747	3622	1.0
TU+TS+Mocha	3931	3931	1.1

Now you might say that RSpec still is 40% larger than even the larger of the two competing combinations, but this is also ignoring the fact that RSpec also includes RBehave which is another higher level integration/acceptance testing framework.

So in retrospect, I’d humbly submit that code bulk isn’t such a good comparison criteria in this case.

So whatever your test framework ‘belief system’ do as you feel best, but do unto others!

I’d still like to hear more about why people think that RSpec is “too magic” though.

By the way on the subject of lying statistics. I’m sure that most of us have by now seen this chart of Ruby job prospects.

Which looks like, and is, GREAT news for Rubyists, but how many have seen this from the same source. It’s the same data, just looked at through a different lens.

I’m still not looking for Java work though!