Talk Like A Duck

I'm working a project which uses RSpec, Bundler, and Rails 2.3.4.

The team has been struggling with the changes in gem management with Bundler. The "Old School" rails way to configure gems for different rails environments is to conditionally execute gem.configure based on the rails environment. Then use the rake:gems:install task to gem install the gems, one for each environment.

Bundler does things a bit differently. It uses groups to segregate gems by environment, so you name the gems needed for the test environment in a :test group. When you run bundle install, it installs all of the gems for all of the groups (unless you opt out of some groups. It uses the groups at run time, to selectively expose only gems applicable to the current environment.

This cause problems for things like the spec rake task. If you run

without explicitly setting the Rails environment.

The way the spec rake task works is that it depends on a rails provided rake task called :environment, which 'boots' the rails environment, then the spec_helper file which each spec whould require, sets ENV['RAILS_ENV'] to 'test' if it's not already set, then it requires needed gem code, like 'spec/rails'.

In the old school approach this works, since those gems are installed and visible. With bundler, it fails since they won't be exposed, since the bundler environment got set up using the development environment, so gems in the :test group aren't available.

In an attempt to fix this, other members on the team were mixing things up, doing things like putting conditional tests of the rails environment in the Gemfile, and then running the bundle command with different overrides of the RAILS_ENV environment variable.

But that way lies madness.

This morning I worked out a solution which involves deferring setting up the bundle environment.

1. I needed to keep the spec task from running the environment task. The project was already using a variation of override_rake_task, so I added an override in a task within the lib directory:

         override_task :spec do
           Rake::Task["spec:original"].execute
         end
       

2. The next step was to bootstrap the rails environment in spec_helper. To do this I started it with:

         ENV["RAILS_ENV"] ||= 'test'
   
         RAILS_ROOT = "#{File.dirname(__FILE__)}/.." unless defined?(RAILS_ROOT)
         environment_path = File.expand_path(File.join(RAILS_ROOT, 'config', 'environment'))
         require(environment_path)
           require 'spec/rails'
           #... any other needed requires
       

   It's crucial to expand the path which ensures that the same string is used whenever the environment file is required. Before I did that I was having problems with the environment being required a second time, presumably for the second spec. If you have any code which patches anything using alias_method_chain, or a similar technique, loading that code a second time can cause infinite loops which can be mystifying.

So this seems to be working. I'll try to update if I find anything else, but in the meantime I hope some folks find it useful.

Yesterday, David released RSpec 1.2.7, which includes a patch I provided to allow the specification of where to find the 'ruby' program when creating a SpecTask, rather than relying Rakes RUBY variable.

Why did I submit this patch you ask, assuming you didn't read the title of this post?

So you can do this in a Rakefile :

multiruby_path = `which multiruby`.chomp
if multiruby_path.length > 0 && Spec::Rake::SpecTask.instance_methods.include?("ruby_cmd")
  namespace :spec do
      desc "Run all specs with multiruby and ActiveSupport"
      Spec::Rake::SpecTask.new(:multi) do |t|
        t.spec_opts = ['--options', "spec/spec.opts"]
        t.spec_files = FileList['spec/**/*_spec.rb']
        t.ruby_cmd = multiruby_path
       end
   end
end

This is derived from something I just added to RiCal but haven't yet released.

What it does is check that you have multiruby, which is part of the zentest gem, installed, and that your version of RSpec supports the new ruby_cmd option. If both conditions are met it makes a spec task which runs the specs using multiruby instead of ruby.

Now it's easy to run specs with the various ruby versions you want to support.

A recent entry by Courtenay on the caboo.se blog has been causing a minor furore in the Ruby community.

It looks like the problem experienced by Courtenay was caused by an interaction between RSpec and the new Rails 2.1.x gem configuration mechanisms, a problem which had been reported to, and solved by, the RSpec team, two weeks, and one week respectively, before the caboo.se post.

One lesson that could and should be learned from this is that with the power of object oriented frameworks, comes the need to be careful in managing upgrades. This is particularly true when multiple, independently developed, frameworks are combined. This is why techniques to manage the configuration of a particular application, including the versions of ‘vendored’ components is so essential.

I wrote about the issues in dealing with framework evolution quite a while ago, it was the key argument I had with the now retired head of a certain large company based near Seattle.

I’ve been a pretty heavy user of RSpec for about a year now, and have updated several times. It’s always been a pleasant experience, albeit not without having to make minor changes to my specs. And no one has forced me to do it, so that I can pick my time. The RSpec team has always been good about eating its own dog food, with pretty extensive self-specification/testing. Of course, it’s impossible to cover 100% of unforeseen problems.

I’ve had more problems with Rails upgrades of late, particularly with some of the recent changes to Active Record. Although Rails has a fairly large test suite, it’s very hard for it to cover all the ways that the many Rails application actually use and stress the framework.

But giving up is my last option, first choice is to keep evolving my code as the frameworks evolve, but to use configuration management to allow me to do it on my schedule.

I was adding to an existing controller today at work and I was stumped because one of the examples in its controller spec was failing due to an exception thrown while rendering a view.

Normally RSpec controller tests, which are usually used to specify controller logic, bypass view rendering unless you specifically use the integrate_views method in the example group. The idea is that you should use view specs to specify the logic and output of views.

Since I was working in existing code, which pre-existed my being here, I’d followed some of the existing example groups in the file. These would typically look something like this:

describe HobbitController, "doing something" do

    before(:each) do
      @controller = HobbitController.new
      #.. more setup stubbing etc.
    end

   #examples
end

I finally realized that that first line in the before block was clobbering RSpecs ability to control whether or not views should be rendered. RSpec actually creates the controller instance variable for you, and extends it with a module which overrides the render method to allow integrate_views to control the rendering while allowing the expect_render expectation to work whether or not rendering actually happens.

So I’ve got a new task to clean up all the other controller specs.

One of my current Rails projects involves generating an RSS feed. While I was working on this the other night, it seemed to be working, so I deployed it to the staging server. Everything looked fine. If I fetched it with Firefox, the browser offered to let me subscribe to the feed with Google Reader, and if I used Safari I'd see a nice view of the feed just like I expected.

So I sent a note via our campfire to check it out, and a colleague replied that his Safari was saying that it was in an invalid format.

So I went to the W3 RSS Feed Validation Service and worked through the validation issues, after which his browser was as happy as mine.

Of course, having been through that, I wanted to make sure that RSS validation was covered in the specs for the project.

I went looking for an existing RSpec matcher, and found a matcher to validate XHTML but nothing for RSS.

I then found the feedvalidator gem which provides a Ruby interface to the SOAP interface to the W3 feed validator. You would think that W3 would be providing a REST interface! The gem already provides assertions for use with Test::Unit, so I just built an RSpec matcher.

class BeValidFeed
  require 'feed_validator'
  require 'tmpdir'
  require 'md5'

  def matches?(response)
    return true if validity_checks_disabled?
    v = W3C::FeedValidator.new()
    fragment = response.body
    filename = File.join Dir::tmpdir, 'feed.' + MD5.md5(fragment).to_s
    begin
      response = File.open filename do |f| Marshal.load(f) end
      v.parse(response)
    rescue   
      unless v.validate_data(fragment)
        @failure = " could not access w3 validator to validate the feed."
        return false
      end
      File.open filename, 'w+' do |f| Marshal.dump v.response, f end
    end
    v.valid?   
  end

  def description
    "be valid xhtml"
  end

  def failure_message
   @failure || " expected xhtml to be valid, but validation produced these errors:\n #{@message}"
  end

  def negative_failure_message
    " expected to not be valid, but was (missing validation?)"
  end

  private
    def validity_checks_disabled?
      ENV["NONET"] == 'true'
    end
end

def be_valid_feed
  BeValidFeed.new
end

I saved this as spec/be_valid_feed.rb

And in a view or controller spec, I can include this file, and test a response with:

response.should be_valid_feed

If you use this in a controller spec, you will need to tell RSpec to integrate_views, or you won't have much of a feed to check. If you use nested example groups, integrate_views needs to be inside the inner group.

About a year ago when I first encountered RSpec, I thought that the idea sounded good, but I was concerned about how much the implementation at that time pushed new methods into Object and Kernel. It felt to me as though it could interfere with the code being specified/tested. Indeed back then there were some problems when RSpec and Rails bumped heads over the use of certain Ruby metaprogramming techniques. I’d been a TDD user advocate for many years, heck I was there right after Kent Beck, “test-infected” Erich Gamma and sat in on some of their early pairing sessions during an annual OTI company technical conference at Montebello in Quebec, when JUnit was in it’s infancy. The cleaner language of RSpec did have it’s attractions, particularly in trying to get across the idea to newcomers that TDD was really writing mini-specifications rather than tests, which helps put them in the right mindset, but for those of us who had already crossed that paradigm shift, or been born on the right side of it, it didn’t seem so important.

Since then the RSpec implementation has matured, and after talking to David Chelimsky and Dave Astels at RubyConf, I decided to give it another look, and, armed with a new perspective on the use of mocks and stubs to isolate specifications and tests in BDD/TDD, I quickly became a convert. I still use other frameworks as external requirements dictate, but RSpec has become my first choice.

These days, though the choice of testing/specification framework seems to have become one of those religious issues which divide the community, almost as much as emacs vs. vim vs. textmate. I run into people all the time who reject RSpec because it’s “too magical!” Although I’ve never been able to get them to expand on that thought. Perhaps it’s based on the kind of concerns I had about it at first, maybe it’s something else. I’d love to have it explained.

And like advocates do, other arguments get expressed, some of which don’t get the scrutiny they deserve.