Talk Like A Duck

Rails 2.1RC1 and the Enhanced Migrations Plugin

Posted by Rick DeNatale Thu, 15 May 2008 19:24:00 GMT

Yesterday I started the work of moving the app to Rails 2.1RC1.

With version 2.1, Rails is moving to timestamp based migration names, instead of the traditional sequence numbering. This effectively does the same thing as the enhanced_migrations plugin, but the implementation is slightly different, more on this in a bit.

After installing or gitting Rails 2.1 into my vendor/rails directory, I ran the rake task to run all of our tests and specs.

Early on this task does a rake db:migrate, which attempted to run ALL of our migrations since Rails 2.1 looks in a different place to determine if a migration has been already run.

Implementation Differences

Until now Rails has tracked migrations using a single row, single column table in each database called schema_version. This table holds the number of the latest migration which has been run.

The enhanced_migrations plugin uses a w table, named migrations_info, with one row for each migration which has been applied to the database. The key used for this table is the migration number which the migration generator sets to Time.now.to_i. The only column besides the id is created_at. So the enhanced_migration plugin captures the time at which the migration was run against the database, which is different than the time it was generated.

Rails 2.1 uses a table called schema_migrations, which also has a row per applied migration, but this table has a string key whose value is the migration number, in Rails 2.1, the migration number is also a utc timestamp but is a string in the the form “yyyymmddhhmmss”

Blake Watters and I noticed that the timestamp forms can’t overlap since enhanced migrations uses a 10-digit timestamp and Rails 2.1 uses a 14-digit one. So we wrote a new migration 1_convert_migration_schema_to_rails21.rb

class ConvertMigrationSchemaToRails21 < ActiveRecord::Migration
  def self.up
    values = select_values('select id from migrations_info')
    values.each {|value| execute("replace into schema_migrations VALUES('#{id}')")}
  end

  def self.down
  end

We then ran rake db:migrate:up VERSION=1

This populated schema_migrations so that it now “knew” that all of our current migrations had already been run.

Not quite there

For many current users of enhanced_migrations, this might be enough. In our case it wasn’t. When I tried to run rake test again, it blew up in the prerequisite task db:test:clone_structure, complaining about bad sql syntax on an insert statement. The insert statement which was logged looked perfectly fine. It was the second of a series of statements inserting rows into our new friend, the schema_migrations table. The SQL in question looked like this:

INSERT INTO schema_migrations (version) VALUES ('0');
INSERT INTO schema_migrations (version) VALUES ('1');

Followed by a similar line for each of our 500 or so migrations.

The rake task for db:test:clone structure, basically breaks the file db/development_structure.sql into chunks delimited by empty lines, and ‘plays’ them against the test database using ActiveRecord::Base.connection.execute. After some head scratching over why this seemingly legal SQL was being rejected, it occurred to us that it was trying to execute all of these lines as one sql statement.

So the next step was to figure out how to get db/development_structure.sql into a form which would cause the task to execute each sql statement individually. This was a simple patch to rails, which once made, allowed rake db:test:prepare to run successfully, which in turn allowed me to actually move on to running the test suite and working out the problems caused by the changes to rails.

I opened up ticket #201 on the Rails lighthouse which contains the Rails patch.

Posted in rails | Tags migrations, rails2.1 | no comments | no trackbacks

It's About Time!

Posted by Rick DeNatale Wed, 23 Apr 2008 20:17:00 GMT

The third edition of Agile Web Development With Rails is now in Beta.

Sam Ruby has become the third author, along with Dave Thomas and DHH. According to PragDave, Sam has converted the Depot App tutorial to be compatible with Rails 2, and the rest of the book is also being revised.

Posted in rails | no comments | no trackbacks

Rails Integration Test File Upload Plugin

Posted by Rick DeNatale Fri, 18 Apr 2008 22:07:00 GMT

Some time ago, I wrote a rails patch which allows http file uploading in integration tests.

    post '/upload' :file => TestUploadedFile("/path/to/blah.txt")

The patch detects a TestUploaded file amongst the parameters, and converts the request into a proper multi-part request.

It will work for both posts and updates.

If you’re running edge rails you’ve already got this. But many have asked for a version for earlier rails versions. At work, I’ve been working on an RSpec story which requires file uploading, and we’re using the 2.0.2 gem version of Rails for deployment purposes, so I whipped up a little plugin which patches 2.0.2 Rails with the changes from Rails changeset 8978. The plugin checks the rails version and throws an exception if it’s anything other than “2.0.2” since I haven’t tried this with anything earlier, and it’s not necessary for anything newer.

You can get it from git hub at git://github.com/rubyredrick/integration_upload_plugin.git

This also marks the first time I’ve used git for anything but gitting Rubinius.

Posted in rails | Tags integration_upload_plugin, open_source_contributions | no comments | no trackbacks

RSpec Anti-Pattern - Don't Set @controller yourself

Posted by Rick DeNatale Wed, 26 Mar 2008 21:30:00 GMT

I was adding to an existing controller today at work and I was stumped because one of the examples in its controller spec was failing due to an exception thrown while rendering a view.

Normally RSpec controller tests, which are usually used to specify controller logic, bypass view rendering unless you specifically use the integrate_views method in the example group. The idea is that you should use view specs to specify the logic and output of views.

Since I was working in existing code, which pre-existed my being here, I’d followed some of the existing example groups in the file. These would typically look something like this:

describe HobbitController, "doing something" do

    before(:each) do
      @controller = HobbitController.new
      #.. more setup stubbing etc.
    end

   #examples
end

I finally realized that that first line in the before block was clobbering RSpecs ability to control whether or not views should be rendered. RSpec actually creates the controller instance variable for you, and extends it with a module which overrides the render method to allow integrate_views to control the rendering while allowing the expect_render expectation to work whether or not rendering actually happens.

So I’ve got a new task to clean up all the other controller specs.

Posted in rails | Tags Rspec | no comments | no trackbacks

Validating RSS Feeds with RSpec

Posted by Rick DeNatale Fri, 08 Feb 2008 15:38:00 GMT

One of my current Rails projects involves generating an RSS feed. While I was working on this the other night, it seemed to be working, so I deployed it to the staging server. Everything looked fine. If I fetched it with Firefox, the browser offered to let me subscribe to the feed with Google Reader, and if I used Safari I'd see a nice view of the feed just like I expected.

So I sent a note via our campfire to check it out, and a colleague replied that his Safari was saying that it was in an invalid format.

So I went to the W3 RSS Feed Validation Service and worked through the validation issues, after which his browser was as happy as mine.

Of course, having been through that, I wanted to make sure that RSS validation was covered in the specs for the project.

I went looking for an existing RSpec matcher, and found a matcher to validate XHTML but nothing for RSS.

I then found the feedvalidator gem which provides a Ruby interface to the SOAP interface to the W3 feed validator. You would think that W3 would be providing a REST interface! The gem already provides assertions for use with Test::Unit, so I just built an RSpec matcher.

class BeValidFeed
  require 'feed_validator'
  require 'tmpdir'
  require 'md5'

  def matches?(response)
    return true if validity_checks_disabled?
    v = W3C::FeedValidator.new()
    fragment = response.body
    filename = File.join Dir::tmpdir, 'feed.' + MD5.md5(fragment).to_s
    begin
      response = File.open filename do |f| Marshal.load(f) end
      v.parse(response)
  	rescue   
      unless v.validate_data(fragment)
        @failure = " could not access w3 validator to validate the feed."
        return false
      end
      File.open filename, 'w+' do |f| Marshal.dump v.response, f end
  	end
    v.valid?   
  end

  def description
    "be valid xhtml"
  end

  def failure_message
   @failure || " expected xhtml to be valid, but validation produced these errors:\n #{@message}"
  end

  def negative_failure_message
    " expected to not be valid, but was (missing validation?)"
  end

  private
    def validity_checks_disabled?
      ENV["NONET"] == 'true'
    end
end

def be_valid_feed
  BeValidFeed.new
end

I saved this as spec/be_valid_feed.rb

And in a view or controller spec, I can include this file, and test a response with:

response.should be_valid_feed

If you use this in a controller spec, you will need to tell RSpec to integrate_views, or you won't have much of a feed to check. If you use nested example groups, integrate_views needs to be inside the inner group.

Posted in ruby, best_practices, rails | Tags Rspec, rss, validation | 4 comments | no trackbacks

Why I Don't Mind Using RSpec - In Fact I've Come to Love It.

Posted by Rick DeNatale Tue, 29 Jan 2008 21:24:00 GMT

About a year ago when I first encountered RSpec, I thought that the idea sounded good, but I was concerned about how much the implementation at that time pushed new methods into Object and Kernel. It felt to me as though it could interfere with the code being specified/tested. Indeed back then there were some problems when RSpec and Rails bumped heads over the use of certain Ruby metaprogramming techniques. I’d been a TDD user advocate for many years, heck I was there right after Kent Beck, “test-infected” Erich Gamma and sat in on some of their early pairing sessions during an annual OTI company technical conference at Montebello in Quebec, when JUnit was in it’s infancy. The cleaner language of RSpec did have it’s attractions, particularly in trying to get across the idea to newcomers that TDD was really writing mini-specifications rather than tests, which helps put them in the right mindset, but for those of us who had already crossed that paradigm shift, or been born on the right side of it, it didn’t seem so important.

Since then the RSpec implementation has matured, and after talking to David Chelimsky and Dave Astels at RubyConf, I decided to give it another look, and, armed with a new perspective on the use of mocks and stubs to isolate specifications and tests in BDD/TDD, I quickly became a convert. I still use other frameworks as external requirements dictate, but RSpec has become my first choice.

These days, though the choice of testing/specification framework seems to have become one of those religious issues which divide the community, almost as much as emacs vs. vim vs. textmate. I run into people all the time who reject RSpec because it’s “too magical!” Although I’ve never been able to get them to expand on that thought. Perhaps it’s based on the kind of concerns I had about it at first, maybe it’s something else. I’d love to have it explained.

And like advocates do, other arguments get expressed, some of which don’t get the scrutiny they deserve.

Posted in ruby, rails | Tags bdd, Rspec, tdd, TestSpec, TestUnit | 3 comments | no trackbacks

Book Review: The Rails Way

Posted by Rick DeNatale Thu, 27 Dec 2007 17:57:00 GMT

The Rails Way came out shortly before Rails 2.0 was released. I must say that it serves as a very valuable reference for those with some Rails experience, and in particular for those who are coming up to speed on Rails 2.0.

Posted in book_reviews, rails | no comments | no trackbacks

Book Review: Pro Active Record

Posted by Rick DeNatale Tue, 30 Oct 2007 16:31:00 GMT

While I generally agree with Josh's assessment, We do differ a bit on which audience the book best serves.

The goal of the book is to cover Active Record in depth, outside of the context of Rails. The result, as I see it, is a book which is useful to intermediate to advanced users of Active Record, those who want to dig in to understanding the implementation and perhaps extending it.

My own reading of the book gave me the impetus to explore the code of Active Record to the extent where I felt comfortable submitting contributions to Rails. Since reading the book, I've written and submitted two active record patches to the Rails Trac. The first fixed an oversight which made the schema.db file dumped for MySQL tables with non-standard primary keys to lack those primary key declarations, and the second is an enhancement which allows the :joins option of methods like find and count in ActiveRecord::Base to take values like the :include option as an alternative to a sql joins clause string. Both have made it into rails edge!

Posted in book_reviews, rails | Tags activerecord | no comments | no trackbacks

My First Serious TextMate Automation

Posted by Rick DeNatale Tue, 09 Oct 2007 12:15:00 GMT

A permanent fixture.
© Henry Faber

One resource has been James Edward Gray II's book on TextMate published by the Pragmatic Programmers. I finally sat down and got serious about writing some automations of my own.

I've got to say that I'm pretty impressed by TextMate. At the last Raleigh.rb hack night I was talking to another vim user. I'd mentioned to him that you can extend TextMate easily in Ruby, without really having experienced it. Today, I wrote a neat little TextMate command to help in building Rails database test fixtures.

It acts like a tab triggered snippet, but it's a smart little critter. If I'm editing a fixture file, say 'test/fixtures/users.yml', I can type item then tab and it will produce the skeleton yaml for a new record, with:

1. A dummy name selected for overtyping.
2. The id set to the next available primary key
Each column name as a yaml key ...
3. ... A tabstop on a value which shows the column type

For example:

Posted in rails | Tags automation, fixtures, testing, textmate | 1 comment | no trackbacks

Observing Fields With Rails in the Browser

Posted by Rick DeNatale Sun, 07 Oct 2007 16:34:00 GMT

observation car
© carolyn

This can all be done within the browser. It just requires manipulation of existing elements within the domain object model, so there's no need to go back to the server.

Rails provides the observe_field helper method which is normally used to generate a remote request when a DOM element changes. It can be used to do what we want, but how to do so isn't well documented, either in the Rails documentation, or via google.

So here's how to do it.

Read more...

Posted in rails | Tags ajax, prototype | 1 comment | no trackbacks