Singleton methods

A lot of folks point to Smalltalk as a source for the kind of metaprogramming techniques that we Ruby programmers take for granted. Smalltalk does allow a lot of metaprogramming, but things like instance-specific behavior aren't part of the standard repertoire of most Smalltalkers. That's why Randal, who is no slouch at Smalltalk, expressed a certain amount of amazement at how Travis did this. Here's his (Travis') example Smalltalk code:

p := 4 @ 3.
p changeClassTo: (p class copy superclass: p class).
p class methodDictionary at: #negated put: (p class methodDictionary at: #transpose).
p negated

For those unfamiliar with Smalltalk, the expression 4 @ 3 creates an instance of Point, which is a 2-d point object with x=4 and y=3.

Let's write a Point class in Ruby which works like a subset of Smalltalk's Point

class Point
  attr_accessor :x, :y
  def initialize(x, y)
    @x, @y = x, y
  end
  
  def negated
    self.class.new(-x, -y)
  end
  
  def transpose
    self.class.new(y, x)
  end
  
  def inspect
    "#{@x} @ #{@y}"
  end
end

p = Point.new(4,3) # => 4 @ 3

p.negated
p.negated          # => -4 @ -3
p.transpose        # => 3 @ 4

Now , the above code might look something like this:

p = Point.new(4,3)
def p.negated
  transpose
end

p.negated # => 3 @ 4

Most Rubyists of any experience will recognize that what I've done is define a singleton method for that sole instance of point which overrides the negate method by calling transpose instead.

Another way to do this which might be a little bit more like what Travis shows in Smalltalk might be:

p = Point.new(4,3)
class <<p
  alias_method :negated, :transpose
end

Let's look at Travis' example and how he creates a 'singleton' class in Smalltalk. Here are the relevant lines:

p changeClassTo: (p class copy superclass: p class).
p class methodDictionary at: #negated put: (p class methodDictionary at: #transpose).

He gets p's class (which is Point up to now), copies it, and sets it's super class to p's class, effectively interposing the copied class object. Next he gets the transpose instance method from the copied class's method dictionary and replaces the negated method. Effectively what Ruby's alias_method does.

Now, I should point out here that Travis's example is probably specific to the Cincom VisualWorks Smalltalk dialect. The changeClassTo: method doesn't seem to be available in Squeak, although there might be a similar method, and I don't recall a similar method in the Smalltalks I've used in the past. Also changeClassTo: isn't entirely guaranteed to work, besides the cases where Ruby can't create a singleton class (for example immediate objects like FixNums), changeClassTo: requires that the 'shape' of the instance conform to the instance variable template defined in the new class, simplifying things a bit, that means that it needs to have either exactly the number of instance variables expected by the class, or at least the number of fixed instance variables expected by the class. Smalltalk classes can have a variable set of indexed instance variables which are placed after all the fixed (named) instance variables.

So let's end this exploration of comparative instance specific method creation between Ruby and Smalltalk before I move on to another interesting difference which Travis' article exposes:

1. Smalltalk provides dialect-specific mechanisms for monkeying with the class of an object and manipulating methods.
2. Ruby has some nice syntactic sugar for doing this in a way which is part of the language definition.

As a result, these techniques are far more commonly used in Ruby than in Smalltalk.

Now for the second difference, one where Smalltalk leaves Ruby a bit behind.

Turtles All the Way Down

One of the really interesting things about Smalltalk is just how much of the runtime system is exposed as 'normal' Smalltalk objects. In Ruby certain things are hidden away (sometimes not so securely) from the Ruby programmer, there's a "Wizard of Oz" behind the curtain which separates to the Ruby program from the VM (or interpreter if you prefer).

Almost everything the VM deals with in Smalltalk pokes up as one or more Smalltalk objects. In Smalltalk everything really is an object, it's turtles all the way down, at least it looks that way.

Travis' motivating problem was to force a race condition to occur on demand. Anyone who has done concurrent programming has learned that race conditions have probabilistic minds of their own.

The lever that Travis pulled was the fact that Smalltalk threads (which Smalltalk calls Processes for historic reasons) are implemented as Smalltalk objects which can be manipulated directly from Smalltalk. The Smalltalk IDE makes use of this. For example the Smalltalk debugger is really just a specialized inspector which inspects a Process object, including showing it's state (the stack frames) and manipulate it (breaking, stepping, and responding when class definitions change).

So Travis could grab the instance of a single process and change its instance-specific behavior to modify the way it handled termination in order to simulate the condition he needed.

This aspect of Smalltalk is something which many maybe too many Smalltalk programmers are familiar with. The second part of that old article link I just snuck in talks about the distributed version of Smalltalk I did at IBM many years ago which added a new kind of Process proxy which tied execution threads which crossed machine boundaries together allowing distributed debugging, exception handling etc.

Can Ruby get along without this? Sure. But it is one thing I miss from Smalltalk

Here’s an example of this style:

class Gadget
  def framilize
    razzlitize if options[:secret_ingredient_6X]
  end

  private
    def razzlitize
    end
end

The problem is that the indentation is “artificial”, there really is no nesting of evaluation scope here.
The way that Module#private and it’s kin work when they are called without arguments is to set a marker onto
Ruby’s eval stack which gets popped when the current evaluation context is exitted. So the effects of
the private method remain in effect until either the end of the current context, or one of the other methods
like public or protected, whichever comes first. This means that if we extend the above example:

class Gadget
  #...
  private
    def razzlitize
    end

  def dazzlitize
  end
end

The indentation can trick us into thinking that dazzlitize is public when it’s really private.

I’d always found the nature of methods like private without arguments to be a minor annoyance with
Ruby syntax. In my humble opinion they should have been able to take a block which delineated their effect.
What if we could write:

class Gadget
  #...
  private do
    def razzlitize
    end
  end

  def dazzlitize
  end
end

Well, actually we can write that, but it doesn’t work as expected, since the block
is silently ignored, and the method doesn’t even get defined.

So I started doing a little metaprogramming, to fix this. It seemed obvious how to do this. You
alias_method Module#private so you can extend it, redefine the method to check for a block and if none
is given, call the original method, otherwise evaluate the block in a way which makes any methods defined
private. And of course you use test driven design to do this.

I made some progress but ran into some problems:

1. The obvious way to evaluate the block:
   

   class module
     alias :old_private, :private
     def private(*args, &blk)
       if blk_given?
          module_eval do
            old_private
            blk.call
          end
       else
         old_private(*args)
       end
   end

   Doesn’t seem to work. The method isn’t made private. I had to resort to getting a list of
   the methods before the block, then calling old_private with the difference between the new methods after
   the block is evaluated, and those before.

   The module_eval now looked something like this:

   module_eval do
     existing = instance_methods(false)
     blk.call
     new_methods = instance_methods(false)-existing
     old_private(new_methods.map {|m| m.to_sym}

   There are some obvious shortcomings in this code, but at this point, the goal was to do the simplest
   thing which could work for the test cases at hands, the complications would be dealt with later.

2. To add insult to injury, calling old_private from within the new definition doesn’t work either,
   the evaluation context gets marked as described above, but it gets popped off when we exit the context
   of the new definition. At this point I decided to punt, temporarily at least, and just define a new
   method Module#with_private which took a block, and leave the existing Module#private alone.
   
3. Then I decided that I should test changing the visibility of an existing method, in other words a test
   case like this:
   

   class Test
     protected
     def meth;end
     with_private do
        def meth;end
     end
   end

   Test#meth should end up private, but my simple diff failed to notice the definition, things were starting
   to get more complicated. It looked like I’d need to hook Module#method_added and probably others. I began to
   wonder whether it was worth it.

Then I started to ponder the fact that the effect of private/protected/public gets popped when the
evaluation context exits. So I said to myself, “Is there a way to take advantage of that?” Well, Virginia,
yes there is.

class Gadget
  #...
  class_eval do
    private

    def razzlitize
    end

  end

  def dazzlitize
  end
end

This seems to work as expected, here’s a link to a
test case file.

Now I still prefer being able to give a block to Module#private and it’s ilk, and I might still work
on that, but in the meantime, class_eval and module_eval seem to provide a way to actually do what I want.

The Initial Approach

In order to control the time, I decided to add a module in Test::Unit::Testcase called TimeMachine, so that a testcase could just include TimeMachine when it needed the function.

The way it’s used is to write something like:

def testSomethingYesterday
   now_as(1.days.ago) do
     #code to be run yesterday here
   end
end

The now_as method takes a time as a parameter. Within the block, Time.now will return that time, and Date.today will return the date corresponding to that time.

Doing this was fairly straightforward. The method aliased the two methods, redefined them, then, within a begin block with an ensure clause to restore the methods afterwards, yielded to the block.

Non-stop Time Trip Only

While this simple approach worked well, it had one drawback. It couldn’t be stacked. If you called now_as again within the block, the inner call would remove the monkeypatched methods when it returned. This first showed up when I had a bug in one of my testcase methods. That was fixed easily enough by rewriting that test.

But when I ran into the code which was using implicit times, I figured it would be easier to make my test helper a bit more sophisticated. I needed a time machine which could make side-trips along it’s round, trip in time.

The TimeMachine as of Now

So here’s my current implementation of Test::Unit:Testcase::TimeMachine:

class Test::Unit::TestCase                    # 1
  module TimeMachine

    def now_as(time)
      time_class = class << Time; self; end   # 5
      date_class = class << Date; self; end
      begin
        Time.class_eval do 
          @now_stack ||= []
          if @now_stack.empty?                # 10
            time_class.class_eval do
              alias_method :old_now, :now
              def now
                @now_stack.last.dup
              end                             # 15
            end
            date_class.class_eval do
              alias_method :old_today, :today
              def today
                Time.now.to_date              # 20
              end
            end
          end
          @now_stack.push(time.dup)
        end                                   # 25 
        yield
      ensure
        Time.class_eval do 
          @now_stack.pop
          if @now_stack.empty?                # 30
            date_class.class_eval {alias_method :today, :old_today}
            time_class.class_eval {alias_method :now, :old_now}
          end
        end
      end                                     # 35
    end
end

The basic idea is to maintain a stack of times in a class instance variable of Time. We define the pseudo now and today methods when the first time is placed on this stack, and restore them when the last time is removed.

The tricky part of this code is knowing when to talk to the Time and Date classes and when to talk to their respective metaclasses. In lines 5 and 6 I grab the two metaclasses so that I can refer to them in the code below (DRY). In line 7 I start the begin block which ensures that things will be restored when now_as has finished.

On line 9, running in the context of the Time class, I ensure that it has an instance variable to contain the stack of now times. Then if the stack is empty, I define the methods. Lines 12-15 run in the context of Time’s metaclass to define now as a class method of Time. Lines 18-21 handle the today method in a similar fashion.

After we’ve ensured that our newly patched methods are there, we push the time on line 24.

The yield, on line 26 runs back in the context of the testcase, which proceeds to do it’s thing.

Once that’s done, succeed or not, the ensure block cleans things up. Lines 29-33 go back to the context of the Time class to manipulate the stack and, if it’s empty again, restore the original methods.

What’s in the Future?

So that’s where the time machine sits right now. It does the simplest thing that could possbily work, right now. It might be nice if it could, perhaps optionally, have the current time change rather than being fixed, in other words, Time.now would return the originally stated time plus whatever time increment had elapsed since the as_now call. But for now, I haven’t needed it, and I’m not planning to use the time machine to artifically skip ahead to find future requirements before I discover them in real time.

Postscript

After posting this yesterday, I discovered that the nesting didn’t really work. I was only pushing the time on to the stack the first time. I’ve just corrected the code above.