Schrödinger’s experiment is really a dramatized version of a similar thought experiment previously published in a paper
co-written by Einstein. Although Einstein intended it as a refutation of the probabilistic nature of quantum mechanics,
the stochastic basis of quantum mechanics has held up over the ensuing seven decades, and such paradoxes have held up.

My point here is that, in a sense, the kind of type checker represented by ducktator. It attempts to observe certain
aspects of an object, and in so doing, has a good chance of killing the duck.

Besides the questions of what duck-typing is or is not, such a set of checks fails under many common usages in ruby.

False Negatives

Relying on responds_to? fails under commonly used Ruby metaprogramming techniques.

It’s very common these days to use method_missing to provide pseudo-methods, or to
create methods on the fly. ActiveRecord and other RubyOnRails componentry makes extensive use
of this technique.

The problem is, of course, that an instance of a class which uses these techniques effectively
gives an invalid response to responds_to?, or may reply false, until the missing
method is invoked, creating the method.

Poking around a bit, I notice that some of the standard library classes, and others liek ActiveRecord, go to some lengths to make responds_to? track with phantom methods implemented via :method_missing, although fixing other reflection methods like :methods and the class method :instance_methods doesn’t seem to be commonly done.
While this is good, it can greatly increase the expense of :responds_to?. ActiveRecord::Base, aliases :responds_to? as responds_to_without_attributes? before redefining :responds_to? for this reason.

In the case of Drb::DrbObject, which is used as the proxy object in Distributed Ruby, handling :respond_to? on behalf of a remote object, requires marshalling a :respond_to? message to the remote object, sending the marshalled message, then reading and unmarshalling the reply. This makes sending :respond_to? to a remote object much more expensive.

And irony of ironies, ductator uses :method_missing to supply phantom methods in it’s
implementation, without implementing a compensating :respond_to?!

False Positives

What’s the Value?

This kind of type checking seems to be an expensive add-on to the real checking which happens
during program execution. The fact that everything is an object in ruby means that the really
nasty errors in a language like C when you use an integer as a pointer don’t happen. It seems
to me that the best that something like ductator can do is to make a more ‘meaningful’ error
description when something goes wrong, but doing that correctly is harder than it first appears,
and in most cases could be accomplished more efficiently by catching errors with rescue clauses
and changing the error reporting, after the fact, which puts the cost of the error on the
offender rather than taxing everyone with run-time pre-flight tests.

Wrapping up for now

In computer jargon, the word “canard” means a “mistaken and confused belief”. In my humble opinion, the view of duck typing taken by ducktator and its ilk is in this sense really “canard” typing.

Le canard est mort, vive long le canard!

Or translated into English:

The canard is dead, long live the duck!

Ruby Variables Don’t Have (Permanent) Classes

In one recent thread, someone suggested adding a method to the “class of a variable.”

The problem with this idea, is that a variable doesn’t really have a class. For example, consider thiscode:

require 'matrix'
   a = 5
   a = a * 1.0
   a = a * Matrix.identity(3)

Here the variable a refers to a sequence of objects, each with a different class. First a Fixnum, then a Float, then a Matrix. While Fixnum, and Float inherit from Numeric, Matrix doesn’t but it can form a duck-type with Numeric in many uses.

This also illustrates that “all variables are the same size.” In a language like C, a variable holding an integer, will probably have a diffferent size as one holding a double (which is what Ruby defines as the representation of a float), and one which holds a 3×3 matrix of doubles certainly will be bigger than one holding a scalar double.

Ruby Variables Hold Object References, not Object State

This encapsulation barrier is key to what makes a language truly “object-oriented,” at least the way I use the term.

The “bag-of-bits” which represents a particular object is visible to the methods of that object’s class, at least in an abstract way. Some details are even hidden to those methods at the Ruby source code level, and are tied up in the language implementation, hand written extensions to Ruby usually do need to deal with those details.

In Smalltalk, the pseudo-variable self is actually strongly typed in the C-sense since instances of Smalltalk classes have a fixed structure, and self in a method is guaranteed to refer to an instance of the class owning the method or one of it’s subclasses, and instance variables are found in known slots within the object.

In Ruby it’s a little different because Ruby instance variables are created dynamically and accessed via a hash table within the object, at least in the 1.8.x implementation.

As a first approximation, we can initially think of variables as holding pointers to objects, but I’ll expose this for the pedagogical lie that it is, a little later in this article.

Mutability, and Aliasing

Here’s one of those stumbling blocks for those who expect variables in a uniformly object-oriented language to work like they do in a language like C or Fortran:

 1: a = [1, 2, 3]
 2: b = [1, 2, 3]
 3 1: a = [1, 2, 3]

 2: b = [1, 2, 3]

 3: c = a

4: a[1] = 0
5: p a #=> [1, 0, 3]
6: p b #=> [1, 2, 3]
7: p c #=> [1, 0, 3]

Everything looks pretty normal up ot line 7. We never did anything with c after we assigned it a value, but it doesn’t seem to have held it’s value.

Or has it?

Well, it actually has, since it’s value is a reference to an object, and that object is the same one referenced by a.

Line 4 might look like an assignment to the variable a, but it’s really a method call to the array which a happens to be referencing at the time. And that method (called []=) changes, or mutates that array. That change will be visible through the variables a, c and any others that reference that particular array. Multiple references to the same object are called aliases to that object. They might be named variables, or referenced which are inside another object:

 8: d = [[1, 2, 3], [4, 5, 6]]
 9: e = d
10: d[1][1] = "Fred"
11: p e #=> [[1, 2, 3], [4, "Fred", 5]]]

While such mutating methods are often very useful, this effect of aliasing is something which the Ruby programmer who uses them needs to keep in mind.

Many built-in Ruby classes have both mutating, and non-mutating versions of methods, for example Array#compact returns a new array which is a copy of the receiver with nil elements removed, leaving the original array intact, while Array#compact! mutates the receiver and removes the nil’s in-place.

As with compact and ocmpact!, it’s standard practice to give such mutating methods a name with a trailing ‘!’ which serves as a warning that they mutate the receiver. It’s a warning to the wise.

That Little White/Pedagogical Lie, or Variables and Object Identity

I said above that thinking that variables hold pointers to objects was a first approximation to reality. The fact is they really don’t

12: a1 = [1, 2, 3]
13: a2 = [1, 2, 3]
14: p a1.object_id #=> -605450882
15: p a2.object_id #=> -605225658

The object_id method returns a number which is unique in the sense that no two active objects will ever have the same object_id, and a given object will always have the same object_id. In the Ruby 1.8.x implementation, object_id actually just gives the bits of the value used to reference the object in the guise of a Fixnum.

So we see here that a1 and a2 are diffferent objects. Let’s try a few more objects:

14: i1 = 1
15: i2 = 1
16: p i1.object_id #=> 3
17: p i2.object_id #=> 3
18: sym1 = :sym
19: sym2 = :sym
20: p sym1.object_id #=> 4090126
21: p sym2.object_id #=> 4090126 

Here’s another surprise. Before when we created two different arrays, we ended up with two different objects. But even though we never assigned i2 to i1, or sym2 to sym1, the ‘two’ 1s have the same object_id, and the ‘two’ symbols named :sym share an object_id, what’s that about?

Certain classes ensure that if ‘two’ of their instances are equal they are the same object. We can see that Fixnum, and Symbol do this, others are NilClass, TrueClass, and FalseClass. For symbols, this conflation of value and identity is part of their definition, for the others, it’s probably not strictly necessary, but it makes some common operations like testing for equality, and in the case of Fixnums arithmetic operations, much faster.

What’s really held in variables is a value which can either be used to recognize that the object is one of these special objects, or that can be mapped efficiently to the real address of the object. The details are implementation-specific, and not really germane to the current topic, so I won’t go into them in this article.

Immutable Objects

#Hypothetical code - with a tribute to Jimi Hendrix
h1: a = 6
h2: b = 6
h3: a[0] = 1; a[1] = 0; a[2] = 0; a[3] = 1
h3: p a #=> 9
h4: p b #=> 9

Since all 6’s are the same object, then if Fixnum had mutating methods, we could make all 6’s turn out to be 9. While Jimi might not mind, my program might not like it.

And it would cause head-scratching debugging sessions. Thirty-two years ago, I found myself in just such a situation. Fortran II had a bug in the language specification which allowed the value of integer literals to be changed when you passed a literal as an argument to a subroutine which assigned to that argument. I spent over a day trying to figure out why my program didn’t work.

For this reason, to quote Martha Stewart, it’s a good thing that the Ruby numeric classes don’t have mutating methods. Even though Bignums and Floats don’t conflate value with identity, it would probably be even more mystifying to find that some instance of 1.5 changed to 1.45 when other’s didn’t. To me it seems far better to head off that particular ‘feature’ at the pass.

Another reason for this choice of immutability is taken from functional programming. Although Ruby is not really an FP language, it takes many concepts from that paradigm. In FP, functions with side-effects are disallowed, and mutation is certainly a side-effect. Taking FP to the limit makes doing things which require side-effects, like I/O, take special tricks. Ruby allows side-effects, and doesn’t require an FP approach, but like other features like regular expressions, understanding functional approaches is a useful tool in the Ruby programmer’s bag.

Immutability at the Instance Level

Ruby Object’s have a freeze method which makes a particular instance immutable:

F1: a1 = [1, 2, 3]
F2: a2 = a1
F3: a1.freeze
F4: a1[1] = "Fred" #=> TypeError: can't modify frozen array
F5: a2[1] = "Fred" #=> TypeError: can't modify frozen array

F6: a3 = [[1, 2, 3], [4, 5, 6]]
F7: a3.freeze
F8: a3[0][1] = "Fred"
F9: p a3 #=> [[1, "Fred", 3], [4, 5, 6]]

Matrix and Mutability

This upsets some who prefer to think of a Matrix as a 2-dimensional array rather than a mathematical construct which acts much like a numeric in linear algebra.

If you want to view matrices that way you’ve got a few choices:

1. Swim-against the current: Monkey-patch Matrix to add mutating methods.
2. Swim with the current: Use the Matrix#to_a and Matrix[] to get a mutable array and then create a new Matrix from it:
   

   M1: m1 = Matrix.identity(3)
   M2: p.m1 #=> Matrix[[1, 0, 0], [0, 1, 0], [0, 0, 1]]
   M3: a1 = m1.to_a
   M4  a1[1][1] = 3
   M4  m1 = Matrix[a1]
   M5: p m1 #=> Matrix[[[1, 0, 0], [0, 3, 0], [0, 0, 1]]]

3. Find a new stream: look at other sources of a mutable 2-dimensional array, for example NArray

The choice, gentle reader, is yours. I hope that my efforts help you make it a more informed choice.

William Crawford, helpfully contributed:

In short, if a type/class/whatever has all the methods of a duck (that
you intend to use), it’s a duck and can be used as a duck would be used,
regardless of what’s actually under the feathers.

In other words, TCPSocket doesn’t just inherit from Socket, Object,
etc… It IS a Socket, it IS an Object, etc.

While this is true, the second paragraph is not a particularly good example to explain duck typing.

Duck typing is not about what an object “is”, but what it can be used for. Whether an object can be used for something depends on who is using it.

Let’s look at DT’s example from the Pickaxe. Cutting the code down to the bare essentials we’ve got:

def csv_from_row(op, row)
   #code to compute res from row
   op << res << CRLF
end

Now what kind of duck does op need to be? Simply an object which accumulates objects via the << method.

His first use was:

result = ""
query.each_row {|row| csv_from_row(result, row)
http.write result
</type:code>

This works, but it's slow as the number of rows gets large. Strings meet the requirements of the op parameter, but they also have other characteristics. The important one here is the subtle effect on GC.  For a more detailed analysis of this see why's article http://whytheluckystiff.net/articles/theFullyUpturnedBin.html

So let's try to find another object which we can put in the role of the op parameter.  Arrays also can be used.  So we have

<typo:code lang="ruby">
result = []
query.each_row {|row| csv_from_row(result, row)
http.write result

However we’ve now got a little problem in that although an array works as the op parameter, http.write probably won’t like it. But that’s easy to fix, just change the last line slightly to accomodate the new duck in the outer context:

A ruby, being a gemstone which is normally cut before being put to use, has facets, unlike a pearl.

The facets of Ruby, the language, let indiduals approach it from different directions. To me, as an old Smalltaker, it tends to look like Smalltalk. I tend to view it with Smalltalk-colored glasses.

I’m sure that, to a programmer deeply steeped in Perl, Ruby looks like Perl with a cleaned-up view of object-oriented features.

Literate Programming

I’ve long been a fan of the goals of literate programming. Source code should be viewed as literature, something which communcates to human readers, and not just fodder for a tool chain which produces something which is executable by a machine.

I read and write Perl on a level similar to how I converse in French. I’m told that I can speak French fairly well, and I can make myself understood. On the other hand I have a hard time understanding spoken French.

Too often, Perl looks to me like a write-only language, an impression which is not uniquely mine.

Some Perl programmers share a common predilection for something which users of another language often called read-only, APL, prided theselves on writing, one-liners. And this has inspired catalogs of Ruby one-liners

I’ve found that a lot of the literature of Perl isn’t very understandable. I’ve found that even Perl code which I’ve written myself is impenetrable when I come back to it weeks, or even days after writing it.

I’m sure that there are Perl programmers who don’t find this to be true, just like my French friends seem to be able to converse without problems, but I continue to only muddle along in both languages.

The key indicator of Perl programs written in Ruby is the use of all those global variables with funny names like $*, $&, $0, etc. Perl programmers love them since they are in Ruby for compatibility with Perl. When I see them, my eyes glaze over, and my mind says “$&^+_”

I’m not trying to antagonize the Perl programmers among the Ruby community, anymore than I try to antagonize my francophone friends, in fact I’ve got great love for some particular francophones and perlographs.

But I’ll probably never understand them as well as I’d like to.

P.S. Not (Directly) Borrowed From Perl

One feature of Perl is the use of sigils as markers on variable names.

Ruby uses sigils to mark some variables as being globals (there’s that $ again, but the names don’t have to be funny), instance variables, or class variables.

The difference is that Perl uses sigils to distinguish the types of variables, something which is unnecessary in Ruby since everything is an object. Ruby instead uses them to find the variable binding in the case of a reference, or where to put it in the case of an initial definition. A Ruby variable comes into existance the first time it is mentioned in the execution of a Ruby program, and it goes into the global variable “pool”, the current instance, or the current instances class depending on the sigil. No declaration is necessary as compared to Smalltalk, for instance, where instance and class variables are ‘declared’ by telling the class about them, and globals are defined by putting them in the system dictionary. While Perl variables aren’t explicitly declared either, the real motivation for sigils in Perl is to associate a type with a variable.

Perl also makes use of the fact that all variables start with a sigil to automatically interpolate variables into strings. Since ruby variables don’t all start with a sigil, Ruby uses #{} as a marker to interpolate an expression int to a string. Note that if that expression is just a global, class or instance variable (all of which start with a sigil) a simple # will do the job.

So Perl uses sigils more as a way to express type, a static concept, in a dynamically executed language, and Ruby uses them to avoid static declarations, and make variable definition and use dynamic.

Sometimes it’s not enough just to have a certain set of methods in a duck’s repetoire.

Let’s look at an, admittedly cooked-up, example:

def reverse_lookup(value, dictionary)
     (dictionary.collect { |kv| kv[1] == value ? kv[0] : nil }).compact
end

irb(main):001:0> load 'reverse_lookup.rb'

=> true

irb(main):002:0> reverse_lookup(1, { 'a' => 1, 'b' => 2, 'c' => 1})

=> ["a", "c"]

The point here is that for this implementation of reverse_lookup, the only message sent to dictionary is collect, soaccording to the “respond_to?” theory any object which has “collect” in it’s repetoire is the right species of duck for the dictionary parameter.

So let’s try another duck:

irb(main):003:0> reverse_lookup(1, [1, 2, 3])
=> [0, 1]

Well, it works in the sense that nothing blows up, but the results look strange, where did that 0 come from?

While you weren’t looking, I snuck a debug parameter into my reverse_lookup method to give an “x-ray” view into what’s happening.

irb(main):004:0> reverse_lookup(1, [1, 2, 3], true)
kv = 1, kv[0]=1, kv[1]=0
kv = 2, kv[0]=0, kv[1]=1
kv = 3, kv[0]=1, kv[1]=1
=> [0, 1]

Now let’s try another array;

irb(main):005:0> reverse_lookup(1, [Array, NilClass])
NoMethodError: undefined method `[]' for NilClass:Class
        from ./reverse_lookup.rb:2:in `reverse_lookup'
        from ./reverse_lookup.rb:2:in `reverse_lookup'
        from (irb):4
        from :0

So this doesn’t work at all.

To sum this up, “duck types” aren’t just collections of messages, whether or not an object will work in any given role also depends on subtleties of the semantics of theimplementation of the corresponding methods, and can also depend on the state of the object as well.

Now this shouldn’t be considered a weakness of “duck typing” compared to “strong typing,” or something in-between the two. Strong-typing systems have similar “flaws” for example strongly typing a variable to an integer won’t protect against division by zerowhcn the variable is used as a divisor, nor will it alone protect against out of bounds errors when it’s used as an array index. The errors which aren’t detected by strong-typing is a much larger set than those which are. In languages which provide limited run-time checking of code which has passed the strong-typing “hurdle” the effects can be disastrous. In my humble opinion, the benefits of dynamic typing far outweigh “unlearning” what has been taught by other languages such as C++ and Java.

Bjarne Stroustrup used to quip, in panel discussions over strong vs. dynamic typing (i.e C++ vs. Smalltalk), that he’d hate to be a passenger on a plane which flashed a light labled “message not understood” in the cockpit when the pilot tried to lower the landing gear. I’d hate even more to be a passenger on a plane whose control software seg-faulted under this condition instead because putting the landing gear down caused a buffer overflow.

The real answer to uncovering and routing out errors is testing, testing, and more testing.

Stealing his example, he posits a class with a method which writes to the end of an open file:

   class Customer
      def initialize(first_name, last_name)
           @first_name = first_name
           @last_name = last_name
      end
      def append_name_to_file(file)
           file  << @first_name << " " << @last_name
      end
   end

He then goes on to show writing a test case for this method which instead of creating a temporary file to use as the parameter to the append_name method, uses the fact that both String and Array define << methods which do pretty much the same thing as the similarly named method in File. So even though the parameter is named file, the method really doesn’t care that it’s a File object, it really only cares that it responds to << by appending the argument to itself, and returns self.

   cust = Customer.new("Joe", "McGillicuddy")
   file_not = ""
   cust.append_name_to_file(file_not) 
      -> "Joe McGillicuddy"
   file_not 
      -> "Joe McGillicuddy"
   file_not2 = []
   cust.append_name_to_file(file_not2) -> ["Joe", " ", "McGillicuddy"]
   file_not2 -> ["Joe", " ", "McGillicuddy"]

So the file argument really doesn’t need to be an instance of File. Now let’s try a real file:

   real_file = File.open("file1")
      -> #<File:file1>
   cust.append_name_to_file(real_file)
      -> IOError: not opened for writing
             from (irb):7:in `write'
             from (irb):7:in `append_name_to_file'
             from (irb):20
             from :0

The point here, of course is that it’s not just the “type” which matters, it’s really if the object is able and prepared to do what is asked of it. Even though real_file is a file, this particular instance it’s up to the job. In one sense, it’s a duck, but it’s more of a duckling, able to walk and talk, but not fly!

IBM Research – Smalltalk and ClassC

My early IBM career gave me a lot of experience with developing software. One of the things which struck me was how, despite its name, most software was hard. Hard to develop in the first place, and harder to change to adapt to new requirements.

Like many others, I first became aware of Smalltalk when Byte magazine published a special issue on the langage in 1981. I led a small team at IBM Watson Research which developed a language called ClassC which was inspired by that Byte issue, and added Smalltalk message sending to C, it was similar in concept, but not in syntax to Brad Cox’s Objective-C. We found out about Objective-C after we’d already done our worst. ClassC got some use for internal projects in IBM after I moved to a new IBM development lab in Cary, NC in 1984.

While I was working on ClassC, the late David N. Smith was dealing with the lawyers at IBM Research, Xerox, and UC Berkeley to get Smalltalk-80 into research. Dave and I became good friends, along with other folks in the IBM Research User Interface Institute like Dave’s “sidekick,” the late Jerry Archibald, and John T. Richards, and we formed an OO-Smalltalk cadre within IBM. These three were also key participants in the birth of the ACM OOPSLA conference. Although I personally missed the first OOPSLA, I was a steady participant for the first decade or so, moderating several panel discussions, participating in workshops, and serving on the commitee for OOPSLA ’93.

One of the advantages of being part of the OOPSLA community was being able to forge friendships and working relationships with many very bright people such as Ward Cunningham, Kent Beck, and all four of the “Gang of Four”

Research to Internal Consultant

After moving to the new lab in Cary, I primarily worked as an inter-divisional consultant, working to drive the requirements of my division, which was writing what was then a new style of applications with graphical direct manipulation user interfaces, into the plans of the Personal Systems division. This got me into a lot of interesting meetings with not only other IBM folks, but also staff from other companies. For example, I was present at the shotgun marriage between IBM Boca Raton, IBM Hursley, and Microsoft which led to OS/2.

My job had given me the opportunity to visit various companies which IBM was investing in, or considering investing in. I got to meet people like Jean-Marie Hulot of Expertelligence whose interface builder implemented in Lisp got re-written in Objective-C to become NeXTStep’s Interface Builder, after he moved to NeXT. A little later, after IBM had made an investment in NeXT, I got to meet Jean-Marie again along with Steve Jobs. I got to argue with Tony Williams and Bill Gates and his team over the merits of their approach to the early design of what became COM, which made the C++ type system a little more dynamic andwith the IBM designers of SOM which tried to make a statically typed systems more resilient to “schema” evolution. Both of these were also (failed) attempts to forge an entente in the “Language wars”

Smalltalk and IBM

With the availability of Digitalk’s SmalltalkV, our little OO cadre started to push for adoption of Smalltalk inside IBM. One of my biggest contributions was to implement a prototype which extended the Smalltalk/V development environment with a direct-manipulation UI and UI Logic (i.e. controller) tool inspired by the Interface builder, which I dubbed Application Builder. When SmalltalkV/PM came out, I ported the Application Builder to it.

The division’s mission had been shifting towards building application development tools, so this prototype fit in. Most of those tools were focused on old big-iron ideas and languages, such as ISPF, and Cobol adapted to personal computers, although targeted for mainframes. I arranged to demonstrate Application Builder to Earl Wheeler, the group executive who oversaw our division and who was in charge of a grand IBM strategy called Systems Application Architecture. Wheeler seemed quite impressed with the App Builder demo and directed us to turn it into a product, and get Smalltalk to be part of SAA, although he wanted it positioned as a “generator” instead of a language for internal IBM political reasons.

I took the point on getting Smalltalk into IBM product plans, while others took the prototype and turned it into what became VisualAge. In my opinion they took the visual programming idea a bit too far, but it did the job of getting Smalltalk out to the world from IBM. And it resulted in IBM Smalltalk, which was implemented by Object Technology International, a company founded by Dave Thomas, although a different Dave Thomas than PragDave. As an aside, what is it about guys named Dave Thomas? Besides OTI-Dave (who is well-known to the Smalltalk community as Big Dave), and PragDave, we have the Dave Thomas from SCTV and Doug of the McKenzie brothers, and the Dave Thomas who founded Wendy’s. Two Canadians, and two Americans.

I also spent a lot of time talking to IBM customers about how object-oriented approaches, and Smalltalk in particular, could help them solve the problems they were having with software being hard.

Smalltalk Distributed

One of my big-idea products was Smalltalk Distributed, a feature for VisualAge/Smalltalk. I led the development on this one.

Distributed applications were the rage at the time. There was another approach to distributing Smalltalk from HP called Distributed Smalltalk, but this was really a Smalltalk implementation of the Object Management Groups CORBA. Although I had been serving as one of the IBM representatives to OMG (as a Smalltalk expert), I didn’t find this idea particularly interesting or appealing. The goal of Smalltalk Distributed was transparent distribution of Smalltalk. Our approach was to implement a distributed implementation of the Smalltalk object model including object storage and garbage collection, message sending, and distributed threads of execution complete with non-local returns. This was all done completely in Smalltalk, using lots of tricks in using metaprogramming much like what the wizard Ruby metaprogrammers do.

This project was technically interesting, and got some use by customers, but it was most valuable as a learning experience. In retrospect, the transparency was both a neat idea, and a problem. Putting a black box around a distrubuted system is probably not as good an idea of putting a distributed system together out of black boxes.

As a result of the Smalltalk Distributed work, I got to spend a month in Sydney Australia, working with Jeff McAffer of OTI, to re-implement some of the ideas such as the distributed threads in a toolkit for building server applications which he called Surfer (or Server) Smalltalk. Jeff subsequently became one of the technical leads of the Eclipse project.

Smalltalk Standardization

Another piece of work I did at IBM was to write a report characterizing the similarities and differences between the various Smalltalk implementations. At that time these were ParcPlace VisualWorks/Smalltalk-80, Digitalk Smalltalk/V, and IBM/Smalltalk (from OTI). By focusing on the externals of the classes and taking inheritance out of the picture, I wrote a report called “Smalltak: A Common Base” which was in effect a rough draft of a reverse-engineered specification for a common Smalltalk language specification.

When the various Smalltalk vendors expressed interest in forging a Smalltalk language standard, X3J20 was formed to develop an ANSI standard, and the common base document became one of the early inputs. I became the secretary of X3J20. The standard was published in January of 1998.

Language Wars – Dynamic vs. Static OO

One of the things which always attracted me to Smalltalk was that it placed encapsulation above all else. As Alan Kay noted in his memoir about the origins of Smalltalk, his original conception of object-oriented programming was that software should be composed of objects which were, in effect, little computers themselves, which encapsulated both data and behavior, and hid the implementation of both from other objects, with objects interacting via sending messages to each other and replying. This uniform object model separates languages like Smalltalk and Ruby from other “object-oriented” languages. The various versions of Smalltalk all shared this model, although they varied as to some of the semantics of message sending and reception.

The idea of classes and inheritance as a way of factoring implementation was actually a rather late addition to Smalltalk. Although Kay acknowledges the Simula language, which also lacked classes and inheritance, as one of the influences on his thinking leading up to Smalltalk, it’s been a popular misconception that the better known Simula-67 was his real influence, when Smalltalk and Simula actually evolved independently.

Kay’s term “object-oriented” got hijacked when Peter Wegner published paper entitled “Dimensions of Object Based Language Design” at the second OOPSLA conference in which he defined “object-oriented” as “objects + classes + inheritance.”

Simula-67 spawned a family of languages which called themselves object-oriented but really used classes and inheritance to create hierarchies of abstract data types. In thes languages which include C++, Oberon, and to a lesser extent, Java became popular bacause they were seen as an easy entree into “object-orientation” to programmers who were more comfortable with the notion of programs manipulating data from “a distance.” This led Alan Kay to observe that

It is unfortunate that much of what is called “object-oriented programming” today is simply old style programming with fancier constructs. Many programs are loaded with “assignment-style” operations now done by more expensive attached procedures.

Because of this morphing of the meaning of object-oriented programming I started to use the term “object programming” instead. In 1991, I wrote an IBM Research Report called “Types From the Client’s Viewpoint,” which very much relates to duck typing and which got cited in a few places. It’s long out of print. So I’ve scanned a hard copy of a draft, to show my thinking at the time.

This debate lives on. It pops up as some of the uneasiness of new Ruby programmers coming from the strong-typing/abstract data type community over thinks like “duck typing.” I plan to explain my thinking about duck typing and other related topics here.

My point here is, I’ve got strong opinions about this topic. While I do believe that the key to wisdom is strong opinions weakly held, I haven’t seen any reason, over twenty-something years to let go of this set of opinions.

Smalltalk to Java

Then Java came along. It gave some of the benefits of Smalltalk like languages in a package which was appealing to the C++ community, a lot of who were starting to become jaded over how C++ (if they actually used the compiler to do more than compile normal C programs), seemed to make development harder. It kind of looks like C++, and has a more “regular” syntax than Smalltalk, which some found wierd, although to the Smalltalkers it was beautiful in it’s sparseness and elegance.

And IBM made the jump to Java. Most of us who had been working on Smalltalk were encouraged to start working on Java and Java tools.

About this time, I took an assignment to OTI, which had been acquired by IBM Canada as a wholly owned subsidiary.

OTI itself was making an investment in Java. The first step was to turn the IBM/Smalltalk virtual machine into what was called the UVM or Universal Virtual Machine. This could run both Java and Smalltalk bytecodes. The Java “natives,” the equivalent of extensions in Ruby were written in Smalltalk instead of C. The first VisualAge for Java was written in Smalltalk on this UVM.

Eventually, as Java evolved this approach became less tenable. So OTI started building a new IDE in Java. Our first use for this was for a system to develop embedded Java applications. IBM was getting very interested in embedded software, which they called “Pervasive Computing” and had formed a division called PvC to pursue it. Embedded applications (e.g. code inside an oscilloscope, or a cell phone) written in Smalltalk was an early focus of OTI predating the relationship with IBM. Java, which was actually first developed within Sun for such embedded applications, was a natural language to employ.

The resulting development was called VisualAge/Micro Environment or VAME. The IDE and UI design was done at the OTI Zurich lab by a team led by Erich Gamma of the “Gang of Four.” This code was almost completely reworked and combined with Jeff McAffer’s component run-time architecture to become the basis for Eclipse.

After working on VAME, I moved over to a group working with customers on embedded Java applications. We majored on automotive applications, since there were a lot of car companies and suppliers who were interested in telematics systems with sophisticated software. As a “standards guy” I represented IBM in standards organizations such as the “Vehicle Expert Group” within OSGi

Back to the Mother Ship for a while

By this time, OTI was being gradually assimilated into the Borg of IBM. My group was first. Although we continued to work out of the OTI lab in Raleigh, NC, we now reported to IBM PvC management in Austin Tx.

Then because I was “the standards guy,” I got re-assigned to the PvC architecture department in Austin which was the home of all the PvC standards guys.

Eventually, OTI got “reorganized” (i.e. broken up) with various pieces going to different IBM divisions.

Free at last

After a year or so of this, and 30+ years of IBM, I realized that I was tired of life in a big corporation. Since I was old enough to still have a traditional pension, it was time to see what life was like on the outside, so I retired in the spring of 2005, and I’ve been happily pursuing my own muse, happily learning and working on new projects that I wanted to work on.

I’ve recently discovered Ruby, and as an old Smalltalker, I’m excited to see what looks like the reincarnation of an old friend. I’m starting this blog to see what I can share with the Ruby community based on my experiences.

Last week, I ran across a thread on the ruby forum called “I’ll have the Duck!” which quickly got lots of responses, and seems to have ended up with jokes.

Most of this thread seems to miss the point of duck typing.

Once you’ve really grokked duck-typing you become like that surgeon. Your perception of the properties of objects that really matter changes. And you learn to use the pathologist (testing) as a powerful ally.

Many programers coming to ruby from languages like C++ and Java are like the psychiatrist, internist, and general praticioner in the duck hunting story.

They want certainty in their typing. They’ve developed the expectation that early type-checking by a compiler will keep them from making mistakes.

The problem with this expectation of the power of early type checking is two-fold.

• It doesn’t really work


• It is overly constraining

It doesn’t work because the bugs caused by type-checking errors is a small subset of buges in general. Besides, in a fully object—oriented language like Ruby or Smalltalk, they are rare (because the methods which actually operates on the data is actually automatically strongly type-bound to the data by virtue of it being a method. It’s overly constraining because it unnecessarily couples the implementations of the provider and users of the services provided by an object.

Rather than droning on about the failures and contraints of early-type checking, which I suspect I’ll address more fully here as time goes on. Allow me to introduce the notion of types in an object-oriented languagethat I’ve come to accept over many years.

Let me suggest that in a fully object-oriented language types are not properties of objects or classes, but properties of variables which code using an object uses to refer to those objects. In saying this I’m recapping the key idea behind an IBM Technical Report I wrote back in 1991 entitled “Types fromthe Client’s Viewpoint” Which is available here as a pdf file.

“But wait a minute!”, you say. “Aren’t types for matching objects with variables, so that programmers won’t make mistakes when they manipulate the object?” To which I reply, “That’s not object-oriented.”

Now of course many will argue with this, but I’d like to revisit the original conception of object-orientation, a term which was coined by Alan Kay, the inventor of Smalltalk, a language from which Ruby has borrowed a lot.

The key idea behind “object-orientation” a la Smalltalk and Ruby is the encapsulation of implementation behind an interface defined by a set of messages. Kay felt that the traditional way of structuring software as programs which processed data, and the attendant separation of program and data was somthing to be avoided. His view is that it isbetter to compose software out of components which all look like little computers, which contain and bind the programs and state together. This was a synthesis of ideas which he had seen over many years, although they had never been codified before he did it.

For an insight into Kay’s thinking, have a look at the article he wrote some years ago about “The Early History of Smalltalk”

Another way of looking at this idea, is that objects should be like little server machines, the exact set of services available and their implementation can vary from server to server, but clients can still use those services in isolation from those details.

In both Ruby and Smalltalk, classes and inheritance are used for implementation sharing, in order to achieve economies in implementing those little computers (objects). This is not an essential feature in Kay’s conception of OO. The original versions of Smalltalk had neither classes nor inheritance which were introduced in later versions.

In parallel with the evolution of Smalltalk, a family of languages based around the concept of Abstract Data Types became an important Computer Science research topic. Abstract Data Types arranged data types into a hierarchy so as to categorize their use as a way to structure more traditional procedural programs, and they called these abstract data types, classes, and the relationship between classes inheritance, so subclassing was simply a synonym for subtyping.

Since, by the time Smalltalk had been made widely known by the publication of a special issue of Byte magazine devoted to Smalltalk-80 in 1981, it had added classes and inheritance, most observers mistakenly thought of classes and inheritance as essential features of the language. One of these was Peter Wegner who wrote an influential paper Dimensions of Object based Language Design which was published in the proceedings of the 1987 OOPSLA conference. In this paper, Wegner defined “object-oriented” languages as those cobining “objects+classes+inheritance.” What he missed was that in Kay’s conception of object-orientation classes and inheritance were merely implementation artifacts, and that the encapsulation of implementation within an object was the big idea which separates object-oriented languages from procedural languages, and despite the similarity in terminology, abstract-data-type languages, which presented instances of those data types as things to be manipulated from the outside, were far from object-oriented, at least as Kay conceived it.

So to wrap this up for now, duck-typing is a natural consequence of the nature of how objects with strongly encapsulated implementations should interact. The notion of objects containing the code which manipulates them, and hiding that implementation behind a wall, through which messages are sent, is a paradigm shift with powerful consequences. Because many so-called “object oriented” languages missed this paradigm shift, many of their users are uaware the power of thinking about software design from this new perspective.

I hope, that in some small way, this blog will help expose the beauty of this paradigm shift.