Yesterday, I happened to catch a bit of the annual Macy*s Thanksgiving Day Parade in New York.
In particular I saw Arlo Guthrie and his daughter perform Arlo's father, Woody's, song "This Land is Your Land" while standing in front of a giant Turkey on the Ocean Spray cranberry float.
I was struck by the combination of Arlo's still youthful face, framed by long white hair. Arlo is getting on in years. Thankfully he has outlived his father, who died of Huntingdon's disease at age 55 by some 8 years.
As an aside, although I grew up in the Connecticut suburbs of New York City, I've only gone to the Macy*s parade once in 1997, after I'd moved to North Carolina. That day was very windy and the crews handling those big parade balloons had their hands more than full. The Cat in the Hat balloon knocked over a lamp post, and a parade-goer was struck in the head with falling debris and suffered a month-long coma, my wife and I witnessed the Barney the Dinosaur being stabbed and stomped on by police to avert another accident. "Oh, the humanity!"
And I was reminded of Arlo again today when Andy Ihnatko picked the song Alice's Restaurant Massacree for his Amazon advent calendar.
For those readers of tender years who might not be familiar with Arlo and Alice's Restaurant, the song tells the tale of a communal Thanksgiving feast in a deconsecrated church in Great Barrington, Massachussetts, the home of his Alice and Ray Block. After the dinner Arlo and a friend volunteered to get rid of the post-feast trash, and loaded it into a VW Microbus, bound for the dump, only to find that the dump was closed for the holiday. So they dumped the trash over a fifteen foot cliff by a country road on top of a pile of trash that was already there, figuring "one big pile is better than two little piles." Two days later they were arrested for littering, when the local constable Officer Obie (no his last name wasn't Fernandez) discovered an envelope with Guthries name on it under the pile of garbage, and Arlo when asked about this said "Yes, sir, Officer Obie, I cannot tell a lie, I put that envelope under that garbage."
This led to a trial, for which he was fined $50 and ordered to pick up the garbage.
The story continues when Guthrie was later ordered to report for induction into the Army. Again, for the benefit of my younger audience, I probably need to explain that, during the Vietnam War, the US had a military draft, unlike today's "volunteer" military which attracts recruits out of a mixture of patriotism and economic necessity.
The upshot of the story was that after going through a series of medical examinations at the induction center, he was nearing being sworn into the Army, except that "the last man" asked a crucial question, "Kid, we only got one question. Have you ever been arrested?"
In reply Arlo told the tale of his conviction for littering, and was sent to the "Group W" bench.
Here's how Arlo decribes Group W.
" Group W's where they put you if you may not be moral enough to join the army after committing your special crime, and there was all kinds of mean nasty ugly looking people on the bench there. Mother rapers. Father stabbers. Father rapers! Father rapers sitting right there on the bench next to me! And they was mean and nasty and ugly and horrible crime-type guys sitting on the bench next to me. And the meanest, ugliest, nastiest one, the meanest father raper of them all, was coming over to me and he was mean 'n' ugly 'n' nasty 'n' horrible and all kind of things and he sat down next to me and said, "Kid, whad'ya get?" I said, "I didn't get nothing, I had to pay $50 and pick up the garbage." He said, "What were you arrested for, kid?" And I said, "Littering." And they all moved away from me on the bench there, and the hairy eyeball and all kinds of mean nasty things, till I said, "And creating a nuisance." And they all came back, shook my hand, and we had a great time on the bench, talkin about crime, mother stabbing, father raping, all kinds of groovy things that we was talking about on the bench.
After that a sergeant came over and gave him a long form to fill out with the details of his crime, ending with the question "Have you rehabilitated yourself?" Arlo took the form to the sergeant and said:
"Sergeant, you got a lot a damn gall to ask me if I've rehabilitated myself, I mean, I mean, I mean that just, I'm sittin' here on the bench, I mean I'm sittin here on the Group W bench 'cause you want to know if I'm moral enough join the army, burn women, kids, houses and villages after bein' a litterbug." He looked at me and said, "Kid, we don't like your kind, and we're gonna send your fingerprints off to Washington."
So What Does This Have to Do with Ruby?
I'm glad you asked that question.
The answer is that the main enumeration method names of Ruby, collect, select, reject, inject... came to Ruby from the Alice's Restaurant Massacree, indirectly via Smalltalk.
Some Ruby programmers, seem to have an aversion to the inject method. It seems particularly irksome to programmers who have come from languages which use map and reduce for collect and inject. Later ruby versions have made aliased map and collect, and reduce and inject. But this leaves map as taking an optional argument, which if passed is used as an artificial first element to be effectively pre-pended to the enumeration being mapped. That optional argument and the name inject comes from Smalltalk's inject:into: method which makes sense to a Smalltalk programmer since you are injecting that initial value into a reduction of the collection using the block given by the second argument. Smalltalk programmers get used to passing the 'right' value for that argument, for example if inject is used to sum a collection the initial argument is normally the identity value for addition, e.g. 0, or might be something else if we want to add the sum to another value.
A couple of months ago, I was listening to a podcast interview with Dan Ingalls who was the first implementor of Smalltalk, and probably the key contributor as Smalltalk evolved from Smalltalk from Smalltalk-71, through Smalltalk-72, and Smalltalk-76 up through Smalltalk-80 which is what most people think of as Smalltalk today. During the interview he was asked about the origin of those enumeration methods of the Smalltalk collection classes. Alan Kay had told the interviewer that they had come from a song. At first Dan didn't remember this but then remembered that there was a song which had a string of words like inject, select, detect etc. As far as I recall, though he didn't name the song.
But I recognized it right away, here's how "Alice's Restaurant Massacree" transitions from the littering trial to the draft:
Came to talk about the draft. They got a building down New York City, it's called Whitehall Street, where you walk in, you get injected, inspected, detected, infected, neglected and selected.
So Dan picked the collection enumeration method selectors in Smalltalk from "Alice's Restaurant", no doubt. I suspect that that initial argument of inject:into: came about because he wanted to use that pattern and map and reduce didn't fit. Actually I'm not sure that map and reduce were commonly used terms at that time.
So if you don't like inject in Ruby, don't blame Matz, blame Dan and Arlo!
I just got an email this morning informing me that my e-mail entry won a contest on boingboing gadgetsto win an original copy of each of the original Xerox Parc Smalltalk-72 and Alto manuals.
I'm looking forward to adding these to my collection of tech mementos.
The winning entry pointed to two articles I'd written here about Alan Kay
- Alan on the meaning of OOP
- An article on a Business Week comic strip about Alan and the invention of the Dynabook, which triggered some memories of some other past associations
And I also threw in this scan of one of my current treasured mementos. This is an autograph page from the supplemental programming manual for Digitalk Smalltalk/V PM which was presented to me at Digitalk's developers conference. I'd been working closely with Digitalk on the release in my capacity as their liaison with IBM, and I gave a talk there.
Alan was also a speaker there. Alan had been instrumental in getting Digitalk to change the name of their product from Methods (which they'd picked to avoid issues with Xerox) to Smalltalk/V. I'm pretty sure that I'd already met Alan at one or two IBM internal OOP conferences. So Alan signed too, and I've highlighted his personalized autograph.
If you want you can click on the picture for a larger view.
Yesterday, Travis Griggs posted an interesting article on his blog about a couple of tricks he used to write a test which needed to ensure that a race condition actually happened during the test.
And Randal Schartz just discovered it too. These two posts point out some interesting similarities and differences between Ruby and Smalltalk.
Singleton methods
A lot of folks point to Smalltalk as a source for the kind of metaprogramming techniques that we Ruby programmers take for granted. Smalltalk does allow a lot of metaprogramming, but things like instance-specific behavior aren't part of the standard repertoire of most Smalltalkers. That's why Randal, who is no slouch at Smalltalk, expressed a certain amount of amazement at how Travis did this. Here's his (Travis') example Smalltalk code:
p := 4 @ 3.
p changeClassTo: (p class copy superclass: p class).
p class methodDictionary at: #negated put: (p class methodDictionary at: #transpose).
p negatedFor those unfamiliar with Smalltalk, the expression 4 @ 3 creates an instance of Point, which is a 2-d point object with x=4 and y=3.
Let's write a Point class in Ruby which works like a subset of Smalltalk's Point
class Point
attr_accessor :x, :y
def initialize(x, y)
@x, @y = x, y
end
def negated
self.class.new(-x, -y)
end
def transpose
self.class.new(y, x)
end
def inspect
"#{@x} @ #{@y}"
end
end
p = Point.new(4,3) # => 4 @ 3
p.negated
p.negated # => -4 @ -3
p.transpose # => 3 @ 4Now , the above code might look something like this:
p = Point.new(4,3)
def p.negated
transpose
end
p.negated # => 3 @ 4Most Rubyists of any experience will recognize that what I've done is define a singleton method for that sole instance of point which overrides the negate method by calling transpose instead.
Another way to do this which might be a little bit more like what Travis shows in Smalltalk might be:
p = Point.new(4,3)
class <<p
alias_method :negated, :transpose
endLet's look at Travis' example and how he creates a 'singleton' class in Smalltalk. Here are the relevant lines:
p changeClassTo: (p class copy superclass: p class).
p class methodDictionary at: #negated put: (p class methodDictionary at: #transpose).He gets p's class (which is Point up to now), copies it, and sets it's super class to p's class, effectively interposing the copied class object. Next he gets the transpose instance method from the copied class's method dictionary and replaces the negated method. Effectively what Ruby's alias_method does.
Now, I should point out here that Travis's example is probably specific to the Cincom VisualWorks Smalltalk dialect. The changeClassTo: method doesn't seem to be available in Squeak, although there might be a similar method, and I don't recall a similar method in the Smalltalks I've used in the past. Also changeClassTo: isn't entirely guaranteed to work, besides the cases where Ruby can't create a singleton class (for example immediate objects like FixNums), changeClassTo: requires that the 'shape' of the instance conform to the instance variable template defined in the new class, simplifying things a bit, that means that it needs to have either exactly the number of instance variables expected by the class, or at least the number of fixed instance variables expected by the class. Smalltalk classes can have a variable set of indexed instance variables which are placed after all the fixed (named) instance variables.
So let's end this exploration of comparative instance specific method creation between Ruby and Smalltalk before I move on to another interesting difference which Travis' article exposes:
- Smalltalk provides dialect-specific mechanisms for monkeying with the class of an object and manipulating methods.
- Ruby has some nice syntactic sugar for doing this in a way which is part of the language definition.
As a result, these techniques are far more commonly used in Ruby than in Smalltalk.
Now for the second difference, one where Smalltalk leaves Ruby a bit behind.
Turtles All the Way Down
One of the really interesting things about Smalltalk is just how much of the runtime system is exposed as 'normal' Smalltalk objects. In Ruby certain things are hidden away (sometimes not so securely) from the Ruby programmer, there's a "Wizard of Oz" behind the curtain which separates to the Ruby program from the VM (or interpreter if you prefer).
Almost everything the VM deals with in Smalltalk pokes up as one or more Smalltalk objects. In Smalltalk everything really is an object, it's turtles all the way down, at least it looks that way.
Travis' motivating problem was to force a race condition to occur on demand. Anyone who has done concurrent programming has learned that race conditions have probabilistic minds of their own.
The lever that Travis pulled was the fact that Smalltalk threads (which Smalltalk calls Processes for historic reasons) are implemented as Smalltalk objects which can be manipulated directly from Smalltalk. The Smalltalk IDE makes use of this. For example the Smalltalk debugger is really just a specialized inspector which inspects a Process object, including showing it's state (the stack frames) and manipulate it (breaking, stepping, and responding when class definitions change).
So Travis could grab the instance of a single process and change its instance-specific behavior to modify the way it handled termination in order to simulate the condition he needed.
This aspect of Smalltalk is something which many maybe too many Smalltalk programmers are familiar with. The second part of that old article link I just snuck in talks about the distributed version of Smalltalk I did at IBM many years ago which added a new kind of Process proxy which tied execution threads which crossed machine boundaries together allowing distributed debugging, exception handling etc.
Can Ruby get along without this? Sure. But it is one thing I miss from Smalltalk
Recently posted on Ruby Talk, under the title ‘This System Must have been Written in Ruby’
At 1950 feet (around 700 meters) the airplane’s left altimeter suddenly and mistakenly registered an altitude of 8 feet (about 2 meters) below sea level and passed the reading on to the automatic control system, Van Vollenhoven said.
But the autopilot reduced gas to the engines and the plane lost speed, decelerating until, at a height of 450 feet (150 meters) it was about to stall. Warning systems alerted the pilots.
It appears that then the pilots immediately gave gas, full gas, however it was too late to recover,’’ ...The plane fell into a freshly plowed field, striking the ground tail first and breaking into three pieces.
I assume the poster’s position is that this is the kind of thing that happens when non-statically typed languages are used to implement mission critical functions.
I use to hear the same argument about C++ vs Smalltalk, from no less a luminary than Bjarne Stroustrup, who was wont to say that he would hate to fly in an airplane whose autopilot might throw a doesNotUnderstand: exception because of a type violation.
Okay, I admit it, the static type zealots are correct. Static typing absolves the programmer from worrying about:
- Conceptual errors in the domain model.
- The fact that a floating point number is a poor approximation of a real number.
- Incorrect mathematical formulae, either due to misunderstanding, or typos.
- User errors.
- Off by one errors
- Buffer overflows
- Bad input data
- Adding instead of subtracting, or vice-versa
- Hardware failures
- Errors in concurrency control
- Your favorite bug here
- The need to properly test the system.
If you believe that, or any significant portion of it, let me tell you about this bridge between Brooklyn and Manhattan which I have to sell you!
And if you believe that and rely on that faith to implement mission critical systems, please feature it in your advertisements, and let us know what mission critical systems were built that way, so that the rest of us can avoid them whenever possible.
Get Real!
I just discovered that my old friend, Ward Cunningham has started putting some short interviews, and “video essays” up on YouTube
In the embedded video above, Ward relates how he came up with the metaphor of debt to explain the need for refactoring in interative development. The key insight is that you can proceed faster with an incomplete knowledge of the problem you are solving, and learn as you go. In many cases this is essential because for many problems the problem itself changes as implementation proceeds, because of effects like new feedback from users once they see the system starting to emerge. Ward compares implementation with incomplete knowledge to working with borrowed money. As the system implementation proceeds, the assumptions made on incomplete or out of date knowledge, which drove the design, or “architecture” if you prefer, of the system, start to make implementation of the more complete or evolved requirements progressively harder. In the debt metaphor, this increased cost of making progress corresponds to paying interest on the loan. Refactoring is a process of redesigning the software to reflect this new problem knowledge, without breaking it, thus “re-financing” the system.
Before I’d heard Ward express this metaphor, I’d come up with another for the same idea. One of my hobbies is metalworking. I saw an evolving system as a piece of metal which starts out as flexible and malleable. But like metal, as you start to form it by hammering, and bending, and other operations, it starts to “work-harden.” This means that the metal becomes more and more resistant to further shaping, because of local changes to it’s crystal structure. Eventually it resists all further efforts, often by failing with a stress fracture. This is why you can take a piece of, say copper, and bend it back and forth several times until it finally breaks at the bend.
In metalworking, you can use a process called annealing which returns the metal to a ductile, maleable state. Of course the analogy here is to refactoring.
I still like both metaphors, I tend to use my work hardening/annealing metaphor when talking to programmers, and Ward’s debt metaphor when talking to business people (the “gold donors.”)
Better late than never
A few weeks back Jared Richardson interviewed me about the talk I gave at the inaugural RubyRX No Fluff, Just Stuff’s first foray into a Ruby oriented conference.
Although this was supposed to be published before the conference, it’s just now made it to the DZone web site. I hope you enjoy it.
For some reason, some people who hang out on ruby-talk seem to have a compulsion to periodically opine that “inject() is always the inefficient solution possible.”
I think that this statement, even if it were true, which I’m pretty sure it isn’t, being a claim for universality, isn’t particularly good advice.
My buddy Kent Beck has generated quite a few memes in the software development community. One of the more well known is “Make it run. Make it right, Make it fast.” Although the roots of this advice can be traced back to Butler Lampson. This is advice that anyone who wants to be a software craftsman needs to take to heart.
The order is important. First you make sure that the code runs, then you need to make it right. Making it right doesn’t mean that it does the right thing, that’s a large part of making it run, it mean’s making the code clear and maintainable, either by a future you, or by someone else. Only then you should make it fast, without unnecessarily affecting your success in keeping it right. And the best way to make it fast is to slavishly follow the 3 rules of optimization
While there are some who just seem to hate inject, it can, as you point out, be much more intention revealing, which helps in making the code right. In Ruby 1.9, the inject method also has the alias reduce, which is even clearer in the cases where there’s no initial value to inject into the enumeration. More languages use reduce for this case, and don’t provide an argument for the initial value at all. Ruby got the name inject from Smalltalk’s inject: aValue into: aBlock collection method where Smalltalk’s keyword message selector syntax does a better job of revealing the meaning of the parameters.
Now, that’s not to say that inject/reduce should be overused, the key tension is whether or not it is actually intention revealing, or overly clever, or even stupid. I confessed here some time ago, that I can be just as guilty as anyone of being cleverly stupid. But I don’t recommend it.
This is just one of those things, like when and when not to use optional parentheses in Ruby, which really need a bit of thoughtful reflection, rather than blind adherence to convention.
Via Boing-Boing, Business week has just put a comic strip on their web site, which tells the story of how Alan Kay, inspired by a visit to Seymour Papert seeded the idea of portable computers.
Long term readers of this blog no doubt know that I hold Alan Kay in high regard. I got to meet him several times, first I think at an internal IBM conference on Object Oriented Programming in the mid 1980s, Another time was at a Digitalk developers conference, where I obtained a prize possession. At the time I was the IBM technical liaison with Digitalk. They released a new version of Smalltalk/V PM at this particular conference, and I was presented a copy of the supplemental manual for the release signed by all of the Digitalk developers, and also by Alan Kay, who wrote something like “to the most non-IBM like IBMer I’ve ever met.”
Reading the business week strip, I was reminded of an even earlier personal connection with Kay and Smalltalk. A year or so before I really got into Objects, I was working on IBM Mainframe software in Poughkeepsie, NY. One summer the project got an intern, named Radia Perlman, who was in graduate school at M.I.T. And is now known as the “mother of the internet” for her invention of spanning-tree protocol. It was only later that I learned that she was one of Papert’s students working on Logo and seeing how kids learned programming with Logo, and if I recall correctly she also visited Xerox PARC, and was involved in porting the concept of the Logo turtle model of graphics to Smalltalk, where it became the Pen object.
What a long strange trip it is!
My former OTI colleague Andrew “Roo” Low just wrote an interesting article about the trade-offs involved in designing a VM (and software in general) to exploit 64-bit machine architectures. Although, Roo discusses JVM implementation, he also grounds it with his experiences developing the IBM/OTI Smalltalk VM, and the lessons would be equally applicable to, say, a Ruby implementation.
I really like seeing the resurgence of interest in my second most favorite programming language
One of the comments to the referenced Gartner Group blog post had a link to an IBM Developer Works article about using using Eclipse as a Smalltalk IDE.
It seems like a strange loop, since Eclipse is the descendant of IBMs old VisualAge. Now I happen to know a bit about this since I was involved at the beginning. I guess that enough time has passed that nothing I say here will affect anyone’s business.
IBM, Smalltalk, and VisualAge
VisualAge started out as a demonstration prototype I wrote using Digitalk Smalltalk/V. The idea was to make something like the NeXT interface builder in Smalltalk as an adjunct to the Smalltalk IDE. For those who are not hip to such things, the Interface Builder, originally a Lisp program before Steve Jobs hired the author, lives on as part of Apple’s XCode tool suite for OS/X for the Mac and the iPhone.
The demo got some key IBM executives excited, and so the VisualAge development organization was born. Development proceeded using Smalltalk/V, along with Object Technology International’s (OTI) Envy/Developer, which was the state of the art version and configuration management for Smalltalk team programming.
As the project progressed, IBM and Digitalk had a falling out over conflicting views of who should be doing what. I’d shown the prototype to them in my position as liaison between the two companies, and they launched their Look and Feel kit which was seen by IBM management as competition from a supplier. So since we already had a relationship with OTI because of Envy/Developer, and OTI was in the business of building Smalltalk VMs, including the VM used by Digitalk’s Smalltalk/V for the Macintosh, IBM switched horses and contracted OTI to build “IBM” Smalltalk. This eventually led to the acquisition of OTI by IBM Canada.
This was about the time we founded the ANSI X3J20 committee to forge a standard across the existing and emerging Smalltalk implementations. I served as the secretary of X3J20. Some of the other members included Peter Deutsch and Glenn Krasner of ParcPlace, which Xerox had spun-off from their Palo Alto Research Center to commercialize Smalltalk-80; George Bosworth of Digitalk; Allen Wirfs-Brock of Instantiations one of the more successful Smalltalk consulting companies in the Portland Oregon suburbs; David Simmons who was the brains behind the iconoclastic Smalltalk/Agents; and Bruce Schuchardt who worked for Servio-Logic or Gemstone, I can’t recall which name they were going under at the time. I’m sure that I’m forgetting some of the players, but I’ll plead senility.
One of the great joys of working in Smalltalk during it’s first “glory days” was the opportunity to work with and exchange ideas with such brilliant people in IBM, OTI, and other companies.
VisualAge grew to become an entire family of development environments for several of the IBM “SAA” programming languages. There was even a VisualAge for Cobol! All of the various flavors featured an IDE written in IBM Smalltalk.
Enter Java
When Java “reared it’s head.” IBM, of course came out with VisualAge for Java. Again, the IDE was written in Smalltalk, although, in order to accomodate Java execution, OTI developed what was called the UVM (or Universal Virtual Machine) whcih could execute both Smalltalk and Java byte-codes, and used Smalltalk to implement the Java primitive functions which were implemented in C in the Sun JVM.
IBM eventually switched its focus from Smalltalk to Java. After considerable thought about this, and conversations with those who were in a position to know, I’m now convinced that this had nothing to do with the relative technical merits, and everything to do with a combined IBM/Sun platform play against Microsoft.
VisualAge for Java did eventually move away from the UVM for Java execution, primarily because the Java language had evolved to provide a standard C api for extensions. OTI began implementation of a new JVM, which was dubbed J9, which was built from the ground up to be usable in embedded systems.
OTI had a reputation of fostering the use of Smalltalk in embedded systems. When I first became aware of them in the early OOPSLA days, they were showing off products like an HP Network Analyser, and a TekTronix Digital Oscilloscope which used Smalltalk to implement their user interfaces. They were also involved with a large semiconductor company in Texas which was planning to use Smalltalk in controllers in an automated IC fabrication facility.
Around the time of the birth of J9, IBM was becoming quite interested in embedded software, under an initiative called Pervasive Computing (PVC). Because of this, OTI now an IBM Subsidiary, started working on VisualAge Micro Edition, which bundled J9 with an cross-platform IDE written, this time, in Java. About this time, I moved from IBM to OTI, and ended up working on the VAME IDE.
The Emergence of Eclipse
Work on VAME was distributed between various OTI labs. The lab in Phoenix acted as the liaison with the IBM PVC folks, the J9 VM work was done in Ottawa, Raleigh did some applications work, and some UI work, but the bulk of the VAME UI framework was done by Erich Gamma’s team at OTI Zurich. The IDE used Swing, as most Java desktop apps did in those days, and apparently that was a bit of a struggle. OTI used to have an annual all-hands internal developer conference in sunny early February Quebec, and I’ll never forget Erich, who is quite the Alpine skier, talking about the UI while showing a slide he took at some ski report showing a sign with a precariously oscillating chair-lift and a skier about to fall, with the text “Don’t Swing!”
This led to the desire to develop a new UI framework and what is now known as the standard widget set or SWT.
And the overall IDE/UI framework which replaced VAME was, of course Eclipse.
And now it seems we’ve gone from a Smalltalk IDE hosting Java development to the reverse!
Where Are They Now
Digitalk eventually merged with ParcPlace to form ParcPlace-Digitalk. For a while, both Smalltalk/V and VisualWorks/Objectworks (the ParcPlace Smalltalk-80 products) were carried, but eventually VisualWorks survived. It’s now shepherded by Cincom, which bought the rights.
Similarly, IBM eventually sold the rights to market and develop VisualAge Smalltalk (a.k.a. VAST) to Instantiations. My old friend and colleague from IBM John O’Keefe “retired” from IBM to lead the development team.
Several of the X3J20 guys went to or through Microsoft. George Bosworth worked for several years on a precursor to today’s Microsoft DLR. Davd Simmons and Allen Wirfs-Brock both work on Javascript for Microsoft.
And, of interest to the Ruby community, Gemstone is now working on Maglev, a Ruby implementation with persistent objects based on their mature Smalltalk product.