Talk Like A Duck

A Meeting with "Gill Bates"

Posted by Rick DeNatale Fri, 15 Jun 2007 20:27:00 GMT

A recent thread on ruby-core which touches on the issues of subclassing prompts me to tell another war story.

Back in the late-1980s to early 1990s, the software industry was starting to warm to the possibilities of object-orientation. I’d gotten to be seen as one of the experts on OO inside IBM so I found myself in a series of interesting meetings, but inside IBM and with other companies.

That’s how I found myself at the headquarters of a well-known company in the Seattle area which at that time was one of IBM’s major partners. This company was the major supplier of IBM’s operating systems for personal computers. I don’t want to give too many hints, so I’ll give a pseudonym to the CEO, let’s just call him “Gill Bates”, and I’ll refer to the company as “BigSoftware” or “BS” for short.

Now some of the guys I used to work with at IBM might remember a co-worker in IBM Cary, and then RTP, named Gil Bates, different guy.

The meeting was to discuss the BS plans for a new operating system with an object-oriented implementation and application programming interface. A small team of BS engineers presented to a small team of IBMers. I found myself sitting across a conference table from the presenters and right next to Gill.

Frameworks, Frameworks, Everywhere

Let me step back a bit and give a little techno-historical perspective for those who weren’t involved in software in the late 1980s/early 1990s.

One of the things which was exciting people at that time was the emergence of application frameworks. MacApp had been around a bit, and had gotten some publicity. This was a framework written in Object Pascal which made writing Macintosh applications considerably easier. MacApp provided inheritable classes which implemented a vanilla application conforming to the Mac UI guidelines. Writing a MacApp application consisted of subclassing the framework classes and adding the behavior which made the app what you wanted it to be.

MacApp spawned other frameworks, ET++ was a port to C++ done in Zurich by a team, which included Erich Gamma, if I recall correctly.

MacApp itself later was ported to C++, which led to Apple’s ill-fated Pink project. I’ll probably tell a war-story or two about Pink here sometime.

Of course the framework notion has long legs, for example Rails is such a framework for writing web-based applications.

Back then, the idea of applying the idea not just to applications but to operating systems seemed natural to many. MacApp and its ilk contained object-oriented wrappers to OS apis. Why not just make the whole OS a framework?

As an OO evangelist I’d been caught up in this as well, selling the idea of frameworks. I was still pushing for application frameworks, but the idea of a framework-based OS seemed pretty neat until I started thinking harder about it.

The Blessing and Curse of Inheritance

Object Orientation to my mind is about encapsulation. The implementation of an object is hidden from the users of the object. Inheritance is a powerful implementation technique, but it breaks the encapulation between superclass, class, and subclass. Like it or not, subclasses need to be written with knowledge of, and track changes to, the implementation of their superclass. That’s okay as long as the superclass remains relatively stable.

The problem is that you can’t expect that kind of stability over time. As the framework evolves, new methods and features get added to the framework provided superclasses, and worse from the point of view of the framework subclasser, the inheritance relationships between the framework classes often change. It’s the natural result of refactoring.

I’d played with about three releases of MacApp, and noticed that with each release, I needed to rewrite substantial parts of my applications if I wanted to step up to the new version. This was okay in that, since the MacApp framework classes became part of my application, the application was a self-contained whole, and would continue to run as long as I didn’t rebuild it with the new framework version.

Dealing with the inevitable refactorings is feasible if you have control all of the code being refactored, or at least can delay refactoring by ignoring new versions of code not under your control. The problem becomes much more difficult when the operating system interfaces require your code to inherit from the system. New operating systems will break existing applications.

Inheritance is a mechanism for sharing code DNA. In that sense it’s like sex. It’s best when it’s restricted to consenting adults, practicing using safe techniques, and aware of the potential consequences.

The State of the Art circa 1990

The difficulty of maintaining compatibility between releases is exacerbated when the techniques used to provide inheritance are statically bound. At the time of my tale, it was conventional wisdom that in order to get reasonable performance out of an object oriented program the classical C++ approach was the only way to go. This involved using arrays of pointers to the method functions, called virtual function tables, and mapping method names to an integer sequence from 1 to the number of methods needed. For a given class, the number of the first method it introduced (i.e. a method which was not just an override of an inherited method) would be one more than the maximum number assigned to its superclass’s methods. These numbers were determined statically at compile time, and needed to be known when a call to a method was compiled.

This meant that a simple act such as adding a new method definition in a class would require recompilation of not just that class, but all of it’s subclasses, and any code which called a method on any instance of the class or its subclasses. Besides making for long build times, it also meant that such a change would break binary compatibility even in cases where subclassing or client code which didn’t use the new method and so didn’t need to have its source code changed.

Inside IBM, something called the System Object Model was being advocated. This was an attempt to make the C++ implementation just described a little more flexible to at least partially address this problem, an issue which was dubbed the “fragile base class” problem.

But the number of refactorings which can be successfuly addressed just by loosening up the method calling mechanisms is a small subset of the refactorings which normally occur over the life-cycle of a framework. Early binding issues or not, even with a dynamic language like Smalltalk or Ruby, maintaining interface compatibility over time is a very difficult problem.

Rails, like MacApp, addresses this by letting applications be frozen to a particular version of the Framework.

Back to the Conference Room

So as you’ll recall, I found myself sitting next to “Gill Bates” while his senior architects presented their framework based operating system to be written in C++. I gently introduced the problems I just described, and it took a while for it to sink in.

I kept re-asking my questions as they came up with inadequate answers. Meanwile I noticed that, sitting next to me, “Gill” was starting to rock back and forth in his chair.

Finally, I said:

I just don’t understand how you are going to maintain compatibility after you refactor the framework for the second release.

The reply was:

We have our best people designing the system, we won’t have to refactor.

At that point, I suggested that the best people would recogize the inevitability of refactoring. At which “Gill” finally chipped in by saying to his presenter:

You guys don’t explain our stuff very well.