Stage Fright, or more on Performance Anxiety

Posted by Rick DeNatale Fri, 22 Dec 2006 17:19:00 GMT

Hmmmmm

I’ve been thinking a bit about the ramifications of the C++ vs. Python vs. Ruby benchmarking fud I wrote about in an earlier post

I’ve been looking at this from the point of view of how Ruby implements Arrays, digging deeply in to array.c, but I just stepped back and I noticed something which might or might not be important.


Besides using an Array rather than a linked list, the Baus Ruby benchmark doesn’t even do the same thing as the C++ one. In C++ the list is being used as a stack, items are added to the head of the list and removed from the head.


In the case of the Ruby benchmark, elements are inserted at the front of the Array using insert(0, obj), but removed using pop, which actually removes the last element. So instead of a stack we have a queue.


The Python code has a variation on the same problem, it uses append which adds to the end of the list, but del(0) which removes the first element.


Now I know that in benchmarking only performance matters, so I guess the old adage Make it work right before you make it work fast doesn’t necessarily apply.

Having said all that, another aspect of this is that the benchmarks take a rather naive approach in the use of the classes. For example, using Array#insert in Ruby is overkill for adding to the beginning or end of an array. More targeted methods such as push, unshift exist which are at least marginally faster because they have less cases to consider.

There’s no doubt that Ruby implementation technology needs to mature. Ruby is one of the most dynmic languages I’ve played with. Instance variable access is one area which is challenging, unlike in Smalltalk, where instance variables are declared and instance variable references can be bound to fixed offsets in compiled methods, Ruby instance variables are acquired dynamically and need more sophisticated run-time access. This is similar to the challenges which the designers of languages like Self posed for themselves and addressed, leading to impressive run-time performance achievements.

Subjectively, Ruby performance is remarkably good, but it can get better, and I’m sure that it will.

Read full article 2 comments


Trackbacks

Use the following link to trackback from your own site:
http://talklikeaduck.denhaven2.com/trackbacks?article_id=128

Comments

  1. Isaac Gouy 7 days later:
    >>Now I know that in benchmarking only performance matters...<< There are other minor considerations - are the programs even doing something vaguely comparable? - are the programs written by someone who has used the languages before? - are we even slightly interested in what the programs are doing? >>Subjectively, Ruby performance is remarkably good...<< What is that supposed to mean? Remarkably good compared to what? Remarkably good for what purpose? Remarkably good in what circumstances? What is "remarkably good" supposed to mean? It's "like a dog's walking on his hinter legs. It is not done well; but you are surprised to see it done at all."?
  2. Phil 3 months later:

    Here’s another attempt to speed up Ruby:

    http://t-a-w.blogspot.com/2007/02/ruby-and-judy.html