Are you aiming at the right bug?

Rick DeNatale — Mon, 12 Mar 2007 12:26:00 -0400

How many times has this happened to you?

You have a really tough bug which defies all attacks. Finally after pondering it for hours, or days or maybe more, you realize that the symptoms you are seeing aren’t due to the posited bug at all, but are really something else.

Some examples.

On ruby-talk, I’ve been trying to help someone who was having problems making Net::SMTP work. He had some code which included:

smtp.open_message_stream('sender@mail.com', ["dummy@dummy.com"]) do

And which was working, except that he really wanted to send an email to an address held in a variable rather than “dummy@dummy.com” He was getting an error if he used ‘a’ variable for the recipient address, and wondered why the array couldn’t take a variable.

At first it looked to me, based on the examples he was giving that he was trying to pass the name of the variable as a string, and I suggested that he just change that parameter to [email]. But he said that that still didn’t work (actually he’d already tried it.

I was getting frustrated, until he rephrased something that he had briefly mentioned in his intial post, that the error he was getting was a ‘tainted sender’ error. The problem was really that he was getting the object referenced in the variable from a web form, and the framework code he was using was, rightly, tainting these values as a protection against security exposures. There was no problem with passing values. The solution was to untaint the object after sniff-testing it.

I’ve done similar things in the past. My theory of what’s wrong becomes an idee fixee which blocks me from seeing the truth.

Two more (colorful) examples:

Some years ago I was building an application in Smalltalk which allowed for boxes to be drawn with connecting lines which were anchored to the boxes so that they moved when a box was moved. The kind of thing you need for a pert chart or other network diagram. Everything was working until I changed the lines into separate objects, when the lines just disappeared. I kept trying to figure out why the line objects weren’t drawing, until it finally dawned on me that they actually were drawing. I was using xor to draw them so that I could easily erase them in order to animate them when the boxes moved. The only way I figured this out was because in desperation, I changed from using a black line to a red one, which allowed me to see the line being drawn, then erased as the color changed from red to cyan. The problem wasn’t in the line object at all but in the animation code which was drawing then erasing the line.
More recently, I’ve been playing with a Ruby application which is a Sudoku puzzle assistant, as an exercise in learning RubyTk among other things. It displays the values of the initial squares given in the puzzle in black, and uses blue for squares which you have set yourself. The model object for the square has a boolean instance variable named initial which is set when the puzzle is initially entered so as to show this difference. Once the application is set in ‘play’ mode no other squares should have this variable change.
I’d gotten pretty far along with this, including having commands which search for constraints on the values of squares, then apply those constraints, etc. I’d recently improved the ability to save the puzzle in a file. My first pass just saved the initial values, more recently I’d added saving the state of the solution as well.

I was happily playing with testing, when I noticed that in one case finding a contraint, changed one of the squares from black to blue. I spent several days trying to figure out why the initial status of that square was being changed. I pulled out all kinds of Ruby-foo to figure this out. I made sure that the variable was only accessed by a reader and writer accessor methods so that I could print a backtrace when it changed, but I never could find the culprit.

Of course the punchline was that there was no such culprit. I finally realized what was happening when I, for what was probably the hundreth time, started the application and read in the test file containing the puzzle. That’s when I realized that the square in question was actually not one of the squares which had a value given in the initial puzzle. The real bug turned out to be that, non-initial squares weren’t being drawn correctly when the puzzle was read. It wasn’t until I did something which caused that square to be re-displayed that the value got drawn in the correct, blue, color.

Gerry Weinberg had a lot to say about such things in his thirty-six year old classic book “The Psychology of Computer Programming.” He calls this psychological set, which is the tendency for us to see what we want/expect to see. It’s what makes proofreading hard, we tend to mentally read a word like ‘dack’ as if it were printed ‘duck.’ He describes an experiment which showed that commented code can be harder to debug than uncommented code because most comments describe what the code is supposed to do rather than what it actually does.

So the next time you get stuck on what seems like a hard debugging session, step back and consider whether your minds-eye or your pre-conceptions are blocking you from considering the real reason for the symptoms you are diagnosing.

Talk Like A Duck: Are you aiming at the right bug?

Are you aiming at the right bug?