Debugging Strategies and Techniques

Debugging is both an art and a science, and as such there are absolutely techniques and strategies that can be learned, but there is also a significant benefit to practice and experience. There are a lot of good resources on this topic, both on the Internet and in print, but this article will gather a few basic techniques (and give you pointers to more information) to get you started.

Resources

The following books contain excellent advice on debugging and testing (which go hand-in-hand, as we’ll discuss below!):

• The Practice of Programming. Brian W. Kernighan and Rob Pike. Addison-Wesley 1999. ISBN 978-0201615869.
• Code Complete (2nd Ed.). Steve McConnell. Microsoft Press 2004. ISBN 978-0735619678.

What is Debugging?

Debugging is the practice of finding existing bugs in your code once they have reared their ugly heads in the form of failures when your program is running. It may involve any of the following, as well as other techniques:

• Inspecting the source for errors
• Examining program output to try to identify patterns
• Using a “debugger,” a piece of software designed for this purpose
• Logging program state as it runs to detect inconsistencies

Testing

The best way to improve your debugging is to not have to do it at all. The best way to not have to debug at all is to write good tests. (Well, aside from writing correct code, which turns out to be harder than many of us feel it should be!) Tests help you identify problems in your code before they are buried deep in your program and difficult to diagnose. If you write your tests along side your code development, you can hopefully prevent the bugs from piling up too deeply in the first place.

There are numerous testing methodologies, and that’s not exactly what this page is about, but we’ll quickly go through a few of them.

Unit Testing

Unit tests are very small, targeted tests that are written at the level of individual functions, objects, or subsystems within your code. A set of unit tests for a function would call that function with a variety of inputs designed to exercise its limits, erroneous conditions, and “expected” values, and compare the results with known good outputs. For example, consider the following C function definition:

/* Sorts a list of `nelems` integers stored in `array` in numerically
 * non-decreasing order. */
void int_sort(int array[], size_t nelems);

Useful unit tests for this function might include:

• Sorting an array of zero elements (nelems == 0)
• Sorting an array of one element
• Sorting two elements that are already in sorted order
• Sorting two elements that are in reverse-sorted order
• Sorting two elements that have the same value
• Sorting the array [INT_MAX, 0, INT_MIN]
• Sorting several randomly-generated arrays of varying sizes:
  • Even sizes
  • Odd sizes
  • Prime sizes

If a battery of these tests all pass, you can have some confidence that the sorting function works correctly. If one or more of the tests fail, you can debug the sort function before it is embedded in your application and causing strange failures only when the user provides input of particular characteristics.

For objects or subsystems, unit testing may include examining the internal state of the individual objects or variables within the subsystem to ensure that they are properly maintained. For example, unit tests for a doubly-linked list class might verify that the links of the list are properly maintained by comparing the prev and next pointers of adjacent nodes to ensure that they are reflexive.

Functional Testing

Functional testing is similar to unit testing, but normally performed with respect to an external specification, and often (but not always) on larger constructions than a single function. For example, functional testing for a programming assignment might involve writing tests based on the requirements in the handout without considering how the code is actually implemented. Given a complete specification, functional tests can often be implemented before development is even started, providing a measure of how far along the development actually is. Suppose that a program requirement says:

This program must read in a list of integers from a text file, one number per line, and print out the same list of integers in sorted order.

A functional test for this program might involve producing a similar set of inputs to the unit tests above, along with corresponding outputs, then running the program and comparing its output to the expected output. Because the functional test observes only the functional behavior of the program being tested, any conforming implementation of the program should pass the functional tests — in this particular example, it could be written in any programming language at all!

Regression Testing

Once you have a function or program that is known to do something correctly, you can start employing regression testing. Regression tests compare behavior of the current version of a function or program to some previous version of the same function or program. If the previous version was known to be correct for a certain input, then the new version should be, as well! This is often accomplished by recording the output of a version of the program in some way for later comparison. With version control systems, the previous version may be able to be checked out directly for comparison.

Regression tests are unique in this list in that they do not necessarily indicate a bug in the program when they fail; instead, they indicate a change in the program. When a regression test fails, it is the programmer’s responsibility to examine the expected output versus the output produced and determine whether or not the change was intentional. If it was, then the new output becomes the standard against which future regression tests are compared.

Debugging

The first thing that a programmer must do when debugging is establish that there is a bug. Sometimes this is easy (e.g., the program is crashing!), and sometimes it is harder. For example, the program output may not be what the programmer expected, but it may not be clear whether this is a bug, a different but equally correct behavior, or a misunderstanding of the program specification.

Once it is established that there is a bug, a wide variety of overlapping and complementary techniques can be brought to bear to identify, locate, and fix the bug. The very next thing to do, however, is to document the bug. Write it down! Sometimes the very act of describing the bug will make its source obvious. At the very least, if you find that you cannot fix the bug, documenting its behavior will be a necessary step in asking for help or advice.

With an identified bug and its description in hand (perhaps something like “inserting a duplicate element into a sorted list causes a segmentation fault,” or “the number of neighbors for a cell on the right edge of the grid is calculated incorrectly”), debugging can begin.

Understanding the Behavior

Before any mechanical techniques for finding and fixing bugs are discussed, the first thing we must point out is that understanding the bug is your most powerful tool, and that there is no substitute for thinking. It is not uncommon for an experienced programmer to identify, locate, and fix a bug immediately upon being told of its existence, simply because she understands the bug and can mentally trace it back to likely causes and suspect code.

When you have identified a bug, sit down and think about three things:

• What did I expect would happen?
• What did happen?
• How are these things different?

Sometimes the mere act of considering these facts will lead to a hypothesis about where the bug is located, or what sort of error it might be. For example, if you expected that a sorting algorithm would treat negative numbers as smaller than zero, but you observed that it treated negative numbers as larger than positive numbers, and you remember that two’s complement negative integers have the same binary representation as very large unsigned integers, you might immediately proceed to the sorting comparison function to see if it is using unsigned math.

Facility with understanding the problem and immediately identifying bugs, or likely locations for bugs, comes only with practice. Don’t feel bad if you seldom have success immediately fixing your bugs once you articulate the problem; keep practicing, keep programming, and you will get better at it.

Repeatability

Bugs are easiest to fix if they are repeatable. This means that you can trigger the bug (more or less) reliably. A bug that only occurs sporadically under unknown conditions can be very difficult to fix. On the other hand, a bug that is known to occur every time a particular input is provided may be much easier.

It is worth it to take the time to find a way to repeat a bug, if you have been looking for the bug for more than just a few minutes. Ideally, you can create an environment under which you can run some simple command or perform some simple action and the bug is triggered. Being able to repeat the bug on demand means that you can try many different techniques to find the bug without having to wait for it to naturally occur.

Localizing the Problem

Identifying the specific part of the program where the bug is occurring is very powerful, as it reduces the amount of code that must be examined to fix the bug. It is also dangerous, because it is possible to become convinced that the bug is in one place, while it is actually someplace else! It is normally relatively easy to at least identify the location where the bug is first observed, which gives the programmer a place to start looking for where it actually occurs. If, for example, you find that a data structure is corrupt in frobnicate.c on line 132, then you can look at the surrounding function to see if it might have occurred there, or trace the offending structure back to wherever it came from to look there.

Code Examination

If you have a string suspicion that a bug is in a particular function, loop, or other localized portion of code, look at that code. In particular, try to look at that code through unbiased eyes. In programming, as in writing, we often overlook our own mistakes — you know what you meant to do, so you sort of assume you actually did it. Step back, look at the code line-by-line, and ask yourself “what does the computer see when it runs this line of code?” Run through a few iterations of the suspicious loop, writing down the state of the local variables and their individual transforms based on the code, not based on your understanding of the algorithm. Look closely at format specifiers and variable references for incorrect usage of whitespace or referencing/dereferencing operators.

Possibly most importantly, don’t begin making changes until you are fairly certain that you understand what the code is doing and how it might be wrong. If you make changes that you don’t understand to code that you don’t understand and the bug goes away, you will never know if you have actually fixed it, or if it is just covered up by some different behavior.

The Debugger

Most languages on most platforms will have a dedicated debugger. For C on a modern Unix platform, for example, you normally will have access to gdb or lldb. For Java there is jdb, and most integrated development environments provide their own debugger (or wrapper for jdb). It is absolutely worth your time to learn at least a little bit about how to use the debugger for your system. The debugger can provide you with one of the most commonly useful pieces of information for debugging a crashing program: the backtrace.

A “backtrace” is the list of functions that were actively in progress at a given point in time. Often, that point in time is “when the program crashed,” but experience with a debugger will help you take backtraces at other points of interest. Like understanding the problem, it is not terribly uncommon that simply seeing the backtrace will trigger an understanding of where and why a bug might be occurring.

Logging State and Progress

Logging the state of your program (printf()-style debugging) is a tried-and-true method of finding the source of an error. If you have established that a certain set of input conditions causes a particular function to go awry, you might insert a sprinkling of print statements into that function to log its progress through a computation. If you print messages that are meaningful and consistent in formatting, you can rapidly scan the output for unexpected values.

Don’t forget that programmers can write programs to help them debug their programs! A very powerful technique for log-style debugging, particularly when the logs get large, is to write a simple program (perhaps in a scripting language where development is rapid and string-handling is easy) to scan the logs for you, and identify unexpected occurrences. If you have 10,000 lines of log output and a quick five line hack of a Python script can narrow that log down to 10 lines of interest, you have more than saved the time it took to write the script.

Logging is not a replacement for using the debugger, but sometimes it is easier and faster than creating a complicated debugger configuration to catch a particular bug. In particular, whereas you have to set up breakpoints and watchpoints and print values of interest every time you run the debugger, your logging can remain a part of the program until the bug is identified (or even afterward, if it can be turned off for normal operation).

Understanding the Problem … Again

Remember to circle back to understanding the problem. As you drill down into the behavior of your program using the above techniques, the information that you have about the bug will change. If you stop periodically to update your mental model of the bug, you will both formulate better experiments to help identify the bug and increase your chances of finding the bug through understanding and experience. It is not uncommon to “find” a bug by seeing an unexpected print statement in one place, making a mental connection to a possible error in another place, and quickly fix the bug by manually examining the source code in the indicated location.

Conclusions

Effective debugging is, first and foremost, a matter of understanding. Understanding what your code is supposed to be doing, understanding what your code is doing, and understanding the relationship between the two. Testing, printing, and the debugger will never be a substitute for reading the specification, examining the code, and thinking about the problem.