Example 4: Debugging Bricks

SIGCSE 2006 Workshop Companion Web Site

Goal

This example will introduce you to how an assignment can be written to focus on testing and debugging, rather than on coding. Such an assignment can be written so that it requires students to write (and turn in) their own tests, or just to identify and fix bugs, or both. For context, imagine that you are given a pair of classes that represent a single brick and a pallet stacked with layers of bricks. The code you will be given contains at least four bugs. You have two tasks: write a complete set of tests for the classes you are given, and find and repair all of the bugs.

Note: If you are unable to complete this example, contact Stephen Edwards for access to a set of completed files.

Learning Objectives

• Familiarity with writing tests on pre-existing code
• Familiarity with assignments focused on testing and/or debugging, rather than writing new code
• Familiarity with automated assessment using reference tests

Part A: Writing Test Cases

Procedure:

1. Set up your IDE

   Follow the instructions in Example 1 if you need help. The remainder of these instructions are written for Eclipse users, but should be similar for other environments.

2. Create a Bricks project

   Again, follow the instructions in Example 1 to create a Bricks project consisting of the files in the Bricks directory. Feel free to collapse other project trees so they do not clutter your view.

   Expand the Bricks project in the left pane so that you can see its contents. Feel free to collapse other project trees so they do not clutter your view.

3. Open the Bricks.java and Palette.java files

   Review the two classes to familiarize yourself with the code in this project. The following picture may help you visualize a palette with bricks on top:

   

4. Open the test classes

   Two test class skeletons have already been provided for you. Open them and take a look--they don't have any test cases yet.

5. Write test cases

   1. Brainstorm test case ideas. Try to come up with a list of test cases that you would use to double-check the behavior of the two classes under consideration.

   2. Construct all of the test cases you have come up with, compiling and running them as you go. If you identify any unexpected behavior, note it, but do not begin making corrections yet. Instead, focus on writing tests that demonstrate the "error" you have found so that you will not have trouble locating it again--then move on to writing other test cases.

   3. When writing test cases that deal with double values, remember that you cannot do this:

          public void testFoo()
          {
              double result = myObject.foo( ... );
              assertEquals( 1.0, result );  // won't compile!
          }
      

      This will not compile, because it is risky to compare double values using ==. A double value normally contains a mere approximation of the desired value, so you may end up in situations were A * B * C does not actually produce a value that is == to C * B * A, although the two are very, very close to each other. Instead, when comparing doubles, assertEquals() requires you to state how "close together" two double values must be in order to be considered "the same":

          public void testFoo()
          {
              double result = myObject.foo( ... );
              assertEquals( 1.0, result, 0.0001 );  // works!
          }
      

      For example, if foo() produced an answer in kilograms, requiring it to be within 0.0001 of the expected result is the same as requiring it to be accurate to within one tenth of a gram. The assertEquals() test will succeed only if the absolute value of the difference between the actual value and the expected value is less than the given tolerance.

   4. Make sure that you have included tests for all public methods of all classes. Further, for methods other than trivial field getters or setters (that simply return or assign to a field), make sure you have done more than simply tested the method once under "normally expected" circumstances--are there any special values to consider, or unusual circumstances where this method is intended to behave differently?

   5. Think about the thoroughness of your testing--do you think you have found all the mistakes in these classes yet?

Part B: Debugging

Procedure:

1. Fix any bugs revealed by your testing

   For each failed test case that you have written, find the cause and fix it.

2. Identify and fix all remaining bugs

   1. There are at least four mistakes in the two classes provided for this exercise. Careful testing may reveal many of them (but not all).

   2. Use all of the techniques at your disposal to find all of the mistakes you can. Some code inspection and logical deduction will be necessary to reveal some of the bugs (and paying careful attention to the warnings in the Problems tab on the bottom of the Eclipse window can help!). For any mistake that affects the behavior of a method, first be sure to write one or more new test cases that demonstrate the presence of the mistake by failing. Fix all mistakes you find.

   3. Keep count of the number of mistakes you have found. When you think you have found all possible mistakes, compare with a friend to see who has found the most. If others have found more than you, keep looking until you catch up.