Writing your own software tests in Java is simple and clean. This guide will show you how to simplify some of the less common tasks as well.
Background info:
For student use, we provide a supplemental library (already available to all assignments on Web-CAT) that enhances basic JUnit features in a number of ways that will make your life easier. There are two steps to using this support: make sure the appropriate jar files are on your classpath, and then use a specific base class for all your test cases (even if you are using JUnit 4 techniques--use the provided base class anyway).
To set up your classpath, download the following jar file:
Place this jar on your project build path.
If you're using Eclipse and you've already added Eclipse's built-in
JUnit library to your project, ensure that student.jar
comes
before Eclipse's JUnit library in the search order--or
just remove Eclipse's JUnit library from the build path entirely.
All the test classes you create should then extend the base class
student.TestCase
. This class is itself a custom
subclass of junit.framework.TestCase
, so all normal
JUnit practices still work. The advantages of using this subclass
include:
Built-in methods to provide stdin content to code being tested, as well as to check stdout content produced.
Better messages for some assertions, and better error checking for others.
Ability to use JUnit 4 annotations and programming style, if you prefer that over JUnit 3.
Protections for System.exit()
calls in
code under test (i.e., if your code calls
System.exit()
, this raises an exception in your
test case that you can catch, rather than terminating the
virtual machine).
Extra assertions that support "fuzzy" string matching (ignoring punctionation, capitalization, spacing, etc.).
A specialized subclass, GUITestCase
, provides
full support for testing Swing applications with graphical
interfaces.
Platform-independent line ending normalization for text, so that tests do not need to include platform-specific line terminators.
Simple methods to load text from a file (for use as either input or expected output), or for writing slightly cleaner multi-line text literals in your code.
For full details, see the
Javadoc documentation
for student.TestCase
.
Import the JUnit classes so you can use them in your code.
import student.TestCase; ...
Your class should be named using the JUnit convention with the name of the class you are testing followed by the word Test.
If you class is named Student
, then your
test will be named StudentTest
.
import student.TestCase; public class StudentTest extends TestCase { // code goes here... }
Test cases are defined using methods with names that start
with test...()
.
Typically, your method will be named following the method you are
testing in your source code. The example to the right tests the
setName()
method, so it is called
testSetName()
. For more complicated methods, you
may have many tests (e.g., testFoo1()
,
testFoo2()
, testFoo3()
, etc.).
You have to test your assumptions/expectations of your code using
an assertXXX()
method call. These are provided by
the TestCase
base class.
import student.TestCase; public class StudentTest extends TestCase { private Student aStudent; public void setUp() { // fixture to be used for testing aStudent = new Student("Joe", "888-2993"); } public void testSetName() { aStudent.setName("Manuel"); assertEquals(aStudent.getName(), "Manuel"); } }
The most common assert...()
methods you will use to
express the expected outcomes in a test case are:
Method | Meaning |
---|---|
assertEquals(x, y); |
Assert that two values are equal, using the
equals(Object) method to compare them.
This is the most commonly used assert method in your arsenal. |
assertEquals(x, y, delta); |
Assert that two floating point values are equal, within
some tolerance delta. Because of the inexactness of
floating point representations, it is incorrect
to test that two floating point values are equal using numeric
equality (==). Fortunately, if you try this while using
student.TestCase , you'll get an appropriate
diagnostic error. Instead, always specify a tolerance value:
how close to the expected value does the computed number have
to be? |
assertTrue(condition); |
Assert that a boolean condition is true. |
assertFalse(condition); |
Assert that a boolean condition is false. |
assertNull(x); |
Assert that a reference is null. |
assertNotNull(x); |
Assert that a reference is not null. |
assertSame(x, y); |
Assert that two references refer to the same destination (i.e., two variables refer to the same object). This is the same as performing an == equality comparison between reference variables. |
There are many more assert methods than those listed here, but these are by far the most common.
Import the JUnit classes so you can use them in your code.
import student.TestCase; public class StudentTest extends TestCase { // code goes here... }
Fixtures are objects that are used in your test cases.
Define your fixtures as private fields in your test class (instance variables).
Initialize your fixture in the setUp()
method.
The setUp()
method is called automatically
before each test method is executed, to ensure each test operates on
a clean set of freshly initialized objects.
import student.TestCase; public class StudentTest extends TestCase { private Student aStudent; // fixture to be used for testing public void setUp() { // initialize it here aStudent = new Student("Joe", "888-2993"); } }
The two test cases on the right run succesfully, since they
each start with a freshly generated copy of the
aStudent
object.
... public void testSetName() { assertEquals(aStudent.getName(), "Joe"); aStudent.setName("Manuel"); assertEquals(aStudent.getName(), "Manuel"); } public void testSomethingElse() { // aStudent .. is new here assertEquals(aStudent.getName(), "Joe"); assertEquals(aStudent.getID(), "888-2993"); } }
main()
with Standard OutputConsider this simple piece of code.
public class HelloWorld { public static void main(String[] args) { System.out.println("Hello world!"); } }
main()
like any other methodThemain()
method is just a static method, so
you can call it directly like any other. Don't forget its
argments!
import student.TestCase; public class HelloWorldTest extends TestCase { public void testMain() { // call main()! HelloWorld.main(null); } }
System.out
The student.TestCase
base class provides
for capturing all output sent to System.out
during a test case. The history contents are reset before
every test method automatically.
Note that if you are using assertEquals()
, the
expected output must match exactly, character for character,
including all newlines.
Also note that systemOut()
's history is
automatically normalized to represent newlines as "\n",
regardless of platform, so just use \n in any expected values.
public void testMain() { HelloWorld.main(null); // Can combine these into one line if needed String output = systemOut().getHistory(); assertEquals("Hello world!\n", output); }
The student.TestCase
base class provides
an assert method called assertFuzzyEquals()
that performs more lenient string comparisons. It ignores
all differences in capitalization, punctuation, or extra
spacing, as well as all whitespace at the beginning and ending
of strings.
The exact details of what features are ignored in fuzzy
comparisons are programmable if necessary. See the API for
the StringNormalizer
class and for TestCase
for more details.
public void testMain() { HelloWorld.main(null); assertFuzzyEquals("hello world", systemOut().getHistory()); }
You can even change all your assertEquals()
calls
to use "fuzzy" comparisons, if you want.
public void testMain() { HelloWorld.main(null); // Preferably, put this in the constructor or setUp(), since // it will affect all assertEquals() that come after it, // unless you turn it off again: setUseFuzzyStringComparisons(true); // Now performs a fuzzy comparison: assertEquals("hello world", systemOut().getHistory()); }
The student.TestCase
base class provides
some helpers for more advanced string tests, including one
for regular expression testing (giving better failure messages
than just using the methods in the String
class).
public void testMain() { HelloWorld.main(null); assertTrue(equalsRegex(systemOut().getHistory(), "[Hh]ello\s.*")); }
main()
with Command Line ArgumentsConsider this simple piece of code.
public class HelloWorld { public static void main(String[] args) { System.out.println("Hello, " + args[0] + "!"); } }
main()
You can provide any arguments you want when you call
main()
in a test case.
public void testMain() { HelloWorld.main(new String[] { "Joe" }); assertEquals("Hello, Joe!\n", systemOut().getHistory()); }
main()
When It Calls System.exit()
Consider this simple piece of code.
public class HelloWorld { public static void main(String[] args) { System.out.println("Hello, " + args[0] + "!"); System.exit(0); } }
You can provide any arguments you want when you call
main()
in a test case.
public void testMain() { try { HelloWorld.main(new String[] { "Joe" }); } catch (student.testingsupport.ExitCalledException e) { // Confirm exit's return value was zero: assertEquals(0, e.getStatus()); } // Check the System.out contents, too: assertEquals("Hello, Joe!\n", systemOut().getHistory()); }
main()
When It Reads From System.in
Consider this simple piece of code.
import java.util.Scanner; public class HelloWorld { public static void main(String[] args) { Scanner in = new Scanner(System.in); System.out.print("Enter your name: "); String name = in.next(); System.out.println("Hello, " + name + "!"); } }
System.in
You can provide your own contents for System.in
within your test case if you are using
student.TestCase
.
public void testMain() { setSystemIn("Joe\n"); // Don't forget the newline! HelloWorld.main(null); assertEquals("Enter your name: Hello, Joe!\n", systemOut().getHistory()); }
You can easily provide multiple lines as well, and
setSystemIn()
will build the input for you,
adding a line terminator at the end of each line.
There's also a multiline()
helper method
you can use in any other situations where you need to write
multi-line strings.
public void testMain() { // If you provide multiple strings, a line terminator // is automatically added after each one: setSystemIn( "This is the first line of text.", "And this will appear on line two.", "", "There is a blank line before this one.", "You can provide as many as you like." ); ... assertEquals(multiline( "expected output line 1", "expected output line 2", "expected output line3" ), systemOut().getHistory()); }
Or you can read the input from a text file using
fromFile()
. Provide a file name, or a File
object, as the argument--be sure not to use absolute path names,
though, since those are user-specific.
public void testMain() { setSystemIn(fromFile("test-input/test3.txt")); HelloWorld.main(null); assertEquals(fromFile("expected-output/test3.txt"), systemOut().getHistory()); }
Notice that the assert method call in the example shown above includes the full text, including the prompt message. In fact, the prompt message is followed by a space, because that space is part of the prompt itself, but it isn't followed by a newline (the newline is part of what is typed on stdin, not part of what is "printed" by the program). This can be a bit confusing if you have many back-and-forth prompts and responses in a test case action.
Sometimes, it is better to use contains()
or
endsWith()
, rather than testing the entire
output using assertEquals()
.
public void testMain() { setSystemIn("Joe\n"); HelloWorld.main(null); asserttrue(systemOut().getHistory().contains("Hello, Joe!")); }
If you are using Web-CAT in your course and want to submit your work from within Eclipse, there is a plug-in you can use. Even if you've taken CS 2114 and already have Eclipse set up for Web-CAT submission, you must reinstall the plugin.
From within Eclipse, open the Help menu and select Install New Software....
In the Work with: field, paste the following URL and press Enter:
http://web-cat.org/eclstats
Click the check box next to "Web-CAT Electronic Assignments" in the list, and click Next. Review your selection and click Next again.
Check "I accept the terms of the license agreements" and then click Finish.
Wait while the plug-ins are downloaded and installed; this may take a few minutes. You may get a warning about installing unsigned code during the installation. Click OK to continue the installation.
When the installation is complete, Eclipse will notify you that it should be restarted in order for the changes to take effect. Click Yes to do so.
Before you can begin working on class assignments, you need to configure a couple settings in Eclipse's preferences. Once Eclipse has restarted, select "Window > Preferences..." (or on Mac OS X, "Eclipse > Preferences...") from the menu.
Click "Web-CAT Assignments" from the left panel and copy and paste the following URL into the panel. (If the line gets wrapped, make sure you don't accidentally put any spaces in it.)
https://web-cat.cs.vt.edu/Web-CAT/WebObjects/Web-CAT.woa/wa/assignments/eclipse?institution=VT
http://web-cat.cs.vt.edu/Web-CAT/assignments.xml
Click OK to save your changes and exit the preferences window.
Eclipse stores settings in your Eclipse workspace, and since you just created a new one, please import the following preferences into your workspace to make your work process smoother. Note that you will have to re-import these preferences if you create new workspaces later.
Download the following file to your computer: preferences.epf
From Eclipse's File menu, choose Import....
In the import dialog, choose General->Preferences and then click "Next".
Now in the "Import Preferences" wizard, click the "Browse" button to the right of the top field labeled "From preference file:" and navigate to the preferences.epf file you just downloaded. Select the downloaded file. Then click "Finish" to import its contents.
There are a few other preferences settings in Eclipse that students find helpful (but unfortunately, they cannot be easily imported). If you like, you can set those now:
In Eclipse, select "Window > Preferences..." (or on Mac OS X, "Eclipse > Preferences...") from the menu.
In the lefthand tree of the preferences dialog, navigate to Java->Code Style->Code Templates. Check the box at the bottom labeled "Automatically add comments for new methods and types", and then click Apply.
If you want Eclipse to remember the last program (or test) you ran and launch it when you press the "run" button, instead of trying to execute the currently selected/visible class, you can select this behavior. In the lefthand tree of the preferences dialog, navigate to Run/Debug->Launching. Under "Launch Operation" at the bottom, select the radio button labeled "Always launch the previously launched application", and then click Apply.
JUnit.org has more information on how to use JUnit.
Documentation for the asserts in JUnit: http://www.junit.org/apidocs/org/junit/Assert.html
Testing GUI Programs is a Prezi presentation that gives a good introduction to LIFT and how it is used.
The LIFT website provides more details on LIFT, including links to two sigcse papers, downloads, examples, and discussion forums.