Adding Software Testing to Programming Assignments
SIGCSE 2006 Workshop Companion Web Site
This web site contains the companion materials for the SIGCSE 2006 Workshop: Adding Software Testing to Programming Assignments, presented by Stephen Edwards from Virginia Tech's Department of Computer Science.
This web site contains electronic copies of all the workshop materials, together with links to installable versions of all required software.
Workshop Abstract
Software testing is a critical skill for practitioners, but it is rarely given appropriate coverage in undergraduate curricula. If we want to change the culture of how students program, they should practice basic testing skills on every assignment all the time. This workshop provides a practical introduction to incorporating software testing activities as a regular part of programming assignments. While this can be done for many languages, the focus here is on leveraging XUnit-style tools and test-first coding techniques to infuse software testing throughout courses that teach object-oriented programming concepts. Empirical evidence suggests that, when assignments are transformed this way, students produce higher-quality results (averaging 28% fewer bugs/KSLOC) and are more likely to turn assignments in on time. Further, test-first strategies preempt "big bang" integration disasters, and students report greater confidence in their own work.
This workshop presents five different models for how one can incorporate testing into assignments, provides live programming demonstrations of the techniques, and discusses the corresponding advantages and disadvantages of each. Approaches to assessment--using testing to assess student code, assessing tests that students write, and automated grading--are all discussed. Advice for writing "testable" assignments is given. Hands-on examples for participants are also provided.
Keywords: computer science education, test-driven development, test-first coding, JUnit, unit testing, automated grading
Level: Beginner
Workshop Contents
Example 1: Building a Gradebook (in Java) or (in C++)
Example 2: Appointments (in Java) or (in C++)
Example 3: Line Counter (in Java)
Example 4: Debugging Bricks (in Java)
Resources
In addition to the materials on this web site, you may find the following on-line resources useful:
This workshop's Test-Driven Development page provides some basic information on TDD together with links to suggested readings available on-line.
The Web-CAT Wiki provides basic information about the Web-CAT automated grading system, including instructions for how to request access for your own courses (via remote access from our server). Anyone can contribute to the wiki just by self-registering.
The Web-CAT Wiki also contains a Test-Driven Development page that provides some basic information on TDD together with links to suggested readings available on-line.
The Web-CAT Wiki also contains a Reading List on automated grading and on integrating software testing into the CS curriculum, including links to our publications on the Web-CAT grader and research results. You will also find a paper describing our Eclipse-related plug-ins for supporting student testing activities.
The Web-CAT Wiki serves as the home for our SourceForge project, which supports community development of both Web-CAT and all our Eclipse plug-ins, including our CxxTest plug-in.
The CS Education Wiki contains more general information on CS Education--feel free to browse its contents. It contains the details of our redesigned CS1 course where we use test-driven development from the very first lab assignment. All of our CS1 course materials are available here, including a full set of our laboratory assignments and programming assignments (unlike the remainder of the wiki, these resources are restricted to educators only and require registration to access). Anyone can contribute to the wiki just by self-registering.
The CS Education Wiki also contains a larger Reading List on CS education in general (teaching objects-first, laboratory practices, and more).
The CS Education Wiki contains the materials presented at the 2004 JETT Workshop hosted by Virginia Tech. This was a Java training event for high school AP CS educators offered as part of the ACM and College Board Java Engagement for Teacher Training project.
Tools You May Need
While you can view the source code on this web-site directly, if you wish to use it in an IDE, be sure to use an environment that supports an appropriate unit testing framework.
For Java examples:
Sun Java Development Kit Standard Edition (home page):
You need to install an appropriate Java development environment to compile the Java examples here. We recommend the latest stable release of JDK v.1.5.x from Sun, although an older v1.4.x version will also suffice for the examples here.
BlueJ (home page):
If you wish to try out a beginner-level environment targeted at CS1 students, you can try out BlueJ. All of our examples work in BlueJ, and this environment has excellent built-in support for JUnit. We use BlueJ in our CS1 course, and have students writing test cases during their very first lab.
Eclipse (home page):
Eclipse is a full-featured IDE targeted at professional developers, particularly in the Java community. It also provides built-in support for JUnit. We recommend our own Virginia Tech Student Edition of Eclipse, which includes a number of enhancements targeted at students. The Java examples provided here will work on nearly any Eclipse distribution, however.
JUnit v3.8.1 (home page): JUnit comes prebundled with both Eclipse and BlueJ. However, if you are not using one of these environments, JUnit is available as a separate jar file you can add to your classpath.
For C++ examples:
Eclipse, Virginia Tech Student Edition (home page):
Although an IDE is not strictly required, if you wish to play around with our C++ examples, we recommend using our preconfigured Eclipse bundle for students. The Windows version we distribute comes as a self-installing executable that contains everything needed for C++ projects. Versions of Mac OS/X and Linux are also available. If you already have an existing Eclipse installation, we provide an update site from which you can install our Eclipse features directly.
CxxTest (home page):
Our C++ support is based on CxxTest, a unit testing framework similar to JUnit, but for C++ code. You can visit the CxxTest home page to read the CxxTest user's guide. The CxxTest support we provide in Eclipse relies on a number of extensions that are not part of the base CxxTest distribution, however (signal handling, heap error tracking, etc.).
g++ (home page):
CxxTest works well with g++, which is the compiler we use in many of our classes. It is available for most platforms. A Windows-compatible version of g++ based on cygwin is included with our Windows distribution of Eclipse for student use. Other versions are also available. If you prefer to use Microsoft Visual C++, instructions for using CxxTest with Visual C++ are available from the CxxTest home page.
Sponsorship
We gratefully acknowledge the support provided to this work by our sponsors. This work is sponsored in part by the National Science Foundation under Grant No. DUE-0127225, by a Virginia Tech Institute for Distance and Distributed Learning research fellowship, and by an Eclipse Innovation Grant from IBM. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation, IBM, or IDDL.