Plug-in Architecture (Pluggable "Subsystems")

The internal infrastructure of the web application is designed using a plug-in architecture, where there are a base set of central services that can be extended by plug-ins that provide various student-accessible services.
There is a well-defined API for plug-ins (called "subsystems").
New subsystems are integrated just by adding them to the application's classpath; no changes to Web-CAT's source code are needed to use new subsystems (no recompilation).
Each subsystem can add arbitrary extensions to the core data model maintained by the application (basically, adding its own entities and relationships to support its own services).
There are currently four subsystems:
- The Core Subsystem provides common support for all other subsystems
- The Grader Subsystem provides Web-CAT's assignment submission and grading features
- The Admin Subsystem provides raw database access for administrators
- The Example Subsystem is a simple coding example that serves as documentation for prospective subsystem authors
Enabled features and application settings are configured and controlled using a properties file loaded at system startup--a plain text file sort of like a .ini file.

The Core Subsystem

Provides the basic structure for all pages, including Web-CAT's common look and feel.
Provides support for a two-level tab-based navigation system that allows users to quickly navigate around the web site. In addition, navigation support also provides a third level of navigation links useful for walking users through a series of steps to complete a task.
Provides a core data model available to all other subsystems, including:
- Entities like users (including students, instructors, and TAs), courses, course sections, etc.
- Support for many-to-many relationships recording which users are instructors for which course sections, which users are enrolled in each section, which users are TAs for which sections, etc.
- Support for multiple sections of any course, multiple instructors for any section, multiple TAs for any section, etc.
- A student may be a TA for one course but a regular student enrolled in some other course (instructors can too!)
Authorization control:
- The actions available to a user are determined by the user's role with respect to a given course (e.g., TAs have different actions than regular students or than instructors).
- Actions may be further restricted based on an individual's access level (global permissions granted by the admin).
Support for multiple institutions:
- Web-CAT can hande multiple sets of users from different organizations, universities, or domains on the same server.
- User names need not be globally unique, just unique within a particular organization or university.
Flexible, extensible authentication:
- The authentication strategy can be different for each institution, and different authentication strategies can be used for different groups of users at one site.
- There is a well-defined API for creating new authentication mechanisms.
- Existing mechanisms include:
  - Simple local passwords stored in Web-CAT's database.
  - External authentication against a POP mail server.
  - External authentication against Virginia Tech's campus-wide authentication service.
  - External authentication against an LDAP server is an easy add-on (we just haven't needed it yet).
- New authentication mechanisms can be added just by placing them on the classpath; no code changes are needed to Web-CAT itself (no recompilation).

The Grader Subsystem

Support configurable management of electronically collected and graded programming assignments.
Instructor-configurable submission settings for an assignment include:
- Due date
- Optional opening date (when to start accepting submissions)
- Optional limit on size of submissions
- Optional limit on number of submissions allowed
- Optional late policy: how long late assignments are accepted and how penalties accrue (e.g., -X points per day for up to Y days or similar)
- Optional early bonuses: extra credit for early completion
Assignment details can be shared across multiple sections of a course.
Submission guidelines can be reused across assignments, course sections, and different courses.
The instructor can provide custom instructions for students on the upload page (including free-form HTML content).
Students upload their submissions via a web browser:
- Multi-file submissions are accepted in the form of a zip or jar archive, and are automatically expanded as needed.
- All submissions and their results are archived independently.
- Support for beginners to make submissions directly from the BlueJ IDE using BlueJ's submitter extension.
- A plug-in is available to support submissions directly from the Eclipse IDE (see EclipsePlugins).
Flexible, extensible actions for grading assignments:
- The actions used to compile, execute, and evaluate student code can be tailored by the instructor, who can upload one or more "scripts" to be used.
- Scripts can be in any language executable on the server--typically Perl, but stand-alone programs, shell scripts, batch files, Java programs, and more can be used.
- An assignment can be graded using any arbitrary sequence of instructor-provided scripts. Each step can be given a separate execution time limit.
- A simple property file is used to communicate information to and from grading scripts, and allows multiple scripts arranged in a pipeline to communicate with each other.
- All student-visible feedback is customizable by the instructor-provided scripts, which can produce results in about any form viewable or downloadable on the web for the student to access.
Flexible, extensible programming language support:
- Automated grading is possible for any programming language that can be installed on the server.
- All the instructor needs to do is ensure the language is installed on the server, and then provide the necessary grading and feedback script(sor use existing ones where possible.
- Scripts have already been written for Java (1.4 and 1.5), Scheme (both MIT Scheme and DrScheme), Prolog (SWI), Pascal (FreePascal), Standard ML (SML/NJ), and c++ (g++ 3.4.x).
- In many cases, new scripts can be adapted from existing ones.
Direct WYSIWIG TA grading of student source code through their web browser:
- TAs or instructors can add their own comments and point deductions to any source file line, enter overall comments on the entire assignment, and view or modify total deductions (even adding positive or negative corrections to the automatically graded portion of a student's score).
- Comments can be directly added through a syntax-highlighted view of the source code, right at the spot where the symptom is most obvious.
- Students get automatic e-mail notification when their assignment grading has been completed.
- This feaure operates for all program source files and text files in supported languages (no longer Java only!). Even "readme" files in a student's project can receive comments.

The Java Grading Script

Our most extensive and powerful grading script is currently set up for processing Java programs. It includes the following features:
- ANT-based automated build of any Java project, including full support for arbitrary package structures.
- Inclusion of instructor-provided library JARs to be used while compiling and executing student code.
- Automatic identification and execution of student-written JUnit tests (using ANT's junit task).
- Support for Clover instrumentation of student code to collect code coverage data (methods, statements, and branches) while student tests are running.
- Support for static analysis of student code using Checkstyle to indentify documentation, formatting, naming, and stylistic errors.
- Support for static analysis of student code using PMD to indentify additional stylistic and coding errors.
- Support for optional instructor-provided reference tests that will be automatically executed against the student code.
- Support for generation of PDF program listings, if desired.
- An integrated web-viewable feedback report is provided to the student, including:
  - A color-coded, syntax-highlighted view of each source file.
  - An execution count for each line, with under-tested lines highlighted in pink (provided by Clover).
  - Individual errors from Checkstyle, PMD, or entered by the TA are presented in-line in the source code view, highlighting the line to which each applies.
- Flexible configuration options for the instructor, including:
  - Control over which Checkstyle rules are enabled and how each is configured.
  - Control over which PMD rules are enabled and how each is configured.
  - The ability to write new PMD checks using xpath expressions (or provide your own Java implementations of PMD or Checkstyle checks).
  - Control over the point deductions used for each static analysis comment.
  - The ability to group many static checks from all tools into arbitrary categories (like "commenting" vs. "layout" vs. "coding style", for example), and set maximum allowable deductions for each, or for all static checks taken together.
  - Control over how to use coverage data to compute the testing portion of a student's score.

The C++ Grading Script

Another popular grading script exists for processing C++ programs. It includes the following features:
- ANT-based automated build of any C++ project (builds using g++), including full support for arbitrary subdirectory structures.
- Automatic identification and execution of student-written CxxTest tests.
- Support for optional instructor-provided reference tests that will be automatically executed against the student code.
- Support for instructor-provided header files and linker libraries.
- An integrated web-viewable feedback report is provided to the student, including a color-coded, syntax-highlighted view of each source file.
- Support for heap-tracking to identify memory leaks and heap corruption within student code.
- Support for trapping assert failures, segmentation faults, and other signals during test execution (signals are mapped to test case failures).
- Full code coverage reporting for student-written code, including modified condition/decision coverage data, with feedback integrated into the color-coded web printout of each source file.
- We are currently working on adding static analysis support for C++, but these features require the integration of additional language-specific tools.

The Admin Subsystem

Provides interactive searching, viewing, and updating of all database entities for all subsystems (for authorized users).

The Example Subsystem

A simple subsystem that serves as an example for developers who want to write new subsystems that provide new services.