Grade of 0.0 for any assignment

Hello,

 

I'm a research assistant at my university, and in our project we want to use an automated grading tool to help tutors grade students' programming assignments.

 

I found Web-CAT to be a very professional and interesting such tool, and I got it installed on my Tomcat server.

 

However, I would have some questions:

 

  1. When I create an assignment and then want to submit a zip file (I took the example assignment submission packages from http://wiki.web-cat.org/WCWiki/TryOutWebCat/UsageInstructions -- the Java calculator examples.), I always get a grade of 0.0 / 100.0.

 

I ensured all my plugins were installed; could I know what might be causing this (in fact, it seems as though the code analysis does not work)?

 

  1. My second question is regarding the MySQL database Web-CAT uses. Can Web-CAT use an existing MySQL database such as that Moodle uses? For example, if we want to integrate this tool in our university, we would like to just use the existing Moodle MySQL database with all courses, students, tutors, etc. Is this possible?

 

And does Web-CAT support other databases such as PostgreSQL or Derby?

 

Thanks.

Groups:

Comments

MWitsiers

Same problems

I encounter the same problems as JS:

Perl is installed on my Wondows 7 computer and the path is correct. A submission is graded 0.0. The JavaTDDPlugin does not seam to work. After submission the subdirectory Results only has the file grading.properties, and the subdirectory _GraderCheckout has no content.

I need some hints to get submissions working.

Stephen Edwards

I'll try to answer your questions

> I always get a grade of 0.0 / 100.0.

This could be caused by a number of factors.  I am assuming that you set up your own server, and this is happening there.  The first thing to do is to go into the file system on your server where Web-CAT is saving uploaded submissions and results.  This would be the "grader storage" location you specified during installation.  Within that directory, if you look into its subdirectories, you'll find one named after the login institution you set up, with a subdirectory inside it named after the semester you created, with a subdirectory inside it named after the course offering ID you used, then subdirectories for each assignment you created, and inside those, subdirectories for each user name that has submitted to the assignment.

Go all the way inside there and find the numbered subdirectory that corresponds to the submission you received a 0.0 for (probaby the highest-numbered subdirectory there).  Inside there, you'll see the submission zip file and a subdirectory called "Results" that contains all the plug-in generated output.  Look in the results file for any files with names starting with stderr or stdout, which may give a clue.  Otherwise, look for a file called ant.log, which has the transcript of the ANT run that performed all the compilation/testing/static analysis.  If you're unsure what the contents of any of these files mean, post the relevant section of the file in a follow-up comment and I can help you diagnose it.

JS

Thank you for your reply, I

Thank you for your reply,

I checked at the folders you mentioned, but there is only a grading.properties file in there. But perhaps my problem is that I did not set the assignments properly. So I will check into this and explore the product more in depth.

By the way, am I right to believe that the plugins for grading are automatically installed when setting up the application?

Also, I'd like to know what server is best to deploy the Web-CAT application. I'm currently using Apache Tomcat, but I was considering using JBoss, or IBM's WebSphere. What server do you suggest in terms of speed and functionality in relation to the Web-CAT app?

Thanks again,

Jeremie

Stephen Edwards

If the folder I mentioned

If the folder I mentioned only contains a grading.properties file, that sounds a bit suspicious.  It sounds like no plug-ins may be configured for grading the assignment.  Can you use the Assignments menu to pull up the properties of your assignment, and then confirm which plug-ins (if any) are listed in the table under "Grading Scheme for All Offerings"?  If that table contains only one row, with a dropdown list containing "select a plug-in to add ...", and nothing else, then no plug-ins have been chosen to process the assignment, and the score will be 0.0 until a person manually changes it (i.e., no automated actions are taken if no plug-ins are selected--the plug-ins control what, if any, automated processing is applied).

Is that the case?

Zkit Wong

Thanks a lot of your reply I

Thanks a lot of your reply

I have already chosen the grading plug-in of "JavaTddPlugin" but still faceing the 0.0 grading problem

Would you kindly advise?

Many thanks.

Kit

 

Stephen Edwards

The situation you describe is

The situation you describe is quite puzzling.  If the plug-in actually runs, then there should be more than just the grading.properties file in the Results directory for the submission (quite a bit more).  If the plug-in runs successfully, various output files describing the submission, as well as an ant.log text file showing the steps used to compile/run the code and tests, will all be present.  If errors occur, then stdout and stderr log files from the failed plug-in would also be present.

The lack of any other files in the Results directory suggests that the plug-in is not actually being executed.  But normally, if the plug-in fails when the server tries to execute it, the submission will be "suspended" (because it couldn't be processed correctly), which will not show a score.  This should also leave clear messages in the log file for Tomcat (or whatever servlet container you are using).  Have you checked the Tomcat log file?  Also, have you checked the file permissions in the storage area to make sure the storage area is readable/writable by the user under which Web-CAT is running?

If the Tomcat log file looks completely clean, I would suggest hand-editing the configuration.properties file for your instance (found approximately at: webapps/Web-CAT/WEB-INF/Web-CAT.woa/configuration.properties) with a plain text editor, and add this line:

log4j.rootLogger=DEBUG

Then stop Tomcat, delete its log (so you get one completely fresh), and restart Tomcat.  Make another Web-CAT submission, see what happens, and then send me the Tomcat log file.  It should contain enough detail to figure out where things are breaking down.

Also, just out of curiosity, have you tried a clean re-install?  That is, completely delete the unpacked war, the database, and the storage area, and try reinstalling from scratch?  I'm assuming you have.

Zkit Wong

Thanks for you reply

Thanks a lot of your reply.

I know that the situation of us may be hard to investigate, thanks again for your help.

 

First, i try to reinstall the web-cat but still fail with the problem.

For the stoage folder's permission, i try granted the permission of the folder can have the "full control" with "Everyone", but still fail with the problem.

Finally, i tried your solution and get the tomcat log as follow link.

https://www.dropbox.com/s/1upkksdg1smf8y0/tomcat7-stdout.2013-08-23.log

Is that the log you need?

Please kindly advise.

Many thanks.

Kit

Stephen Edwards

Thanks for the log.  I

Thanks for the log.  I grabbed it yesterday and spent some time combing through it (wow, way more info than I had planned!).

Indeed, it does look like there is some issue preventing the server from actually executing the plug-in.  Let me ask a few more questions about your configuration:

What OS/version are you using (appears to be Windows, but I haven't tested Web-CAT on Windows 8 yet)?

Also, what perl distribution and install location are you using?

Overall, though, it appears more like Web-CAT never completed the process of actually attempting to execute the plug-in.  The log info "jumps" from where it is unpacking the reference tests and any other files needed to prepare for executing the plug-in, and then straight to pulling together the resulting feedback from the plug-in.  The (unconditional) debug output indicating it is actually trying to fire up the plug-in is just completely absent from the log, which I can't easily explain.  None of the log messages related to executing the plug-in are present in the log at all.  However, there are also no exeception messages or anything indicating any errors occurred anywhere near that part of the process (the log has tons of exception traces, nearly all of which are simply part of debugging output so that relevant stack traces for certain events are forced into the debug log, when log settings are turned up this high).

So, some more questions:

Did you upload your own reference test(s) for this assignment, or were you trying grading without any reference tests for your very first assignment (perfectly OK to do, but I want to make sure)?

Have you *ever* tried uploading any files to your Web-CAT instance for use in assignment grading (reference tests or anything else)?

Also, can you check the "work area" directory you configured (not the storage location, but the temporary working area for compiling/executing students' code) to see what it contains?  Check it, and see if it has a _GraderCheckout subdirectory, and if so, what does that folder contain?

Zkit Wong

Thanks a lot of your advise

Thanks a lot of your advise and effort and sorry for my late reply.

There are the information you need:

1 .I am using the window7 32 bit for my operating system.

 

2. For this assignment, (actually for this log), i have uploaded my reference test.

Also, i have tried again without upload my reference test which using default after sent you the log.The result is also fail.

Besides, i am trying grading without any reference tests for my very first assignment. Surely, i have tried upload my reference test again.

 

3. In the working area, there is a _GraderCheckout subdirectory, however, the folder is empty.

Seems it is a bad news?

 

4. For the perl distribution, i am quite not understand about it. If it is the key item, would you kindly advise how to check it? 

 

Many thanks.

Kit

Stephen Edwards

Perl is required

OK, it seems like answer (4) is the likely problem.  Web-CAT requires Perl to be installed on the server machine.  On nearly all linux or OS X servers, it would be present.  However, it does not come standard as part of Windows and must be installed separately.  Most people who run web servers on Windows machines have it installed (and it typically comes as an integral part of a typical WAMP-like stack).  However, it sounds like you may not have it installed at all.

To check, open a command line window (under Windows, this is usually labeled as "Command Prompt" and appears in the "Accessories" folder on the start menu, or use the start menu's search feature).  Execute the command "perl -v" on the command line, which will print out the perl version for the currently installed version of perl (assuming it is installed somewhere on your PATH).  If the response is "'perl' is not recognized as an internal or external command ...", then it probably isn't installed.

Assuming you don't have it installed, you definitely need it.  Perl is open-source and free.  I'd suggest ActivePerl, which you can get from here: http://www.activestate.com/activeperl (choose the free community edition). Install it, be sure to add it to the PATH in your system environment, and then try again.

Not having Perl installed will definitely prevent Java grading entirely. Nearly all of the available plug-ins are written in Perl (often with ANT for build management).

Zkit Wong

Thanks a lot for your

Thanks a lot for your advice.

Seems the problem is near solved, thanks a lot.

 

Actually, i installed the ActivePerl which you suggested now.

After the installation, i tried to restart tomcat and create a new assignment but the result is also fail.

Would you kindly advise? Is there any setting of ActivePerl or any other suspecting problem?

 

Best regards,

Kit Wong

Stephen Edwards

> the result is also fail Can

> the result is also fail

Can you be more specific about the symptoms.  Are they the same?  That is, is the Results directory created for the new submission still empty except for the grading.properties file?  Or is there other content in that directory?  Also, did the assignment get "suspended" (that is, Web-CAT detected an error while trying to run the plug-in), or does it simply show a 0.0 score with no other feedback?

BTW, when you installed ActiveState Perl, did you add it to your PATH?  That is, if you open a new command line window, does "perl -v" show you the perl version?  Also, if you change the system path, you typically have to restart the machine in order for services to pick up the PATH change (i.e., for the tomcat process to pick up the new PATH setting).

Zkit Wong

Sorry for the lack of the

Sorry for the lack of the information.

1)The result directory is also included the grading.properties file only.

2)In the working direstory, there is a _GraderCheckout exist but it is also empty which is same as before.

3)There is no special message occur and just showing the grad 0.0 which same as before. Also, i check the system status in the home page, there is no Queued jobs,Halted assignments or Stalled jobs exist.

4) There is a new log which may help you to investigate if you need.

https://www.dropbox.com/s/78aigswhqd1l1fl/tomcat7-stdout.2013-08-30.log

Actually, the step is same as before, Stop the tomcat > clean the log> start tomcat > create assignment >choose the reference test which i uploaded before > save it > submit the assignment > get the 0.0 result

Besides, actually, in both last time and this time, there are othe logs such as tomcat7-stderr.2013-08-30, catalina.2013-08-30, localhost_access_log.2013-08-30, i can send you if you need it.

 

Sorry and thanks again to help me and pay a large effort to investigate my problem.

Many thanks

 

Kit

Stephen Edwards

OK, so the symptoms are still

OK, so the symptoms are still the same as before. The log indicates the same behavior as before.

When you installed ActiveState Perl, did you add it to your PATH? That is, if you open a new command line window, does "perl -v" show you the perl version? Also, if you change the system path, you typically have to restart the machine in order for services to pick up the PATH change (i.e., for the tomcat process to pick up the new PATH setting).

Zkit Wong

Sorry for late reply.After

Sorry for late reply.

After the installation of ActivePerl, i have tried sucessfully with perl -v in cmd.

(The message is like :

This is perl 5, version 16, subversion 3 (v5.16.3) builthread

(with 1 registered patch, see perl -V for more detail))     Also, after the installtion, i have restart the computer to confirm that the installation have been complete.  However, the symptoms is same as before as i describe.Any other advice? Many thanksKit

Zkit Wong

Besides the perl problem, i

Besides the perl problem, i would like to reinstall once again.

However, i am not sure whether the step of reinstall last time is correct or not.

For the step of last time, i go to tomcat to choose undeploy Web-CAT.

Then i delete the work and storage folder manually.

Finally, i choose deploy the Web-CAT.war again.

Is that correct?

Would you kindly advise the reinstall step?

Zkit Wong

Futher question

Hello,

May i know the problem you mentioned is fixed or not?

It is because i also have the same problem of getting no gradeing (which means getting grade 0.0) 

and i also get the grading.properties only but not have the ant.log.

Kit

JS

Hello, No, unfortunately, the

Hello,

No, unfortunately, the problem I was having with grades of 0.0 is not yet solved. I just reinstalled Web-CAT on a new machine and so will eventually check if the problem still persists.

If you look into "web-cat storage directory/UserScripts/[institution name]/[admin name]/JavaTddPlugin", there are many .xml files plus the grading.properties. Perhaps these files contain specific configurations to get the grader to work.

I will let you know if and when I solve this issue.

Regards,

Jeremie

Hao Loi

Grade 0.0

Hi, I just install a new server and this problem is happening to me.  I always get zero for my student assignments as soon as I check "students must submit tests" under "Reusable options".  I am using Unbuntu 14.05 and JDK 1.8.  Did you find out how to resolve the problems?  I would greatly appreciated for any help.