MATLAB: How to hide the Cody Coursework Test Suite from Students

cody courseworkMATLAB

With the introduction of Cody Coursework, I'm often asked how to hide the Test Suite in Cody Coursework.

Best Answer

First, we need to understand why MathWorks has made the Test Suite visible to students. Students are encouraged to use MATLAB on their computer to solve problems. To make this possible, students must be able to copy and paste the Test Suite code into a MATLAB script which, in turn, calls their solution function. Once satisfied, they then copy their solution function into Cody Coursework for assessment. For this reason, if your real concern regards student attempts to cheat the system, please first try the information offered here before using the approach discussed below.

If you're still determined to hide the test suite, I'd recommend hiding some and leaving some public.

First, compile a MATLAB function containing test suite code you'd like hidden as Protected MATLAB code.

Second, post this .p file somewhere online accessible by Cody Coursework. This must not be behind the school's firewall, it must be publicly accessible. One option is to use the public folder of a free dropbox account. If using DropBox, it will help to "copy the public link" using their web interface.

Third, implement a test case of your test suite to call your protected test suite function, for example:

   %%Execute Hidden Tests
   urlwrite('http://url/ProfsFunction.p','./ProfsFunction.p');
   rehash
   assert(ProfsFunction());

Fourth, add a test case to prevent student attempts to call on the .p file themselves, for example by parsing the students' solution file directly,

   %%Test for attempts to cheat
   found = 0;
   fid = fopen('StudentsFunction.m');
   while(~feof(fid))
      s = fgetl(fid);
      k = regexp(s,'urlwrite\(|ftp\(|mget\(|web\(|fprintf\(|fwrite\(|sprintf\(');
      if ~isempty(k)
          found = 1;
      end
   end
   fclose(fid);
   assert(isequal(found,0))

Fifth, use the solution map as a "line of last defense." Typically, honest solutions which are correct follow similar patterns. Horizontal bars of these patterns form on the solution map. The few stray solutions below these bars which Cody Coursework has marked correct merit manual code inspection.

There are many possibilities the power of the MATLAB language offers for implementing test suites. We will add more information and examples in future, and I encourage you to share your own insights as answers or comments below.

Related Solutions

MATLAB: How to write an effective Test Suite in Cody Coursework

With this in mind, consider how a student might "game the system" presented with the following Test Suite from the Times2 problem on Cody:

   assert(isequal(times2(1),2));
   assert(isequal(times2(11),22));
   assert(isequal(times2(-3),-6));
   assert(isequal(times2(29),58));

Here, the Test Suite simply calls the students' function, times2, using explicit data values to confirm the students' code returns the expected result. This approach is morally hazardous, as a student can simply write a case or series of if statements to compromise the approach.

Each Test Suite is a MATLAB script. Here we draw on this power to more effectively assess students' code in response to the Times2 problem,

   X=rand(1);
   Y_answer = x*2;
   assert(times2(x),Y_answer)

This still poses two issues, 1. it reveals the answer the students' code needs to implement (i.e. return x * 2), and 2. it does not apply well to complex programming problems.

Now that we've examined what not to do, let's discuss a more effective approach for assessing students' MATLAB code. Faculty already have a reference solution to each programming problem assigned, traditionally offered to teaching assistant staff for manual grading. Protected MATLAB code offers the key of how we use these reference solutions as part of a generalized recipe to write an effective Test Suite.

First, compile your reference solution as Protected MATLAB code.

Third, implement a test case of your test suite to use your protected reference solution code, for example:

   %%Test Functional Correctness of Students’ Code
   urlwrite('http://url/ProfsFunction.p','./ProfsFunction.p');
   addpath('.');
 
   for i=1:10
     data_vector=rand(1000,1);
     prof_answer = ProfsFunction(data_vector);
     student_answer = StudentsFunction(data_vector);
     assert(isequal(prof_answer,student_answer), 'student code gave an incorrect answer');  
   end

Adapting this for more complex problems will require you adapt the data_vector randomization to something appropriate for your subject matter. Alternatively, you might wish to incorporate a data file from which your Test Suite randomly selects input/output values.

Fourth, add a test case to prevent student attempts to call on the .p file themselves, for example by parsing the students' solution file directly,

   %%Test for attempts to cheat
   found = 0;
   fid = fopen('StudentsFunction.m');
   while(~feof(fid))
      s = fgetl(fid);
      k = regexp(s,'urlwrite\(|ftp\(|mget\(|web\(|fprintf\(|fwrite\(|sprintf\(');
      if ~isempty(k)
          found = 1;
      end
   end
   fclose(fid);
   assert(isequal(found,0))

MATLAB: How to create problems in Cody Coursework which require students to process data file(s)

Here's how:

First, prepare the data file(s) you wish students' code to access.

Second, post the data file(s) somewhere online accessible by Cody Coursework. This must not be behind the school's firewall, it must be publicly accessible. One option is to use the public folder of a free dropbox account. If using DropBox, it will help to "copy the public link" using their web interface.

Third, design your Test Suite to load the data file into the students' current working folder before calling the students' code to test. Here's an example:

   %%Prepare data files for student
   in_f='http://url/datafile.csv';
   out_f='./datafile.csv';
   urlwrite(in_f,out_f);
   % check to ensure the file loaded ok
   fid=fopen(out_f);
   assert(isequal(CheckFile(),1));

other tests follow

Alternatively, for more advanced students, you could replace the third step by offering students a function template with code to bring the data into the local folder in which their code will be run by Cody Coursework. For example:

   function result = student_code(input)
     % hint for students, keep this template code
     urlwrite('http://url/datafile.csv','./datafile.csv');
     addpath('.');
     % students' TODO: insert your code here:
     % students, please do not modify code outside these comments
   end

Either way, students can now use standard MATLAB file operations to open and manipulate the data file(s) loaded into their current working folder.

Please note, to ensure system reliability, Cody Coursework resets system state each time a solution attempt is tested. This is why the file is loaded on each execution of the Test Suite.

Best Answer

Related Solutions

MATLAB: How to write an effective Test Suite in Cody Coursework

MATLAB: How to create problems in Cody Coursework which require students to process data file(s)

Related Question