AUTOMATED EVALUATION OF PROGRAMMING CODE

1 . A system for evaluating code, comprising:

a processor configured to:

determine a configuration associated with a programming problem;

generate a random input data;

provide the programming problem based at least in part on the determined configuration;

receive an output data that corresponds to the random input data, wherein the output data was generated by a code responsive to the programming problem; and

evaluate the output data; and

a memory coupled to the processor and configured to provide the processor with instructions.

2 . The system of claim 1 , wherein the system is configured to automatically evaluate the output data without human intervention.

3 . The system of claim 1 , wherein the programming problem is a web programming problem type associated with rendering a webpage.

4 . The system of claim 1 , wherein the programming problem is a regular expression matching programming problem type associated with locating a desired pattern within text content.

5 . The system of claim 1 , wherein the programming problem is a graphical rendering programming problem type associated with a turtle graphics rendering.

6 . The system of claim 1 , wherein the programming problem is provided as a part of an employment interview process.

7 . The system of claim 1 , wherein the processor is further configured select at random one programming problem type from a group of predetermined programming problem types and the programming problem is based at least in part on the randomly selected programming problem type.

8 . The system of claim 1 , wherein the programming problem is at least in part determined based on information associated with an intended user of the programming problem.

9 . The system of claim 1 , wherein the configuration is associated with one or more of the following: a programming language, a programming constraint, a data relationship, a data dependency, an allowed programming element, an allowed programming library element, and a time limit.

10 . The system of claim 1 , wherein determining the configuration includes dynamically determining at least a portion of the configuration.

11 . The system of claim 1 , wherein determining the configuration includes randomly determining a schema of a database that will contain the random input data.

12 . The system of claim 1 , wherein at least a portion of the random input data is included in the programming problem as an example input data.

13 . The system of claim 1 , wherein the random input data includes test input data that is utilized when evaluating the output data.

14 . The system of claim 1 , wherein the processor is further configured to generate a reference output data corresponding to the random input data.

15 . The system of claim 14 , wherein at least a portion of the reference output data is included in the programming problem as an example output data.

16 . The system of claim 14 , wherein evaluating the received output data includes comparing the received output data with the reference output data.

17 . The system of claim 1 , wherein evaluating the output data includes evaluating the code.

18 . The system of claim 1 , wherein evaluating the output data includes determining a quantitative metric associated with a correctness of the code.

19 . A method for evaluating code, comprising:

determining a configuration associated with a programming problem;

generating a random input data;

providing the programming problem based at least in part on the determined configuration;

receiving an output data that corresponds to the random input data, wherein the output data was generated by a code responsive to the programming problem; and

using a processor to evaluate the output data.

20 . A computer program product for evaluating code, the computer program product being embodied in a tangible computer readable storage medium and comprising computer instructions for:

determining a configuration associated with a programming problem;

generating a random input data;

providing the programming problem based at least in part on the determined configuration;

receiving an output data that corresponds to the random input data, wherein the output is data was generated by a code responsive to the programming problem; and

evaluating the output data.