How does check50 works?

Question

I am curious about how check50 works internally. What does it do to make that kind of tests for each program (each pset?). What if I want to implement something like check50? Any advice?

Answer 1

check50, or any autotester/autograder, is a very simple framework with a complex implementation. The complexity would be a function of the complexity of the program being tested. The basic ingredients are: a scripting language, a set of inputs, expected outputs. You can get a very good idea of what check50 does by viewing the details in the sandbox output. For instance, the pseudo-code for this sample pset1/hello sandbox output (detail) is something like:

Does hello.c exist? Yes == pass, No == fail.
Does hello.c compile? Yes == pass, No ==fail.
Does hello.c give correct output? Yes == pass, No == fail.

Now all you have to do is write that program in your favorite scripting language :)

Crafting tests is an art unto itself. You need to build tests to determine:

• Does the program follow the spec exactly?
• Does the program handle edge cases, corner cases? (interesting discussion found here)
• Does the program give correct output?

Take "programming autograding" for a spin in your favorite search engine for a plethora of insightful articles on this topic.