check50, or any autotester/autograder, is a very simple framework with a complex implementation. The complexity would be a function of the complexity of the program being tested. The basic ingredients are: a scripting language, a set of inputs, expected outputs. You can get a very good idea of what check50 does by viewing the details in the sandbox output. For instance, the pseudo-code for this sample pset1/hello sandbox output (detail) is something like:
Does hello.c exist? Yes == pass, No == fail.
Does hello.c compile? Yes == pass, No ==fail.
Does hello.c give correct output? Yes == pass, No == fail.
Now all you have to do is write that program in your favorite scripting language :)
Crafting tests is an art unto itself. You need to build tests to determine:
- Does the program follow the spec exactly?
- Does the program handle edge cases, corner cases? (interesting discussion found herehere)
- Does the program give correct output?
Take "programming autograding" for a spin in your favorite search engine for a plethora of insightful articles on this topic.
