This is the story of Firms regression testing tool. It started out as a short shell script and evolved into a standalone Python program by now.
Most Simple Regression Testing
In the beginning you start with the most simple thing that works. A bash script and a bunch of c-files.
#!/bin/bash
for f in *.c; do
cparser "$f"
done
This just compiles each c-file. Which is enough to verify that the compiler does not crash.
Growing
Over time the script becomes bigger as useful features are added. The compiled program is executed and its output compared to reference output. Limits are set to terminate infinite loops in the compiler. Parameters are introduced, so you can run it with other compilers.
Each single feature is relatively simple addition and keeping it as a bash script is reasonable. As complexity and power grows each step becomes harder.
Rewrite
At some point some (over?) motivated guy rewrites everything in Python, because some more complicated feature is hard to implement with bash. The feature is to have expectations of each test result. The testsuite is used as a kind of bug tracker, so not all tests pass. This means a test that fails, but is expected to fail is ok. However a succeeding test, that is expected to fail should be reported. He carefully maintains backwards compatability so both programs are interchangeable. The new feature makes the developers adopt the Python version.
More growth is enabled and more features are added. For example, colored output. As the testsuite grows further and multicore computer become common, parallelisation is added.
Standalone
Then libFirm is used for another compiler. Thus, the script is reused for another testsuite. At this point it becomes reasonable to externalise the tool. So, if you want a similar tool, it is now available. However, the tool is still pretty much undocumented, so you need to read Python code. Fortunately, using it is trivial. It is only a challenge to create the testsuite configuration.
Conclusion
This is not intended as an advertisement for sisyphus. This documents one example, how projects grow from simple scripts to full applications.