Description
Hints for Golden Master Testing in Python
What do we do when the only test requirements we have are "the new system should have the exact same results as the old system"? Golden Master Testing may help. The idea behind golden master testing, also called characterization testing, is quite simple: We don't write test cases that specify what the expected result is. Instead, we take the output from a prior, working version of a program (the golden master) and compare it to the result of the current version. If there is a difference between the output of the current system and the golden master, we may have found a bug.
If implemented right, Golden Master testing can be very useful to test legacy systems or data processing programs with complex input and outputs. Golden Master Testing is also a good addition to our regression testing processes.
In practice, implementing Golden Master testing is not quite that easy. Just checking if the outputs are equal will often not work: If the output includes times and dates or random elements, a simple comparison will not be enough. Luckily for use, with Python, we have the perfect tool to process all kinds of outputs and only look at the parts of the output that are important for the outcome of the test.
In this talk, we will look at best practices for Golden Master Testing with Python. We will see techniques to identify and quantify the relevant differences between our golden master and the current output.