Description
Tom Viner - Testing with two failure seeking missiles: fuzzing and property based testing [EuroPython 2015] [23 July 2015] [Bilbao, Euskadi, Spain]
[Full Video of Talk][0]
Testing with purely random data on it's own doesn't get you very far. But two approaches that have been around for a while have new libraries that help you generate random input, that homes in on failing testcases.
First [Hypothesis][1], a Python implementation and update of the Haskell library QuickCheck. Known as property based testing, you specify a property of your code that must hold, and Hypothesis does its best to find a counterexample. It then shrinks this to find the minimal input that contradicts your property.
Video: https://www.youtube.com/watch?v=YGim-BL_ebE#t=53m17s
Second, [American fuzzy lop][2] (AFL), is a young fuzzing library that's already achieved an impressive trophy case of bug discoveries. Using instrumentation and genetic algorithms, it generates test input that carefully searches out as many code paths as it can find, seeking greater functional coverage and ultimately locating crashes and hangs that no other method has found. I'll be showing how with [Python-AFL][3] we can apply this tool to our Python code.
[0]: https://youtu.be/YGim-BL_ebE?t=53m17s [1]: https://hypothesis.readthedocs.org/en/latest/ [2]: http://lcamtuf.coredump.cx/afl/ [3]: http://jwilk.net/software/python-afl