Guessing relevant test modules with Fault Line

Launchpad's test suite takes a long time to run. Far too long to wait for. And the likelihood that you've broken any given test is pretty small. So you want to test a small subset, at least at first.

You can usually guess which tests to run; if you've changed "archive.py", you should probably run "test_archive". But some connections are easier to miss, so I've hacked up a Fault Line, a bzr plugin that uses past changes to guess which test files correlate to the files you've changed. You can run it like so:

bin/test -m $(bzr fault-line --module-regex)

This will look at all the files you've changed, look at their recent history, and see which files tended to change in the last 100 revisions where you changed the specified files. The --module-regex option causes it to output a regular expression, assuming that the test files are modules. Otherwise, it would just output a list of the test files it found.

Thanks to Jelmer Vernoij, this is even easier to achieve for testing bzr. You just need to can run "bzr selftest --auto".

Yes, Fault Line is a hacky, limited tool. But they have their place. Who knows, maybe it will become more general.