art with code

2009-01-15

PreArray testing stats

As I finished the first pass at PreArray unit tests yesterday, I thought to take a quick look at the testing statistics for the PreArray module. Before the test writing pass, PreArray contained 299 non-blank lines (of which 252 source, the rest tests and section headers.) With the tests, the line count rose to 1357 (of which 290 source), giving around 1020 new test lines, for a code:tests ratio of 1:3.7.

I quickly counted the number of bugfixes with gitk, and found 16 bugs fixed. So, one bug found (and fixed) for every 64 lines of tests. And one bug per every 18 lines of code for 55 bugs/ksloc for PreArray.

Compared to the previous numbers of bug per 50 test lines and 24 bugs/ksloc, PreArray has more tests per line of code, less bugs found per test, and a more than doubled bug rate per line of code (24 bugs/ksloc vs 55 bugs/ksloc.) What are the main differences between PreArray and the previously tested part? The main thing increasing bug rate might be that PreArray does [unsafe] indexed access to arrays, making sub/slice indexing harder than with lists, as the indexes need to be clamped to the array's dimensions.

Lists were perhaps less buggy because they have nicer provability (the x::xs -pattern matching, and a naturally recursive coding style.) Whereas arrays tend towards random access, which is harder to reason about.

And quite a few of the list functions and the preceding stuff was copied from prelude.hs, which would lower bug rate :)

No comments:

Blog Archive