The most stringent coverage policy is expressed in the sample directory in the file "coveragePolicy.js:"
policy.aimsFor.sourceFiles
.toHave( valueOf("elementCounts.samePackageMissed").notMoreThan(0) );
As you might guess, this expresses the idea that every line and branch in each file should be covered by a unit test in the same package. There are many many other policies that you can configure in language like this, as we will discuss shortly.
The coverage policy lives in a single file specified in the "coveragePolicy" property of the package-coverage-report ant task or the Gradle plugin. If the value you supply is a directory, the task will assume the name of the file is "coveragePolicy.js" in that directory. The "sample" code base has an example in ant and in Gradle.
Here are several small variations on the first example above. Here, think of each line individually as a policy.
// Every line and branch covered by a test in same package
policy.aimsFor.sourceFiles
.toHave( valueOf( "elementCounts.samePackageMissed").notMoreThan(0) );
// Every line covered in the same package; we do not care about branches.
policy.aimsFor.sourceFiles
.toHave( valueOf( "lineCounts.samePackageMissed").notMoreThan(0) );
// Every branch and line covered by a test in whatever package
policy.aimsFor.sourceFiles
.toHave( valueOf( "elementCounts.missed" ).notMoreThan(0) );
// At least 99% coverage in each source file
policy.aimsFor.sourceFiles
.toHave( valueOf( "elementCounts.rate" ).notLessThan( .99 );
// At least 95% of lines coverage by tests in the same package
policy.aimsFor.sourceFiles
.toHave( valueOf("lineCounts.samePackageRate").notLessThan(.95) );
(While the arguments to "valueOf" expressions are fully explained in the JSON format documentation, it is probably enough to know you can consider elementCounts or lineCounts, and use the following numeric properties on either:
covered, missed, rate samePackageCovered, samePackageMissed, samePackageRate
You may have no expectation of test coverage for some parts of your code.
policy.aimsFor.sourceFiles .toHave( valueOf( "elementCounts.samePackageMissed").notMoreThan(0) ) .exceptIf( valueOf( "name" ).matches( "GeneratedSourceFile\.java$" ) );
Here, "name" refers to the name of the source file being tested, because the requirement applies to "sourceFiles." Let's imagine, in addition, there are some packages where a less stringent coverage standard is all that is expected:
policy.aimsFor.sourceFiles
.toHave( valueOf( "elementCounts.samePackageMissed").notMoreThan(0) )
.exceptIf( valueOf( "parent.name" ).matches( "legacy" ) )
.inWhichCase( valueOf( "lineCounts.rate" ).notLessThan(.8) );
.exceptIf( valueOf( "name" ).matches( "GeneratedSourceFile\.java$" ) )
Please note two things in this example. First, "parent.name" is the name of the package containing the name of the given source file. Second, if multiple "exceptIf" clauses apply, only the last one counts. So, the policy makes not requirements on a source file "com.foo.legacy.GeneratedSourceFile.java," because it matches the second clause, irrespective of the fact it also matches the first.
This strange languae is, in fact, real JavaScript. In fact, the "valueOf" expressions actually creates predicate objects that can be combined with boolean expressions. Moreover, declarations can be externalized for clarity or re-use:
// Exclude a package and anything with a certain name
var itIsAGeneratedFile =
valueOf( "parent.name" ).matches( "^com.foo.generated.package" )
.or( valueOf( "name" ).matches( "GeneratedSourceFile\.java$" ) );
policy.aimsFor.sourceFiles
.toHave( valueOf( "elementCounts.samePackageMissed").notMoreThan(0) )
.exceptIf( itIsAGeneratedFile );
The predicate objects provide "and(expr)," "andNot(expr)," and "orNot(expr)" function, as well as the "or()" illustrated here. You can also invert any predicate with ".not()" with no argeuments after any predicate. In fact, "orNot(expr)" is just a more readable version of "or( expr.not() )."
While there are a lot of practical advantages to simple and precise policies, the configuration language supports more complicated policies, too. For example, the following 4 clauses, taken together, express a policy that has relatively stringent goals for the entire project, but allows a bit of leeway on each package, yet still allows only 10 missed elements in any individual source file. We also throw in some exceptions for legacy packages. (We introduce the introduction of the "wholeProject" and "package" enumerations, which are analogous to the "sourceFiles" enumeration used previously.)
// This package and subpackages have a lower standard
var legacyPackageRegex = "^com.foo.legacy";
// Overall rate of 98%...
policy.aimsFor.wholeProject
.toHave( valueOf( "elementCounts.rate" ).notLessThan( .98 ) );
// ... and each individual package will have at least 95% coverage...
policy.aimsFor.packages
.toHave( valueOf( "elementCounts.rate").notLessThan(.95) )
.exceptIf( valueOf( "name" ).matches( legacyPackageRegex )
.inWhichCase( valueOf( "elementCounts.rate" ).notLessThan(.65) );
// ... and 90% coverage by tests in that same package...
policy.aimsFor.packages
.toHave( valueOf( "elementCounts.samePackageRate").notLessThan(.90) )
.exceptIf( valueOf( "name" ).matches( legacyPackageRegex );
// ... and no more than 10 missed elements in any individual file
policy.aimsFor.sourceFiles
.toHave( valueOf( "elementCounts.missed").notMoreThan(10) )
.exceptIf( valueOf( "parent.name" ).matches( legacyPackageRegex );
PurrPackage is written to support a whatever policy works for your team, and also, to allow answer to evolve over time.
When writing PurrPackage, we had some ideas in mind, of course. We believe that 100% per-package code coverage is the best coverage goal, all other things being equal. However, we also believe in other things that tend to make strict 100% coverage unrealistic, such as respecting business prioirites, taking an incremental approach to development, and using existing code and libraries when possible.
We imagine approaching an existing project with a a coverage goal that reflect the status quo, with the aim to gradually create areas with 100% coverage in the course of working on the code for other reasons. Over time, we expect the rigorously tested area will grow. In general, we prefer a few specific exceptions to a stringent general goal to the alternatives in either direction: on one hand, relaxing the general goal, or, on the other, resigning oneself to never actually meeting the goal.