How to Configure Hudson to Monitor Test Coverage Stats

Goal:   configure a Hudson project so it will squawk if the test coverage stats drop below the current coverage levels.

I researched this a while ago, and perhaps this will spare a few folks some effort.

It turns out there are two separate conditions that are related:

1.  job states:    successful / unstable / broken / disabled
 this is displayed as the color of the dot next to an individual build.

2.   job stability (weather icon):
"While a job may build to completion and generate the target artifacts without issue, Hudson will assign a stability score to the build (from 0-100) based on the post-processor tasks, implemented as plugins, that you have set up to implicitly evaluate stability."  These can include unit tests (JUnit, etc.), coverage (Cobertura, Rcov, etc.), and static code analysis (FindBugs). The higher the score, the more stable the build.

settings:
 bright sun (80-100)
 partly cloudy (60-79)
 cloudy (40-59)
 raining (20-39)
 stormy (0-19)


Now for the details about coverage metric settings:

 If you go into "configure" on your project, and have "Publish (coverage) report" turned on, you'll see there are rows (in Cobertura, for things like "classes" "methods" "lines") and then there are three columns.  Here's what they mean:

bright sun (left column):
 the minimum coverage level required for a bright sunny weather indicator on the dashboard.

stormy (middle column):
 the minimum coverage level to avoid stormy icon.

plain sun (rightmost column)
 the minimum test coverage required for a stable build.
 so you should put your current coverage HERE, and your build will be marked unstable if you go below your current coverage percentage.


My interpretation is the first two columns affect your weather icon (job stability), and the third column affects the job state (color of the dot by an individual build).


- Naomi

sources:

http://www.javaworld.com/javaworld/jw-12-2008/jw-12-hudson-ci.html?page=7

http://books.google.com/books?id=YoTvBpKEx5EC&pg=PA369&lpg=PA369&dq=hudson+setting+cobertura+coverage+metrics+targets&source=bl&ots=eJw1L5oit9&sig=6fnE54EDRICZsN6nNcYXKbF8cXQ&hl=en&ei=5wvCTOy3MYXEsAOn9dhB&sa=X&oi=book_result&ct=result&resnum=3&ved=0CCUQ6AEwAg#v=onepage&q&f=false