continuous integration an unit tests

The standard should be that tests are executed as part of build, and that a failing test results in failing build, meaning no deployable unit or artifact is uploaded to nexus/artifactory. But every now and then there is a need to build the projects, even if tests are failing. Then you can allow for unstable build temporarily.

While effort is put into fixing the test env and tests, we can use maven properties to relax the regression test regime temporarily.
The relaxed regression test regime should only be allowed for  hours or maybe days, but not for weeks, or months!
The first thing to look for when coming to a new customer is the CI-server. You can figure out commit activity, dependencies, test execution, other tools and plugins used by CI-server.
I will never forget coming in to the “Unnamed” customer, and the chief architect was happy to show me the CI-server. But the CI-server was a mess. Failing jobs, and had been failing for months. A lot of disabled jobs, mainly to avoid annoying mails. This was clearly not a well-maintained CI-server.
And then when going into more detail of jobs, the tests was completely skipped for several of them, meaning they where not even compiled.
It was even deployed into Dev-env and SysTest-env without compiling a single test. But then when moving on further to AccTest, someone asked about test execution, and they did not even compile.

Disabling tests because they become to much hassle indicates you write the wrong type of tests. Even if the functional developer is busy producing functionality, it is the wrong priority if they are not able to produce one single test that can be rerun automatically, as part of regression testing from CI-server.
Do not maintain a large number of tests. Reduce the test set to cover the functionality being delivered from component. Test actual business value. If it is a data-driven component, include database access in the test setup.

-Dmaven.test.failure.ignore=true will ignore any failures occurred during test execution, will execute tests, but failing tests will not stop the build, it will be marked as unstable. This one is a good choice
-DskipTests=true would compile the test classes but skip test execution entirely, make sure you enable test execution as soon as temporary problem is resolved. Not so good choice
-Dmaven.test.skip=true would not even compile the tests, should only be used if the skipTests is still causing trouble, meaning the tests are not even compiling…This is bad, do you really have to do this

refer to http://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html for details

Comment are closed.