Tycho/JUnit/Jacoco for the Industrial SQL Connector for Mylyn

In the previous installments I created a repeatable Maven build from Hudson, and I set up static code analysis using SonarQube. It was revealed that there is considerable technical debt, and half of that is caused by 0% Coverage. Getting Test Coverage in place has to be done before tackling any of the other issues like duplication, complexity and violations.

Project Setup

The Project setup for the Industrial SQL Connector for Mylyn is slightly different from other examples found on the internet. The main abstract functionality is handled in the main two plugins:

• com.industrialtsi.mylyn.core for headless stuff and connection handling, and
• com.industrialtsi.mylyn.ui for the UI part.

Functionality for a specific kind of database is then "injected" using a fragment project. Three example projects are included:

• com.industrialtsi.mylyn.demo.memory is a very simple in memory task list for demonstration and testing purposes.
• com.industrialtsi.mylyn.demo.jpa accesses a simple Derby database but using EclipseLink JPA annotations
• com.industrialtsi.mylyn.demo.derby contains a link to a Derby database using the apache ibatis xml based SQL access language, that will allow quite complicated JOIN and UNION statements in its queries.
• org.apache.ibatis finally contains the ibatis stuff packaged in a separate plugin.

All the test code for all these projects finally is placed in a single plugin:

• com.industrialtsi.mylyn.tests contains al test code.

Getting the test code to compile under Tycho

The first step is to add the test project to the main POM in industrialtsi.mylyn.maven. When we run a simple maven build compilation fails:

[ERROR] Failed to execute goal org.eclipse.tycho:tycho-compiler-plugin:0.19.0:compile (default-compile) on project com.industrialtsi.mylyn.tests: Compilation failure: Compilation failure:
[ERROR] /Users/maarten/Workspaces/workspace-industrial-google/com.industrialtsi.mylyn.tests/src/com/industrialtsi/mylyn/demo/derby/test/DerbyIbatisPersistorTest.java:[18]
[ERROR] import com.industrialtsi.mylyn.core.persistence.DerbyIbatisPersistor;
[ERROR] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ERROR] The import com.industrialtsi.mylyn.core.persistence.DerbyIbatisPersistor cannot be resolved
[ERROR] /Users/maarten/Workspaces/workspace-industrial-google/com.industrialtsi.mylyn.tests/src/com/industrialtsi/mylyn/demo/derby/test/DerbyIbatisPersistorTest.java:[43]
[ERROR] return new DerbyIbatisPersistor();
[ERROR] ^^^^^^^^^^^^^^^^^^^^
[ERROR] DerbyIbatisPersistor cannot be resolved to a type
[ERROR] 2 problems (2 errors)

When compiling in the workspace, eclipse is very helpful in finding all the needed dependencies. As the missing import is located in a fragment of another plugin, normal dependency tricks via the MANIFEST.MF of com.industrialtsi.mylyn.tests or as dependency in the POM file don't work. The quick solution was to include the missing jar as Extra Classpath Entry in the build.properties of com.industrialtsi.mylyn.tests. If anybody knows a better way, please let me know!

Adding tycho-surefire to pom.xml

Next we need to configure the tycho surefire plugin to run the tests and process the results. I'm can rely on the tycho-surefire plugin to load my tests plugin and its direct dependencies. But I want to run a full workbench which I define with the <application> and <product> tags. I also want the com.industrialtsi.mylyn.demo.memory and com.industrialtsi.mylyn.demo.derby fragments to be loaded, which I do with the <dependencies> section. Then I disable the more complicated test till later.

<plugin>
    <groupId>org.eclipse.tycho</groupId>
    <artifactId>tycho-surefire-plugin</artifactId>
    <version>${tycho-version}</version>
    <configuration>
        <argLine>${ui.test.vmargs}</argLine>
        <useUIHarness>true</useUIHarness>
        <useUIThread>true</useUIThread>
        <product>org.eclipse.platform.ide</product>
        <application>org.eclipse.ui.ide.workbench</application>
        <dependencies>
            <dependency>
                <type>eclipse-plugin</type>
                <artifactId>com.industrialtsi.mylyn.demo.memory</artifactId>
                <version>0.9.10</version>
            </dependency>
            <dependency>
                <type>eclipse-plugin</type>
                <artifactId>com.industrialtsi.mylyn.demo.derby</artifactId>
                <version>0.9.10</version>
            </dependency>
        </dependencies>
        <includes>
            <include>**/*Test.java</include>
        </includes>
        <excludes>
            <exclude>**/IbatisPersistorTest.*</exclude>
            <exclude>**/PersistorsManagerTest.*</exclude>
            <exclude>**/DemoDerbyTest.*</exclude>
            <exclude>**/DerbyIbatisPersistorTest.*</exclude>
            <exclude>**/IbatisCorePluginTest.*</exclude>
            <exclude>**/TaskCreationTest.*</exclude>
        </excludes>
        <!-- Kill test JVM if tests take more than 10 minutes (600 seconds)
            to finish -->
        <forkedProcessTimeoutInSeconds>600</forkedProcessTimeoutInSeconds>
    </configuration>
</plugin>

Controlling memory and start Thread on Mac

Running on a Mac requires different startup parameters passed with the <argLine> tag. This is handeld best in the profiles section of the POM:

<profiles>
    <profile>
        <id>macosx</id>
        <activation>
            <os>
                <family>mac</family>
            </os>
        </activation>
        <properties>
            <ui.test.vmargs>-Xmx512m -XX:MaxPermSize=256m -XstartOnFirstThread</ui.test.vmargs>
        </properties>
    </profile>
    <profile>
        <id>other-os</id>
        <activation>
            <os>
                <family>!mac</family>
            </os>
        </activation>
        <properties>
            <ui.test.vmargs>-Xmx512m -XX:MaxPermSize=256m</ui.test.vmargs>
        </properties>
    </profile>
</profiles>

ready to run the tests in the workspace

Then we need to run a Tycho build using the m2e tools. It is important to realize that Plug-in Unit Tests are run in the integration test phase, so we need to specify that as a target.

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.industrialtsi.mylyn.core.dto.IndustrialQueryParamsTest
Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.039 sec
Running com.industrialtsi.mylyn.test.db.core.GenericQueryParamsTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.01 sec

Results :

Tests run: 23, Failures: 0, Errors: 0, Skipped: 0

[INFO] All tests passed!
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO] 
[INFO] com.industrialtsi.mylyn.maven ..................... SUCCESS [0.073s]
[INFO] org.apache.ibatis ................................. SUCCESS [1.477s]
[INFO] com.industrialtsi.mylyn.core ...................... SUCCESS [1.605s]
[INFO] com.industrialtsi.mylyn.demo.memory ............... SUCCESS [0.386s]
[INFO] com.industrialtsi.mylyn.ui ........................ SUCCESS [0.997s]
[INFO] com.industrialtsi.mylyn.feature ................... SUCCESS [2.616s]
[INFO] com.industrialtsi.mylyn.demo.derby ................ SUCCESS [0.685s]
[INFO] org.apache.ibatis.feature ......................... SUCCESS [2.321s]
[INFO] com.industrialtsi.mylyn.demo.derby.feature ........ SUCCESS [2.329s]
[INFO] com.industrialtsi.mylyn.demo.jpa .................. SUCCESS [1.254s]
[INFO] com.industrialtsi.mylyn.demo.jpa.feature .......... SUCCESS [2.399s]
[INFO] com.industrialtsi.mylyn.site ...................... SUCCESS [3.129s]
[INFO] com.industrialtsi.mylyn.tests ..................... SUCCESS [7.804s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 57.488s
[INFO] Finished at: Tue Jan 21 11:33:31 CET 2014
[INFO] Final Memory: 59M/554M
[INFO] ------------------------------------------------------------------------

Adding Jacoco coverage to the pom.xml

Lets start with adding the Jacoco plugin to the POM file.

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.6.4.201312101107</version>
    <executions>
        <execution>
            <id>prepare-integration-tests</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
            <configuration>
                <include>com.industrialtsi.*</include>
                <include>org.junit.*</include>
                <!-- Where to put jacoco coverage report -->
                <destFile>${sonar.jacoco.reportPath}</destFile>
                <append>true</append>
            </configuration>
        </execution>
    </executions>
</plugin>

Configuring a multi-module plugin project for coverage with Jacoco has some special hurdles which easily lead to frustration. All coverage code must be placed in a single *.exec file to analyzed later, this is handled by the <destFile>${sonar.jacoco.reportPath}</destFile> line. This property is defined in the properties section:

<sonar.jacoco.reportPath>${project.build.directory}/../../com.industrialtsi.mylyn.tests/target/jacoco.exec</sonar.jacoco.reportPath>

Tycho has a special argline variable to pass to the tycho-surefire plugin named tycho.testArgLine that we must pass on in the tycho-surefire plugin:

    <artifactId>tycho-surefire-plugin</artifactId>
    <version>${tycho-version}</version>
    <configuration>
        <argLine>${ui.test.vmargs} ${tycho.testArgLine}</argLine>

Now we can run again and get to see that jacoco.exec is actually generated in the workspace. We can now run the unit tests from the test plugin and measure code coverage with jacoco from a maven build.

not to be forgotten: SonarQube setup

We are running SonarQube from Hudson using the Run Standalone Sonar Analysis build step. We want that analysis to reuse the jacoco.exec file generated during the maven integration-test step. The standalone step uses sonar-project.properties for configuration. So we add:

sonar.dynamicAnalysis=reuseReports
sonar.jacoco.reportPath=../com.industrialtsi.mylyn.tests/target/jacoco.exec

If we had run SonarQube from maven using the sonar:sonar or verify targets, we should configure sonar in the pom.xml.

<properties>
    ...
    <sonar.java.coveragePlugin>jacoco</sonar.java.coveragePlugin>
    <sonar.dynamicAnalysis>reuseReports</sonar.dynamicAnalysis>
    ...
</properties>

I do both know that I'm working on it so I have flexibility later.

Now commit to SCM and start a Hudson build

Start a Hudson build manually or wait for SCM polling to kick in, but the result looks promising: 3.63% Coverage is not much, but easy to expand now that this works.

Finally: results in SonarQube

I managed to reduce Technical Debt with $ 390 and one man day compared to the start of this blog post. Seems like a low yield for a half day work, but I guess the calculations do not allow for the ramp up cost: running the first coverage is hardest. Progress should be easier now that the build and static analysis infrastructure is in place. Unit Test Coverage is up from 0% with only the two simplest tests executing. Next steps are activating the tests disabled earlier and adding more test. To be continued...