Merging Per-Suite JUnit Reports into Single File with Gradle + Kotlin

While I was preparing to open-source my tool to list the contents of nested archive files (as previously mentioned in my introductory blog post), I also set up a GitLab CI pipeline for the project. This pipeline should build the project, run the test suite, and assemble the application into a runnable artifact. If possible, it should also show the test results in the GitLab web UI.

GitLab CI has support for parsing JUnit XML files (see the docs for the config file and the handling of JUnit test reports). At the time, I thought that GitLab requires that the name of each report file be specified exactly, i.e. that it didn’t support globs/filename patterns.

I have since found out that filename patterns are, in fact, supported. (This is just hidden in the examples, and not mentioned in the reference docs at the moment.) As a result, I don’t actually need to use the merged test results described in this article.

Further note: at the time of writing this, GitLab only shows the parsed test results in merge requests, not anywhere else in their UI. (This is currently only documented by omission from the docs, and not clearly highlighted.)

Nevertheless, what I found out may be useful to other people (or myself in the future). So I’m writing up the steps anyway.

What we are trying to do

When you run JUnit tests with Gradle, the test reports are written out as one XML file per test suite (usually one test file).

If you want to have a single XML file for the entire test run (which the JUnit XML format supports), you have to merge them afterwards.

How hard can that be, right?

When I searched around the web for how to do this kind of merging with Gradle, I quickly came across this Stack Overflow answer on merging per-suite JUnit report XMLs into a single XML file, and a few similar articles. All answers that I found recommended calling an Ant task from Gradle, and using the ant-junitplugin to merge the XML files.

However, any code I could find used Gradle’s Groovy syntax, while my build script uses the newer Kotlin DSL for Gradle. And as it turns out, converting a Groovy script to Kotlin DSL is not trivial.

So instead of just stating my final solution (which you can find at the end of this post), I’ll also describe how I got there.

Groovy version

The aforementioned Stack Overflow answer recommends using the following bits in your (Groovy) build script:

configurations {
    antJUnit
}

dependencies {
    antJUnit 'org.apache.ant:ant-junit:1.9.7'
}

task mergeJUnitReports {
    ext {
        resultsDir = file("$buildDir/allreports")
        targetDir = file("$buildDir/test-results/merged")
    }

    doLast {
        ant.taskdef(name: 'junitreport',
                    classname: 'org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator',
                    classpath: configurations.antJUnit.asPath)

        ant.junitreport(todir: resultsDir) {
            fileset(dir: resultsDir, includes: 'TEST-*.xml')
            report(todir: targetDir, format: 'frames')
        }
    }
}

The first thing we do here is define a custom configuration called antJUnit, which we then use to declare a dependency on the ant-junit plugin.

The big block at the bottom defines a new Gradle task called mergeJUnitReports. Inside it, we first define two extra properties, resultsDir and targetDir, so we can more easily use these paths in the subsequent code. Then we define what should happen when executing this task, using doLast.

The action for the Gradle task first defines a new Ant task (or re-defines it if it exists), called junitreport and using XMLResultAggregator from ant-junit as its implementation. Right afterwards, we run this junitreport task, telling it to write its output to resultsDir, to read its input from resultsDir using a filename mask of Test-*.xml, and to additionally generate an HTML report in targetDir.

Kotlin DSL version, v1

To convert the Groovy version to Kotlin DSL, this migration guide on Gradle’s website came it quite handy.

Overall, we have to use strings more explicitly than in Groovy. This is particularly annoying when interacting with Ant and ant-junit, since there are no fully type-safe bindings. Instead, Groovy metaprogramming makes it possible to provide methods dynamically at runtime. In effect, Groovy allows you to call arbitrary identifiers, as long as the receiver object can handle them dynamically.

This doesn’t work in Kotlin (at least not the same way). The hard way to express the same code in Kotlin would be to manually call GroovyObject’s invokeMethod(String, Object), but that severely reduces the readability of the build script. After some more searching around, I was saved by the same migration guide mentioned above: using withGroovyBuilder {} on a GroovyObject allows you to run a closure in which you can call strings like methods (implemented with Kotlin extension methods). Coupling this with the to operator to create key-value pairs provides a reasonably compact notation for the very generic Ant interface.

Furthermore, the “Extras” section of this article describes how to use ExtraPropertiesExtension (what is called ext in Groovy and extra in Kotlin DSL). Unfortunately, the usage in Kotlin is somewhat more verbose than in Groovy.

This brings us to the follow literal translation of the above Groovy script into Kotlin DSL:

val antJUnit by configurations.creating

dependencies {
    antJUnit("org.apache.ant", "ant-junit", "1.10.5")
}

task("mergeJUnitReports") {
   extra.set("resultsDir", file("$buildDir/test-results/test"))
   extra.set("targetDir", file("$buildDir/test-results/merged"))

    doLast {
        ant.withGroovyBuilder {
            "taskdef"(
                    "name" to "junitreport",
                    "classname" to "org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator",
                    "classpath" to antJUnit.asPath
            )

            // generates an XML report
            "junitreport"("todir" to extra.get("resultsDir")) {
                "fileset"(
                        "dir" to extra.get("resultsDir"),
                        "includes" to "TEST-*.xml")
                "report"( // additionally generates an HTML report
                        "todir" to extra.get("targetDir"),
                        "format" to "frames")
            }
        }
    }
}

Kotlin DSL version, v2

After migrating the script to Kotlin DSL, I can now also make some small improvements:

I don’t currently need an HTML report, so I can just remove the entire “report” argument from the invocation of junitreport. This also makes targetDir obsolete.
I only need the remaining path resultsDir very locally in this Gradle task, so there is actually no need to use extra. Instead, a regular Kotlin variable does the job just fine.

val antJUnit by configurations.creating

dependencies {
    antJUnit("org.apache.ant", "ant-junit", "1.10.5")
}

task("mergeJUnitReports") {
    val resultsDir = file("$buildDir/test-results/test")

    doLast {
        ant.withGroovyBuilder {
            "taskdef"(
                    "name" to "junitreport",
                    "classname" to "org.apache.tools.ant.taskdefs.optional.junit.XMLResultAggregator",
                    "classpath" to antJUnit.asPath
            )

            // generates an XML report
            "junitreport"("todir" to resultsDir) {
                "fileset"(
                        "dir" to resultsDir,
                        "includes" to "TEST-*.xml")
            }
        }
    }
}

Summary

With the code snippets in this post, you should be able to adjust your Gradle Kotlin DSL build script to merge multiple JUnit XML files into a single one.

As an alternative to converting the original Groovy code to Kotlin DSL, you can also include a Groovy file into your Kotlin build script. This may serve you better in certain situations, e.g. if the script in question is more complex than this, or if someone else who doesn’t know Kotlin maintains that part of the build script.