如何使用 Gradle 重新 运行 仅失败的 JUnit 测试 类？

Question

受到 this neat TestNG task, and this SO question 的启发，我想我会快速地为 Gradle.

中仅失败的 JUnit 测试重新 运行 做点什么

可是找了半天，没找到类似的方便的。

我想出了以下方法，它似乎工作得很好，并为我的项目中每个 Test 类型的任务添加了一个 <testTaskName>Rerun 任务。

import static groovy.io.FileType.FILES

import java.nio.file.Files
import java.nio.file.Paths

// And add a task for each test task to rerun just the failing tests
subprojects {
    afterEvaluate { subproject ->
        // Need to store tasks in static temp collection, else new tasks will be picked up by live collection leading to Whosebug 
        def testTasks = subproject.tasks.withType(Test)
        testTasks.each { testTask ->
            task "${testTask.name}Rerun"(type: Test) {
                group = 'Verification'
                description = "Re-run ONLY the failing tests from the previous run of ${testTask.name}."

                // Depend on anything the existing test task depended on
                dependsOn testTask.dependsOn 

                // Copy runtime setup from existing test task
                testClassesDirs = testTask.testClassesDirs
                classpath = testTask.classpath

                // Check the output directory for failing tests
                File textXMLDir = subproject.file(testTask.reports.junitXml.destination)
                logger.info("Scanning: $textXMLDir for failed tests.")

                // Find all failed classes
                Set<String> allFailedClasses = [] as Set
                if (textXMLDir.exists()) {
                    textXMLDir.eachFileRecurse(FILES) { f ->
                        // See: http://marxsoftware.blogspot.com/2015/02/determining-file-types-in-java.html
                        String fileType
                        try {
                            fileType = Files.probeContentType(f.toPath())
                        } catch (IOException e) {
                            logger.debug("Exception when probing content type of: $f.")
                            logger.debug(e)

                            // Couldn't determine this to be an XML file.  That's fine, skip this one.
                            return
                        }

                        logger.debug("Filetype of: $f is $fileType.") 

                        if (['text/xml', 'application/xml'].contains(fileType)) {
                            logger.debug("Found testsuite file: $f.")

                            def testSuite = new XmlSlurper().parse(f)
                            def failedTestCases = testSuite.testcase.findAll { testCase ->
                                testCase.children().find { it.name() == 'failure' }
                            }

                            if (!failedTestCases.isEmpty()) {
                                logger.info("Found failures in file: $f.")
                                failedTestCases.each { failedTestCase -> 
                                    def className = failedTestCase['@classname']
                                    logger.info("Failure: $className")
                                    allFailedClasses << className.toString()
                                }
                            }
                        }
                    }
                }

                if (!allFailedClasses.isEmpty()) {
                    // Re-run all tests in any class with any failures
                    allFailedClasses.each { c ->
                        def testPath = c.replaceAll('\.', '/') + '.class'
                        include testPath
                    }

                    doFirst {
                        logger.warn('Re-running the following tests:')
                        allFailedClasses.each { c ->
                            logger.warn(c)
                        }
                    }
                }

                outputs.upToDateWhen { false } // Always attempt to re-run failing tests
                // Only re-run if there were any failing tests, else just print warning
                onlyIf { 
                    def shouldRun = !allFailedClasses.isEmpty() 
                    if (!shouldRun) {
                        logger.warn("No failed tests found for previous run of task: ${subproject.path}:${testTask.name}.")
                    }

                    return shouldRun
                }
            }
        }
    }
}

在 Gradle 中有更简单的方法吗？有什么方法可以让 JUnit 以某种方式输出合并的失败列表，这样我就不必吞下 XML 报告了吗？

我正在使用 JUnit 4.12 和 Gradle 4.5。

Answer 1

这是一种方法。完整文件将在末尾列出，可用 here.

第一步是为每个失败的测试写一个小文件（称为 failures）：

test {
    // `failures` is defined elsewhere, see below
    afterTest { desc, result -> 
        if ("FAILURE" == result.resultType as String) {
            failures.withWriterAppend { 
                it.write("${desc.className},${desc.name}\n")
            }
        }
    }
}

在第二部分中，我们使用测试 filter（文档 here）将测试限制为 failures 文件中存在的任何测试：

def failures = new File("${projectDir}/failures.log")
def failedTests = [] 
if (failures.exists()) {
    failures.eachLine { line ->
        def tokens = line.split(",")
        failedTests << tokens[0] 
    }
}
failures.delete()

test {
    filter {
        failedTests.each { 
            includeTestsMatching "${it}"
        }
    }
    // ...
}

完整文件为：

apply plugin: 'java'

repositories {
    jcenter()
}

dependencies {
    testCompile('junit:junit:4.12')
}   

def failures = new File("${projectDir}/failures.log")
def failedTests = [] 
if (failures.exists()) {
    failures.eachLine { line ->
        def tokens = line.split(",")
        failedTests << tokens[0] 
    }
}
failures.delete()

test {
    filter {
        failedTests.each { 
            includeTestsMatching "${it}"
        }
    }

    afterTest { desc, result -> 
        if ("FAILURE" == result.resultType as String) {
            failures.withWriterAppend { 
                it.write("${desc.className},${desc.name}\n")
            }
        }
    }
}

Answer 2

Test Retry Gradle 插件正是为此而设计的。它将重新运行每个失败的测试一定次数，如果总体上发生太多失败，则可以选择使构建失败。

plugins {
    id 'org.gradle.test-retry' version '1.2.1'
}

test {
    retry {
        maxRetries = 3
        maxFailures = 20 // Optional attribute
    }
}