[Updated] (SUREFIRE-1751) Surefire report shows flaky tests as failures

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

[Updated] (SUREFIRE-1751) Surefire report shows flaky tests as failures

Hudson (Jira)

     [ https://issues.apache.org/jira/browse/SUREFIRE-1751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Anu updated SUREFIRE-1751:
    Attachment: Test.zip

> Surefire report shows flaky tests as failures
> ---------------------------------------------
>                 Key: SUREFIRE-1751
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1751
>             Project: Maven Surefire
>          Issue Type: Bug
>    Affects Versions: 3.0.0-M4
>            Reporter: Anu
>            Priority: Critical
>         Attachments: Test.zip, image-2020-02-13-11-41-03-906.png
> We have been using maven-surefire-plugin in 2.22.1 and Junit4.13 for our testing. Now we are trying to upgrade junit to junit5 and saw that the r{color:#6a8759}erunFailingTestsCount parameter isn't working with surefire plugin version. I upgraded surefire to 3.0.0.M4 and verified that the failing tests are now being retried. But the issue is flaky tests show up in the report as Failures.
> {color}
> {color:#6a8759}ie. the log shows
> {color}
> [WARNING] Tests run: 10455, Failures: 0, Errors: 0, Skipped: 993, Flakes: 10
> But the actual surefire report shows like below
> !image-2020-02-13-11-41-03-906.png!
> This is blocking our junit5 upgrade task. Can you please provide a fix for this at the earliest? Please let me know if you need anything else from me to help with the investigation.

This message was sent by Atlassian Jira