IT test in src/test/java is not executed by failsafe:integration-test, or fails oddly with verify

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

IT test in src/test/java is not executed by failsafe:integration-test, or fails oddly with verify

David M. Karr
I work on a large multiproject build that mostly produces OSGi bundle artifacts.  The codebase has a lot of unit tests that work fine from the build.

We also have quite a few IT tests, but we only execute those from Eclipse, as the nature of the framework makes those tests unreliable.

I now want to write a small test of IT tests that WILL be run from maven, in a CI build.  I verified that the test behaves properly when run from Eclipse.

I put the test in "src/test/java", named it with the "*IT.java" pattern, and then tried to run "failsafe:integration-test".  It ran failsafe, but it didn't find any tests.

So, on advice from someone on StackOverflow, I added an "executions" block to the plugin declaration.  Adding "integration-test" to the goals list didn't make any difference, but when I added "verify" and then ran that goal from the command line, I got very odd results.

The following is an elided version of the pom:

    <?xml version="1.0"?>
        <project
                xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"
                xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
                <modelVersion>4.0.0</modelVersion>
                <parent>
                        ...
                </parent>
                <artifactId>usl-features-install</artifactId>
                <name>usl-features-install</name>
                <url>http://maven.apache.org</url>
                <packaging>bundle</packaging>
                <properties>
                        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
                </properties>
                <build>
                        <plugins>
                                <plugin>
                                        <groupId>org.apache.maven.plugins</groupId>
                                        <artifactId>maven-failsafe-plugin</artifactId>
                                        <version>2.20.1</version>
                                        <executions>
                                                <execution>
                                                        <goals>
                                                                <goal>verify</goal>
                                                                <goal>integration-test</goal>
                                                        </goals>
                                                </execution>
                                        </executions>
                                </plugin>
                                <plugin>
                                        <groupId>org.apache.felix</groupId>
                                        <artifactId>maven-bundle-plugin</artifactId>
                                </plugin>
                                <plugin>
                                        <groupId>org.codehaus.mojo</groupId>
                                        <artifactId>build-helper-maven-plugin</artifactId>
                                        <executions>
                                                <execution>
                                                        <id>attach-artifacts</id>
                                                        <phase>package</phase>
                                                        <goals>
                                                                <goal>attach-artifact</goal>
                                                        </goals>
                                                        <configuration>
                                                                <artifacts>
                                                                ...
                                                                </artifacts>
                                                        </configuration>
                                                </execution>
                                        </executions>
                                </plugin>
                        </plugins>
                </build>
        </project>

When I run "mvn failsafe:integration-test", I see the following:

    [INFO] ------------------------------------------------------------------------
        [INFO] Building usl-features-install 3.1.0-SNAPSHOT
        [INFO] ------------------------------------------------------------------------
        [INFO]
        [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default-cli) @ usl-features-install ---
        [INFO]
        [INFO] -------------------------------------------------------
        [INFO]  T E S T S
        [INFO] -------------------------------------------------------
        [INFO]
        [INFO] Results:
        [INFO]
        [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
        [INFO]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESS
        [INFO] ------------------------------------------------------------------------

Running "mvn verify" did the following:

    [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default) @ usl-features-install ---
        [INFO]
        [INFO] -------------------------------------------------------
        [INFO]  T E S T S
        [INFO] -------------------------------------------------------
        [INFO]
        [INFO] Results:
        [INFO]
        [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
        [INFO]
        [INFO]
        [INFO] --- maven-failsafe-plugin:2.20.1:verify (default) @ usl-features-install ---
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD FAILURE
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 5.968 s
        [INFO] Finished at: 2017-10-25T10:41:26-07:00
        [INFO] Final Memory: 35M/498M
        [INFO] ------------------------------------------------------------------------
        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.20.1:verify (default) on project usl-features-install: There are test failures.
        [ERROR]
        [ERROR] Please refer to C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\failsafe-reports for the individual test results.
        [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
        [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process

Notice that it reports on failsafe twice.  The first time still doing nothing, and the second one failing.

The mentioned "dump file" had the following stacktrace, confusing the matter even more:

    java.lang.NoSuchMethodError: org.apache.maven.surefire.report.RunListener.testSetStarting(Lorg/apache/maven/surefire/report/ReportEntry;)V
                at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:235)
                at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
                at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
                at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
                at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
                at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: IT test in src/test/java is not executed by failsafe:integration-test, or fails oddly with verify

Robert Patrick
In our integration tests' parent POM, we configured the compiler plugin to run the test-compile goal:

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-compiler-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>test-compile</id>
                            <phase>test-compile</phase>
                            <goals>
                                <goal>testCompile</goal>
                            </goals>
                            <configuration>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>

And the failsafe-plugin like this:

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-failsafe-plugin</artifactId>
                    <executions>
                        <execution>
                            <id>system-integration-test</id>
                            <phase>integration-test</phase>
                            <goals>
                                <goal>integration-test</goal>
                            </goals>
                            <configuration>
                                <argLine>-Xmx2048m -XX:PermSize=128m -XX:MaxPermSize=256m</argLine>
                                <environmentVariables>
                                    <BUILD_DIR>${project.build.directory}</BUILD_DIR>
                                    <A2C_HOME>${a2c-home}</A2C_HOME>
                                </environmentVariables>
                                <systemProperties>
                                    <property>
                                        <name>java.io.tmpdir</name>
                                        <value>${project.build.directory}</value>
                                    </property>
                                </systemProperties>
                            </configuration>
                        </execution>
                        <execution>
                            <id>system-test-verify</id>
                            <phase>verify</phase>
                            <goals>
                                <goal>verify</goal>
                            </goals>
                        </execution>
                    </executions>
                </plugin>

In each test submodule POM, we simply add the following:

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-failsafe-plugin</artifactId>
            </plugin>


-----Original Message-----
From: KARR, DAVID [mailto:[hidden email]]
Sent: Wednesday, October 25, 2017 12:56 PM
To: [hidden email]
Subject: IT test in src/test/java is not executed by failsafe:integration-test, or fails oddly with verify

I work on a large multiproject build that mostly produces OSGi bundle artifacts.  The codebase has a lot of unit tests that work fine from the build.

We also have quite a few IT tests, but we only execute those from Eclipse, as the nature of the framework makes those tests unreliable.

I now want to write a small test of IT tests that WILL be run from maven, in a CI build.  I verified that the test behaves properly when run from Eclipse.

I put the test in "src/test/java", named it with the "*IT.java" pattern, and then tried to run "failsafe:integration-test".  It ran failsafe, but it didn't find any tests.

So, on advice from someone on StackOverflow, I added an "executions" block to the plugin declaration.  Adding "integration-test" to the goals list didn't make any difference, but when I added "verify" and then ran that goal from the command line, I got very odd results.

The following is an elided version of the pom:

    <?xml version="1.0"?>
        <projectxsi:schemaLocation="http://maven.apache.org/POM/4.0.0http://maven.apache.org/xsd/maven-4.0.0.xsd"xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
                <modelVersion>4.0.0</modelVersion>
                <parent>
                        ...
                </parent>
                <artifactId>usl-features-install</artifactId>
                <name>usl-features-install</name>
                <url>https://urldefense.proofpoint.com/v2/url?u=http-3A__maven.apache.org&d=DwIFAg&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057SbK10&r=Ql5uwmbofQMW0iErugdCnFgO-CBGr_pt_OzwdxJosG0&m=Gdqqce41TF232jlBHJhZ-SIZC-k-d9PREKAleK9pw0E&s=qa36PW3OrelDpW11bDjEo6LjewMSAUzW7lmp0RUt2OE&e= </url>
                <packaging>bundle</packaging>
                <properties>
                        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
                </properties>
                <build>
                        <plugins>
                                <plugin>
                                        <groupId>org.apache.maven.plugins</groupId>
                                        <artifactId>maven-failsafe-plugin</artifactId>
                                        <version>2.20.1</version>
                                        <executions>
                                                <execution>
                                                        <goals>
                                                                <goal>verify</goal>
                                                                <goal>integration-test</goal>
                                                        </goals>
                                                </execution>
                                        </executions>
                                </plugin>
                                <plugin>
                                        <groupId>org.apache.felix</groupId>
                                        <artifactId>maven-bundle-plugin</artifactId>
                                </plugin>
                                <plugin>
                                        <groupId>org.codehaus.mojo</groupId>
                                        <artifactId>build-helper-maven-plugin</artifactId>
                                        <executions>
                                                <execution>
                                                        <id>attach-artifacts</id>
                                                        <phase>package</phase>
                                                        <goals>
                                                                <goal>attach-artifact</goal>
                                                        </goals>
                                                        <configuration>
                                                                <artifacts>
                                                                ...
                                                                </artifacts>
                                                        </configuration>
                                                </execution>
                                        </executions>
                                </plugin>
                        </plugins>
                </build>
        </project>

When I run "mvn failsafe:integration-test", I see the following:

    [INFO] ------------------------------------------------------------------------
        [INFO] Building usl-features-install 3.1.0-SNAPSHOT
        [INFO] ------------------------------------------------------------------------
        [INFO]
        [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default-cli) @ usl-features-install ---
        [INFO]
        [INFO] -------------------------------------------------------
        [INFO]  T E S T S
        [INFO] -------------------------------------------------------
        [INFO]
        [INFO] Results:
        [INFO]
        [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
        [INFO]
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD SUCCESS
        [INFO] ------------------------------------------------------------------------

Running "mvn verify" did the following:

    [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default) @ usl-features-install ---
        [INFO]
        [INFO] -------------------------------------------------------
        [INFO]  T E S T S
        [INFO] -------------------------------------------------------
        [INFO]
        [INFO] Results:
        [INFO]
        [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
        [INFO]
        [INFO]
        [INFO] --- maven-failsafe-plugin:2.20.1:verify (default) @ usl-features-install ---
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD FAILURE
        [INFO] ------------------------------------------------------------------------
        [INFO] Total time: 5.968 s
        [INFO] Finished at: 2017-10-25T10:41:26-07:00
        [INFO] Final Memory: 35M/498M
        [INFO] ------------------------------------------------------------------------
        [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.20.1:verify (default) on project usl-features-install: There are test failures.
        [ERROR]
        [ERROR] Please refer to C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\failsafe-reports for the individual test results.
        [ERROR] Please refer to dump files (if any exist) [date]-jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
        [ERROR] org.apache.maven.surefire.booter.SurefireBooterForkException: There was an error in the forked process

Notice that it reports on failsafe twice.  The first time still doing nothing, and the second one failing.

The mentioned "dump file" had the following stacktrace, confusing the matter even more:

    java.lang.NoSuchMethodError: org.apache.maven.surefire.report.RunListener.testSetStarting(Lorg/apache/maven/surefire/report/ReportEntry;)V
                at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:235)
                at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
                at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
                at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
                at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
                at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: IT test in src/test/java is not executed by failsafe:integration-test, or fails oddly with verify

David M. Karr
> -----Original Message-----
> From: Robert Patrick [mailto:[hidden email]]
> Sent: Wednesday, October 25, 2017 11:32 AM
> To: Maven Users List <[hidden email]>
> Subject: RE: IT test in src/test/java is not executed by
> failsafe:integration-test, or fails oddly with verify
>
> In our integration tests' parent POM, we configured the compiler plugin
> to run the test-compile goal:

Thanks for showing how you've configured these plugins, but what in this is intended to address the issues that I described?

>                 <plugin>
>                     <groupId>org.apache.maven.plugins</groupId>
>                     <artifactId>maven-compiler-plugin</artifactId>
>                     <executions>
>                         <execution>
>                             <id>test-compile</id>
>                             <phase>test-compile</phase>
>                             <goals>
>                                 <goal>testCompile</goal>
>                             </goals>
>                             <configuration>
>                             </configuration>
>                         </execution>
>                     </executions>
>                 </plugin>
>
> And the failsafe-plugin like this:
>
>                 <plugin>
>                     <groupId>org.apache.maven.plugins</groupId>
>                     <artifactId>maven-failsafe-plugin</artifactId>
>                     <executions>
>                         <execution>
>                             <id>system-integration-test</id>
>                             <phase>integration-test</phase>
>                             <goals>
>                                 <goal>integration-test</goal>
>                             </goals>
>                             <configuration>
>                                 <argLine>-Xmx2048m -XX:PermSize=128m -
> XX:MaxPermSize=256m</argLine>
>                                 <environmentVariables>
>
> <BUILD_DIR>${project.build.directory}</BUILD_DIR>
>                                     <A2C_HOME>${a2c-home}</A2C_HOME>
>                                 </environmentVariables>
>                                 <systemProperties>
>                                     <property>
>                                         <name>java.io.tmpdir</name>
>
> <value>${project.build.directory}</value>
>                                     </property>
>                                 </systemProperties>
>                             </configuration>
>                         </execution>
>                         <execution>
>                             <id>system-test-verify</id>
>                             <phase>verify</phase>
>                             <goals>
>                                 <goal>verify</goal>
>                             </goals>
>                         </execution>
>                     </executions>
>                 </plugin>
>
> In each test submodule POM, we simply add the following:
>
>             <plugin>
>                 <groupId>org.apache.maven.plugins</groupId>
>                 <artifactId>maven-compiler-plugin</artifactId>
>             </plugin>
>             <plugin>
>                 <groupId>org.apache.maven.plugins</groupId>
>                 <artifactId>maven-failsafe-plugin</artifactId>
>             </plugin>
>
>
> -----Original Message-----
> From: KARR, DAVID [mailto:[hidden email]]
> Sent: Wednesday, October 25, 2017 12:56 PM
> To: [hidden email]
> Subject: IT test in src/test/java is not executed by
> failsafe:integration-test, or fails oddly with verify
>
> I work on a large multiproject build that mostly produces OSGi bundle
> artifacts.  The codebase has a lot of unit tests that work fine from the
> build.
>
> We also have quite a few IT tests, but we only execute those from
> Eclipse, as the nature of the framework makes those tests unreliable.
>
> I now want to write a small test of IT tests that WILL be run from
> maven, in a CI build.  I verified that the test behaves properly when
> run from Eclipse.
>
> I put the test in "src/test/java", named it with the "*IT.java" pattern,
> and then tried to run "failsafe:integration-test".  It ran failsafe, but
> it didn't find any tests.
>
> So, on advice from someone on StackOverflow, I added an "executions"
> block to the plugin declaration.  Adding "integration-test" to the goals
> list didn't make any difference, but when I added "verify" and then ran
> that goal from the command line, I got very odd results.
>
> The following is an elided version of the pom:
>
>     <?xml version="1.0"?>
> <projectxsi:schemaLocation="http://maven.apache.org/POM/4.0.0http:/
> /maven.apache.org/xsd/maven-
> 4.0.0.xsd"xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www
> .w3.org/2001/XMLSchema-instance">
> <modelVersion>4.0.0</modelVersion>
> <parent>
> ...
> </parent>
> <artifactId>usl-features-install</artifactId>
> <name>usl-features-install</name>
> <url>https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__maven.apache.org&d=DwIFAg&c=RoP1YumCXCgaWHvlZYR8PQcxBKCX5YTpkKY057Sb
> K10&r=Ql5uwmbofQMW0iErugdCnFgO-
> CBGr_pt_OzwdxJosG0&m=Gdqqce41TF232jlBHJhZ-SIZC-k-
> d9PREKAleK9pw0E&s=qa36PW3OrelDpW11bDjEo6LjewMSAUzW7lmp0RUt2OE&e= </url>
> <packaging>bundle</packaging>
> <properties>
> <project.build.sourceEncoding>UTF-
> 8</project.build.sourceEncoding>
> </properties>
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-failsafe-
> plugin</artifactId>
> <version>2.20.1</version>
> <executions>
> <execution>
> <goals>
> <goal>verify</goal>
> <goal>integration-test</goal>
> </goals>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.felix</groupId>
> <artifactId>maven-bundle-plugin</artifactId>
> </plugin>
> <plugin>
> <groupId>org.codehaus.mojo</groupId>
> <artifactId>build-helper-maven-
> plugin</artifactId>
> <executions>
> <execution>
> <id>attach-artifacts</id>
> <phase>package</phase>
> <goals>
> <goal>attach-artifact</goal>
> </goals>
> <configuration>
> <artifacts>
> ...
> </artifacts>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
> </build>
> </project>
>
> When I run "mvn failsafe:integration-test", I see the following:
>
>     [INFO] -------------------------------------------------------------
> -----------
> [INFO] Building usl-features-install 3.1.0-SNAPSHOT
> [INFO] ------------------------------------------------------------
> ------------
> [INFO]
> [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default-
> cli) @ usl-features-install ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------
> ------------
>
> Running "mvn verify" did the following:
>
>     [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default) @
> usl-features-install ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
> [INFO] --- maven-failsafe-plugin:2.20.1:verify (default) @ usl-
> features-install ---
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Total time: 5.968 s
> [INFO] Finished at: 2017-10-25T10:41:26-07:00
> [INFO] Final Memory: 35M/498M
> [INFO] ------------------------------------------------------------
> ------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-
> failsafe-plugin:2.20.1:verify (default) on project usl-features-install:
> There are test failures.
> [ERROR]
> [ERROR] Please refer to C:\users\<myuid>\git\oce_usl\usl-
> parent\usl-features-install\target\failsafe-reports for the individual
> test results.
> [ERROR] Please refer to dump files (if any exist) [date]-
> jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
> [ERROR]
> org.apache.maven.surefire.booter.SurefireBooterForkException: There was
> an error in the forked process
>
> Notice that it reports on failsafe twice.  The first time still doing
> nothing, and the second one failing.
>
> The mentioned "dump file" had the following stacktrace, confusing the
> matter even more:
>
>     java.lang.NoSuchMethodError:
> org.apache.maven.surefire.report.RunListener.testSetStarting(Lorg/apache
> /maven/surefire/report/ReportEntry;)V
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Pro
> vider.java:235)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.ja
> va:161)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassL
> oader(ForkedBooter.java:373)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedB
> ooter.java:334)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:
> 119)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407
> )
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: IT test in src/test/java is not executed by failsafe:integration-test, or fails oddly with verify

David M. Karr
In reply to this post by David M. Karr
> -----Original Message-----
> From: KARR, DAVID
> Sent: Wednesday, October 25, 2017 10:56 AM
> To: [hidden email]
> Subject: IT test in src/test/java is not executed by
> failsafe:integration-test, or fails oddly with verify
>
> I work on a large multiproject build that mostly produces OSGi bundle
> artifacts.  The codebase has a lot of unit tests that work fine from the
> build.
>
> We also have quite a few IT tests, but we only execute those from
> Eclipse, as the nature of the framework makes those tests unreliable.
>
> I now want to write a small test of IT tests that WILL be run from
> maven, in a CI build.  I verified that the test behaves properly when
> run from Eclipse.
>
> I put the test in "src/test/java", named it with the "*IT.java" pattern,
> and then tried to run "failsafe:integration-test".  It ran failsafe, but
> it didn't find any tests.
>
> So, on advice from someone on StackOverflow, I added an "executions"
> block to the plugin declaration.  Adding "integration-test" to the goals
> list didn't make any difference, but when I added "verify" and then ran
> that goal from the command line, I got very odd results.
>
> The following is an elided version of the pom:
>
>     <?xml version="1.0"?>
> <projectxsi:schemaLocation="http://maven.apache.org/POM/4.0.0http:/
> /maven.apache.org/xsd/maven-
> 4.0.0.xsd"xmlns="http://maven.apache.org/POM/4.0.0"xmlns:xsi="http://www
> .w3.org/2001/XMLSchema-instance">
> <modelVersion>4.0.0</modelVersion>
> <parent>
> ...
> </parent>
> <artifactId>usl-features-install</artifactId>
> <name>usl-features-install</name>
> <url>https://urldefense.proofpoint.com/v2/url?u=http-
> 3A__maven.apache.org&d=DwIFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=OsTemSXEn-
> xy2uk0vYF_EA&m=W15vi5llM1SmJsj7-
> g2VFzZ5ukS3vXQL3Rsy29lSpys&s=vk_ZTOxGLHdUOjOv4Env7MsI-5-
> FKM1Di5FmCyO24sA&e= </url>
> <packaging>bundle</packaging>
> <properties>
> <project.build.sourceEncoding>UTF-
> 8</project.build.sourceEncoding>
> </properties>
> <build>
> <plugins>
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-failsafe-
> plugin</artifactId>
> <version>2.20.1</version>
> <executions>
> <execution>
> <goals>
> <goal>verify</goal>
> <goal>integration-test</goal>
> </goals>
> </execution>
> </executions>
> </plugin>
> <plugin>
> <groupId>org.apache.felix</groupId>
> <artifactId>maven-bundle-plugin</artifactId>
> </plugin>
> <plugin>
> <groupId>org.codehaus.mojo</groupId>
> <artifactId>build-helper-maven-
> plugin</artifactId>
> <executions>
> <execution>
> <id>attach-artifacts</id>
> <phase>package</phase>
> <goals>
> <goal>attach-artifact</goal>
> </goals>
> <configuration>
> <artifacts>
> ...
> </artifacts>
> </configuration>
> </execution>
> </executions>
> </plugin>
> </plugins>
> </build>
> </project>
>
> When I run "mvn failsafe:integration-test", I see the following:
>
>     [INFO] -------------------------------------------------------------
> -----------
> [INFO] Building usl-features-install 3.1.0-SNAPSHOT
> [INFO] ------------------------------------------------------------
> ------------
> [INFO]
> [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default-
> cli) @ usl-features-install ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD SUCCESS
> [INFO] ------------------------------------------------------------
> ------------
>
> Running "mvn verify" did the following:
>
>     [INFO] --- maven-failsafe-plugin:2.20.1:integration-test (default) @
> usl-features-install ---
> [INFO]
> [INFO] -------------------------------------------------------
> [INFO]  T E S T S
> [INFO] -------------------------------------------------------
> [INFO]
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> [INFO]
> [INFO] --- maven-failsafe-plugin:2.20.1:verify (default) @ usl-
> features-install ---
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] BUILD FAILURE
> [INFO] ------------------------------------------------------------
> ------------
> [INFO] Total time: 5.968 s
> [INFO] Finished at: 2017-10-25T10:41:26-07:00
> [INFO] Final Memory: 35M/498M
> [INFO] ------------------------------------------------------------
> ------------
> [ERROR] Failed to execute goal org.apache.maven.plugins:maven-
> failsafe-plugin:2.20.1:verify (default) on project usl-features-install:
> There are test failures.
> [ERROR]
> [ERROR] Please refer to C:\users\<myuid>\git\oce_usl\usl-
> parent\usl-features-install\target\failsafe-reports for the individual
> test results.
> [ERROR] Please refer to dump files (if any exist) [date]-
> jvmRun[N].dump, [date].dumpstream and [date]-jvmRun[N].dumpstream.
> [ERROR]
> org.apache.maven.surefire.booter.SurefireBooterForkException: There was
> an error in the forked process
>
> Notice that it reports on failsafe twice.  The first time still doing
> nothing, and the second one failing.
>
> The mentioned "dump file" had the following stacktrace, confusing the
> matter even more:
>
>     java.lang.NoSuchMethodError:
> org.apache.maven.surefire.report.RunListener.testSetStarting(Lorg/apache
> /maven/surefire/report/ReportEntry;)V
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Pro
> vider.java:235)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.ja
> va:161)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassL
> oader(ForkedBooter.java:373)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedB
> ooter.java:334)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:
> 119)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407
> )

If it matters, I tried running with "-X".  The relevant part of the build output is this:
------------------
        [DEBUG] Forking command line: cmd.exe /X /C ""C:\Program Files\Java\jdk1.8.0_102\jre\bin\java" -javaagent:C:\\Users\\<myuid>\\.m2\\repository\\org\\jacoco\\org.jacoco.agent\\0.7.8\\org.jacoco.agent-0.7.8-runtime.jar=destfile=C:\\users\\<myuid>\\git\\oce_usl\\usl-parent\\usl-features-install\\target\\jacoco.exec -jar C:\cygwin64\tmp\surefire29734070406705707\surefirebooter6462732870353032119.jar C:\cygwin64\tmp\surefire29734070406705707 2017-10-25T15-17-49_742-jvmRun1 surefire1801590010194716332tmp surefire_05587249170077752672tmp"
        [INFO]
        [INFO] Results:
        [INFO]
        [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
        [INFO]
        [INFO]
        [INFO] --- maven-failsafe-plugin:2.20.1:verify (default) @ usl-features-install ---
        [DEBUG] Configuring mojo org.apache.maven.plugins:maven-failsafe-plugin:2.20.1:verify from plugin realm ClassRealm[plugin>org.apache.maven.plugins:maven-failsafe-plugin:2.20.1, parent: sun.misc.Launcher$AppClassLoader@5c647e05]
        [DEBUG] Configuring mojo 'org.apache.maven.plugins:maven-failsafe-plugin:2.20.1:verify' with basic configurator -->
        [DEBUG]   (s) basedir = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install
        [DEBUG]   (s) reportsDirectory = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\surefire-reports
        [DEBUG]   (s) skip = false
        [DEBUG]   (f) summaryFile = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\failsafe-reports\failsafe-summary.xml
        [DEBUG]   (s) testClassesDirectory = C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\test-classes
        [DEBUG]   (s) testFailureIgnore = false
        [DEBUG]   (f) session = org.apache.maven.execution.MavenSession@6bf13698
        [DEBUG] -- end configuration --
        [DEBUG] Failsafe report directory: C:\users\<myuid>\git\oce_usl\usl-parent\usl-features-install\target\surefire-reports
        [INFO] ------------------------------------------------------------------------
        [INFO] BUILD FAILURE
--------------------------

I don't know if it matters, but this is including the jacoco agent that we use for unit tests.  Could that be causing failsafe to not find any tests?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]