Quantcast

Configuring the surefire plugin - a question

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Configuring the surefire plugin - a question

Paul Hammant
Hi folks,.

I've a fast WebDriver using build that I blogged about: A 16 Second Java
Webapp Build (Including WebDriver Tests)
<http://paulhammant.com/2017/02/05/a-16-second-java-webapp-build-including-webdriver-tests/>
.

Jooby (like SpringBoot and SparkJava) give new options for testing - it can
be instantiated in a JUnit test. Everything can be done in Surefire now,
and the Failsafe plugin isn't needed for these.  Don't believe me - watch
the video in the blog entry above, it's not long.

New problem. I want unit tests to run in this order:

1. unit
2. integration (may invoke service calls headlessly)
3. function (will use WebDriver)


I can't work out what magic I have to do with executions to allow that to
happen.

Here is how far I got:


https://github.com/paul-hammant/todobackend-jooby/blob/master/pom.xml#L74

It is all a bit second class, because I'd have to do ...

mvn clean test -Punit-tests
mvn test -Pintegration-tests
mvn test -Pfunctional-tests

... to simulate a pipeline, and I would be happy to just rely on
annotations for classifications.

I really want to do ...

mvn clean test -DexecutionOrder=unit,integration,functional
-DstopBuildAtExecutionBoundariesForTestFailures


... and shave seconds off the build.

How do I configure that tersely and elegantly in the surefire plugin today?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Configuring the surefire plugin - a question

João Cabrita
I'd say you could add executions to the surefire plugin with different
categories:
https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html#Using_JUnit_Categories

Look at this gist (I've omitted some details):
https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c

AFAIK, even if the executions are all bound to the same phase in the
lifecycle, they execute in the order specificied in the POM.
Beware this is the behavior I've observed and can't confirm it is specified
behavior.



João Cabrita

On 16 February 2017 at 10:06, Paul Hammant <[hidden email]> wrote:

> Hi folks,.
>
> I've a fast WebDriver using build that I blogged about: A 16 Second Java
> Webapp Build (Including WebDriver Tests)
> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
> build-including-webdriver-tests/>
> .
>
> Jooby (like SpringBoot and SparkJava) give new options for testing - it can
> be instantiated in a JUnit test. Everything can be done in Surefire now,
> and the Failsafe plugin isn't needed for these.  Don't believe me - watch
> the video in the blog entry above, it's not long.
>
> New problem. I want unit tests to run in this order:
>
> 1. unit
> 2. integration (may invoke service calls headlessly)
> 3. function (will use WebDriver)
>
>
> I can't work out what magic I have to do with executions to allow that to
> happen.
>
> Here is how far I got:
>
>
> https://github.com/paul-hammant/todobackend-jooby/blob/master/pom.xml#L74
>
> It is all a bit second class, because I'd have to do ...
>
> mvn clean test -Punit-tests
> mvn test -Pintegration-tests
> mvn test -Pfunctional-tests
>
> ... to simulate a pipeline, and I would be happy to just rely on
> annotations for classifications.
>
> I really want to do ...
>
> mvn clean test -DexecutionOrder=unit,integration,functional
> -DstopBuildAtExecutionBoundariesForTestFailures
>
>
> ... and shave seconds off the build.
>
> How do I configure that tersely and elegantly in the surefire plugin today?
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Configuring the surefire plugin - a question

rfscholte
When you only want to change the pom (not the tests), a set of  
executionblocks will do the trick:

       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-surefire-plugin</artifactId>
         <version>2.18.1</version>
         <executions>
           <execution>
             <id>unit-tests</id>
             <configuration>
             <includes>
               <include>**/*Unit*.java</include>
             </includes>
             </configuration>
           </execution>
           <execution>
             <id>functional-tests</id>
             <configuration>
             <includes>
               <include>**/*WebDriver*.java</include>
             </includes>
             </configuration>
           </execution>
           <execution>
             <id>integration-tests</id>
             <configuration>
             <includes>
               <include>**/*Integration*.java</include>
             </includes>
             </configuration>
           </execution>
         <executions>
       </plugin>

On Thu, 16 Feb 2017 11:22:37 +0100, João Cabrita  
<[hidden email]> wrote:

> I'd say you could add executions to the surefire plugin with different
> categories:
> https://maven.apache.org/surefire/maven-surefire-plugin/examples/junit.html#Using_JUnit_Categories
>
> Look at this gist (I've omitted some details):
> https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c
>
> AFAIK, even if the executions are all bound to the same phase in the
> lifecycle, they execute in the order specificied in the POM.
> Beware this is the behavior I've observed and can't confirm it is  
> specified
> behavior.
>
>
>
> João Cabrita
>
> On 16 February 2017 at 10:06, Paul Hammant <[hidden email]> wrote:
>
>> Hi folks,.
>>
>> I've a fast WebDriver using build that I blogged about: A 16 Second Java
>> Webapp Build (Including WebDriver Tests)
>> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
>> build-including-webdriver-tests/>
>> .
>>
>> Jooby (like SpringBoot and SparkJava) give new options for testing - it  
>> can
>> be instantiated in a JUnit test. Everything can be done in Surefire now,
>> and the Failsafe plugin isn't needed for these.  Don't believe me -  
>> watch
>> the video in the blog entry above, it's not long.
>>
>> New problem. I want unit tests to run in this order:
>>
>> 1. unit
>> 2. integration (may invoke service calls headlessly)
>> 3. function (will use WebDriver)
>>
>>
>> I can't work out what magic I have to do with executions to allow that  
>> to
>> happen.
>>
>> Here is how far I got:
>>
>>
>> https://github.com/paul-hammant/todobackend-jooby/blob/master/pom.xml#L74
>>
>> It is all a bit second class, because I'd have to do ...
>>
>> mvn clean test -Punit-tests
>> mvn test -Pintegration-tests
>> mvn test -Pfunctional-tests
>>
>> ... to simulate a pipeline, and I would be happy to just rely on
>> annotations for classifications.
>>
>> I really want to do ...
>>
>> mvn clean test -DexecutionOrder=unit,integration,functional
>> -DstopBuildAtExecutionBoundariesForTestFailures
>>
>>
>> ... and shave seconds off the build.
>>
>> How do I configure that tersely and elegantly in the surefire plugin  
>> today?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Configuring the surefire plugin - a question

Paul Hammant
Thanks João, thanks Robert.

I've taken Robert's snippet and expanded it a little to do what I want:

    Diff:
https://github.com/paul-hammant/todobackend-jooby/commit/9626a3155eddbaea74bbf66a3e899b81227842ee
    (repo: paul-hammant/todobackend-jooby* branch: expectations*)

I found that I had to be explicit about excludes too, and have a precursor
exclude that is outside of the three executions.

I did a bunch of trial and error, but this was the minima.

The test of correctness:

  $ mvn install | grep "Time elapsed"
  Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.237 sec
- in todobackend.TodoUnitTest
  Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.51 sec
- in todobackend.TodoIntegrationTest
  Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 11.608
sec - in todobackend.TodoWebDriverTest
  (ignore some WebDriver noise to std-err)
I can't say I really understand the rules about additive includes and
excludes, but my job is to make speedy builds, not fully understand every
angle bracket of Maven.

Regards,

- Paul


On Fri, Feb 17, 2017 at 2:00 PM, Robert Scholte <[hidden email]>
wrote:

> When you only want to change the pom (not the tests), a set of
> executionblocks will do the trick:
>
>       <plugin>
>         <groupId>org.apache.maven.plugins</groupId>
>         <artifactId>maven-surefire-plugin</artifactId>
>         <version>2.18.1</version>
>         <executions>
>           <execution>
>             <id>unit-tests</id>
>             <configuration>
>             <includes>
>               <include>**/*Unit*.java</include>
>             </includes>
>             </configuration>
>           </execution>
>           <execution>
>             <id>functional-tests</id>
>             <configuration>
>             <includes>
>               <include>**/*WebDriver*.java</include>
>             </includes>
>             </configuration>
>           </execution>
>           <execution>
>             <id>integration-tests</id>
>             <configuration>
>             <includes>
>               <include>**/*Integration*.java</include>
>             </includes>
>             </configuration>
>           </execution>
>         <executions>
>       </plugin>
>
>
> On Thu, 16 Feb 2017 11:22:37 +0100, João Cabrita <[hidden email]>
> wrote:
>
> I'd say you could add executions to the surefire plugin with different
>> categories:
>> https://maven.apache.org/surefire/maven-surefire-plugin/
>> examples/junit.html#Using_JUnit_Categories
>>
>> Look at this gist (I've omitted some details):
>> https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c
>>
>> AFAIK, even if the executions are all bound to the same phase in the
>> lifecycle, they execute in the order specificied in the POM.
>> Beware this is the behavior I've observed and can't confirm it is
>> specified
>> behavior.
>>
>>
>>
>> João Cabrita
>>
>> On 16 February 2017 at 10:06, Paul Hammant <[hidden email]> wrote:
>>
>> Hi folks,.
>>>
>>> I've a fast WebDriver using build that I blogged about: A 16 Second Java
>>> Webapp Build (Including WebDriver Tests)
>>> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
>>> build-including-webdriver-tests/>
>>> .
>>>
>>> Jooby (like SpringBoot and SparkJava) give new options for testing - it
>>> can
>>> be instantiated in a JUnit test. Everything can be done in Surefire now,
>>> and the Failsafe plugin isn't needed for these.  Don't believe me - watch
>>> the video in the blog entry above, it's not long.
>>>
>>> New problem. I want unit tests to run in this order:
>>>
>>> 1. unit
>>> 2. integration (may invoke service calls headlessly)
>>> 3. function (will use WebDriver)
>>>
>>>
>>> I can't work out what magic I have to do with executions to allow that to
>>> happen.
>>>
>>> Here is how far I got:
>>>
>>>
>>> https://github.com/paul-hammant/todobackend-jooby/blob/
>>> master/pom.xml#L74
>>>
>>> It is all a bit second class, because I'd have to do ...
>>>
>>> mvn clean test -Punit-tests
>>> mvn test -Pintegration-tests
>>> mvn test -Pfunctional-tests
>>>
>>> ... to simulate a pipeline, and I would be happy to just rely on
>>> annotations for classifications.
>>>
>>> I really want to do ...
>>>
>>> mvn clean test -DexecutionOrder=unit,integration,functional
>>> -DstopBuildAtExecutionBoundariesForTestFailures
>>>
>>>
>>> ... and shave seconds off the build.
>>>
>>> How do I configure that tersely and elegantly in the surefire plugin
>>> today?
>>>
>>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Configuring the surefire plugin - a question

rfscholte
On Fri, 24 Feb 2017 04:32:17 +0100, Paul Hammant <[hidden email]> wrote:

> Thanks João, thanks Robert.
>
> I've taken Robert's snippet and expanded it a little to do what I want:
>
>     Diff:
> https://github.com/paul-hammant/todobackend-jooby/commit/9626a3155eddbaea74bbf66a3e899b81227842ee
>     (repo: paul-hammant/todobackend-jooby* branch: expectations*)
>
> I found that I had to be explicit about excludes too, and have a  
> precursor
> exclude that is outside of the three executions.
>
> I did a bunch of trial and error, but this was the minima.
>
> The test of correctness:
>
>   $ mvn install | grep "Time elapsed"
>   Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.237  
> sec
> - in todobackend.TodoUnitTest
>   Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.51  
> sec
> - in todobackend.TodoIntegrationTest
>   Tests run: 4, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 11.608
> sec - in todobackend.TodoWebDriverTest
>   (ignore some WebDriver noise to std-err)
> I can't say I really understand the rules about additive includes and
> excludes, but my job is to make speedy builds, not fully understand every
> angle bracket of Maven.

Just a small hint about the concept of includes/excludes.
A test is executed if and only if it matches any include (or all if there  
are no includes) AND doesn't match any exclude.

The problem I see is that by default you exclude every test (**/*). This  
implies that all includes are useless. By overriding the excludes per  
execution-block you kind of fixed that.

Robert

>
> Regards,
>
> - Paul
>
>
> On Fri, Feb 17, 2017 at 2:00 PM, Robert Scholte <[hidden email]>
> wrote:
>
>> When you only want to change the pom (not the tests), a set of
>> executionblocks will do the trick:
>>
>>       <plugin>
>>         <groupId>org.apache.maven.plugins</groupId>
>>         <artifactId>maven-surefire-plugin</artifactId>
>>         <version>2.18.1</version>
>>         <executions>
>>           <execution>
>>             <id>unit-tests</id>
>>             <configuration>
>>             <includes>
>>               <include>**/*Unit*.java</include>
>>             </includes>
>>             </configuration>
>>           </execution>
>>           <execution>
>>             <id>functional-tests</id>
>>             <configuration>
>>             <includes>
>>               <include>**/*WebDriver*.java</include>
>>             </includes>
>>             </configuration>
>>           </execution>
>>           <execution>
>>             <id>integration-tests</id>
>>             <configuration>
>>             <includes>
>>               <include>**/*Integration*.java</include>
>>             </includes>
>>             </configuration>
>>           </execution>
>>         <executions>
>>       </plugin>
>>
>>
>> On Thu, 16 Feb 2017 11:22:37 +0100, João Cabrita  
>> <[hidden email]>
>> wrote:
>>
>> I'd say you could add executions to the surefire plugin with different
>>> categories:
>>> https://maven.apache.org/surefire/maven-surefire-plugin/
>>> examples/junit.html#Using_JUnit_Categories
>>>
>>> Look at this gist (I've omitted some details):
>>> https://gist.github.com/kewne/2b909ab5e8035a4e44e406fa35e3276c
>>>
>>> AFAIK, even if the executions are all bound to the same phase in the
>>> lifecycle, they execute in the order specificied in the POM.
>>> Beware this is the behavior I've observed and can't confirm it is
>>> specified
>>> behavior.
>>>
>>>
>>>
>>> João Cabrita
>>>
>>> On 16 February 2017 at 10:06, Paul Hammant <[hidden email]> wrote:
>>>
>>> Hi folks,.
>>>>
>>>> I've a fast WebDriver using build that I blogged about: A 16 Second  
>>>> Java
>>>> Webapp Build (Including WebDriver Tests)
>>>> <http://paulhammant.com/2017/02/05/a-16-second-java-webapp-
>>>> build-including-webdriver-tests/>
>>>> .
>>>>
>>>> Jooby (like SpringBoot and SparkJava) give new options for testing -  
>>>> it
>>>> can
>>>> be instantiated in a JUnit test. Everything can be done in Surefire  
>>>> now,
>>>> and the Failsafe plugin isn't needed for these.  Don't believe me -  
>>>> watch
>>>> the video in the blog entry above, it's not long.
>>>>
>>>> New problem. I want unit tests to run in this order:
>>>>
>>>> 1. unit
>>>> 2. integration (may invoke service calls headlessly)
>>>> 3. function (will use WebDriver)
>>>>
>>>>
>>>> I can't work out what magic I have to do with executions to allow  
>>>> that to
>>>> happen.
>>>>
>>>> Here is how far I got:
>>>>
>>>>
>>>> https://github.com/paul-hammant/todobackend-jooby/blob/
>>>> master/pom.xml#L74
>>>>
>>>> It is all a bit second class, because I'd have to do ...
>>>>
>>>> mvn clean test -Punit-tests
>>>> mvn test -Pintegration-tests
>>>> mvn test -Pfunctional-tests
>>>>
>>>> ... to simulate a pipeline, and I would be happy to just rely on
>>>> annotations for classifications.
>>>>
>>>> I really want to do ...
>>>>
>>>> mvn clean test -DexecutionOrder=unit,integration,functional
>>>> -DstopBuildAtExecutionBoundariesForTestFailures
>>>>
>>>>
>>>> ... and shave seconds off the build.
>>>>
>>>> How do I configure that tersely and elegantly in the surefire plugin
>>>> today?
>>>>
>>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Configuring the surefire plugin - a question

Paul Hammant
I tried many combinations but couldn't make it shorter with the same behavior :(
- ph ---
Just a small hint about the concept of includes/excludes.
A test is executed if and only if it matches any include (or all if there  
are no includes) AND doesn't match any exclude.

The problem I see is that by default you exclude every test (**/*). This  
implies that all includes are useless. By overriding the excludes per  
execution-block you kind of fixed that.

Robert

.



Loading...