Next Generation Integration Testing for Plugins/Core

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Next Generation Integration Testing for Plugins/Core

Karl Heinz Marbaise-3
Hi to all,

I've invested some time to get a thing working in a different way which
nags me for a long time.

Integration tests for maven plugins and for maven core...

So created a prototype based on a JUnit Jupiter extension.

The following is the JUnit Jupiter extension (currently very hacky code;
not intended to win a competition for good code!)

https://github.com/khmarbaise/maven-it-extension

which contains some documentation which is of course not ready yet...
but the implementation and usage (see maven-ear-plugin) gives me at the
moment already a very good impression how easy it can be to write
integration tests for a maven plugin etc.

Example from the docs(not 100% working like that yet):

@MavenIT
class FirstMavenIT {

   @MavenTest
   void the_first_test_case(MavenProjectResult result) {
     assertThat(result)
       .build()
         .isSuccessful()
       .and()
       .project()
         .hasTarget()
           .withEarFile()
             .containsOnlyOnce("META-INF/MANIFEST.MF")
         .log()
           .info().contains("Writing data to file")
       .cache()
           .withEarFile("G:A:V")
           .withPomFile("G:A:V")
           .withMetadata().contains("xxx");
   }
}


I created a branch "maven-it-extension" on Maven EAR Plugin which shows
that it can be used in combination with maven-invoker-plugin:install
goal and using maven-failsafe-plugin to run the tests for
maven-ear-plugin (some of them at the moment. Not migrated all of them yet).

Example which already works:
https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java


WDYT ?

Kind regards
Karl Heinz Marbaise

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Romain Manni-Bucau
Hi Karl

Not sure id do a MavenIT annotation - test is enough probably - but i like
jupiter style.

Im less exited by assertj but it is probably a habit thing.

Wonder if you evaluated to run in a fake filesystem like jimfs or so and
enable the pom+files to be defined on the test method? Goal would be to not
split setup and asserts (given and then colocalized, then being the mvn
exec).

Hope it makes sense.

Le mar. 29 oct. 2019 à 21:47, Karl Heinz Marbaise <[hidden email]> a
écrit :

> Hi to all,
>
> I've invested some time to get a thing working in a different way which
> nags me for a long time.
>
> Integration tests for maven plugins and for maven core...
>
> So created a prototype based on a JUnit Jupiter extension.
>
> The following is the JUnit Jupiter extension (currently very hacky code;
> not intended to win a competition for good code!)
>
> https://github.com/khmarbaise/maven-it-extension
>
> which contains some documentation which is of course not ready yet...
> but the implementation and usage (see maven-ear-plugin) gives me at the
> moment already a very good impression how easy it can be to write
> integration tests for a maven plugin etc.
>
> Example from the docs(not 100% working like that yet):
>
> @MavenIT
> class FirstMavenIT {
>
>    @MavenTest
>    void the_first_test_case(MavenProjectResult result) {
>      assertThat(result)
>        .build()
>          .isSuccessful()
>        .and()
>        .project()
>          .hasTarget()
>            .withEarFile()
>              .containsOnlyOnce("META-INF/MANIFEST.MF")
>          .log()
>            .info().contains("Writing data to file")
>        .cache()
>            .withEarFile("G:A:V")
>            .withPomFile("G:A:V")
>            .withMetadata().contains("xxx");
>    }
> }
>
>
> I created a branch "maven-it-extension" on Maven EAR Plugin which shows
> that it can be used in combination with maven-invoker-plugin:install
> goal and using maven-failsafe-plugin to run the tests for
> maven-ear-plugin (some of them at the moment. Not migrated all of them
> yet).
>
> Example which already works:
>
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
>
>
> WDYT ?
>
> Kind regards
> Karl Heinz Marbaise
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Karl Heinz Marbaise-3
Hi Romain,

On 29.10.19 22:40, Romain Manni-Bucau wrote:
> Hi Karl
>
> Not sure id do a MavenIT annotation - test is enough probably - but i
> like jupiter style.

MavenIT[1] annotation contains more information like global/local cache,
the default goals which are used for the build, debugging or not (just
to mention some; I think there will be more).

Also so the MavenTest annotation is needed to define specific things for
Maven like activeProfiles etc.

[1]:
https://github.com/khmarbaise/maven-it-extension/blob/master/src/main/java/org/apache/maven/jupiter/extension/MavenIT.java

See also:

https://github.com/khmarbaise/maven-it-extension/blob/master/src/test/java/org/apache/maven/jupiter/it/MavenIntegrationIT.java


>
> Im less exited by assertj but it is probably a habit thing.

I'm not sure if I see your issue with AssertJ. Have you used it? AssertJ
brings clear expression and readable tests in general and gives me a
simple way to create custom assertions to support special parts for a
Maven build like Log file, project structure, target directory contents,
content of archive files etc. ?

like this:

  assertThat(result)
    .project()
      .hasTarget()
        .withEarFile()
           .containsOnlyOnce("META-INF/MANIFEST.MF");

I've never seen a assertion framework which makes it possible to write
tests in a more or less fluent way ...

What would be your choice?


>
> Wonder if you evaluated to run in a fake filesystem like jimfs or so and
> enable the pom+files to be defined on the test method?

This is exactly where I have decided against at the moment cause
construction of a full maven project with all it's pom file(s)
directories source code etc. is based on the things I want to solve to
complicated...we already have such things[2] also in plexus testcase you
can do such things or more easier today just use mockito ...

Currently I have gone the path to have a convention where to find the
projects which are used to be tested like maven-invoker-plugin already
does ...and can also being executed manually within the directory
without any change ...helps a lot in case error hunting ...

Can you explain the hint about fake filesystem like jimfs a little bit
more cause I don't know jimfs etc. and what you have in mind?


[2]:
https://maven.apache.org/plugin-testing/maven-plugin-testing-harness/index.html

 > > Goal would be to
> not split setup and asserts (given and then colocalized, then being the
> mvn exec).

Maybe I misunderstand that? Can you give an example ?

Kind regards
Karl Heinz Marbaise


>
> Hope it makes sense.


>
> Le mar. 29 oct. 2019 à 21:47, Karl Heinz Marbaise <[hidden email]
> <mailto:[hidden email]>> a écrit :
>
>     Hi to all,
>
>     I've invested some time to get a thing working in a different way which
>     nags me for a long time.
>
>     Integration tests for maven plugins and for maven core...
>
>     So created a prototype based on a JUnit Jupiter extension.
>
>     The following is the JUnit Jupiter extension (currently very hacky code;
>     not intended to win a competition for good code!)
>
>     https://github.com/khmarbaise/maven-it-extension
>
>     which contains some documentation which is of course not ready yet...
>     but the implementation and usage (see maven-ear-plugin) gives me at the
>     moment already a very good impression how easy it can be to write
>     integration tests for a maven plugin etc.
>
>     Example from the docs(not 100% working like that yet):
>
>     @MavenIT
>     class FirstMavenIT {
>
>         @MavenTest
>         void the_first_test_case(MavenProjectResult result) {
>           assertThat(result)
>             .build()
>               .isSuccessful()
>             .and()
>             .project()
>               .hasTarget()
>                 .withEarFile()
>                   .containsOnlyOnce("META-INF/MANIFEST.MF")
>               .log()
>                 .info().contains("Writing data to file")
>             .cache()
>                 .withEarFile("G:A:V")
>                 .withPomFile("G:A:V")
>                 .withMetadata().contains("xxx");
>         }
>     }
>
>
>     I created a branch "maven-it-extension" on Maven EAR Plugin which shows
>     that it can be used in combination with maven-invoker-plugin:install
>     goal and using maven-failsafe-plugin to run the tests for
>     maven-ear-plugin (some of them at the moment. Not migrated all of
>     them yet).
>
>     Example which already works:
>     https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
>
>
>     WDYT ?
>
>     Kind regards
>     Karl Heinz Marbaise

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Romain Manni-Bucau
Le mar. 29 oct. 2019 à 22:58, Karl Heinz Marbaise <[hidden email]> a
écrit :

> Hi Romain,
>
> On 29.10.19 22:40, Romain Manni-Bucau wrote:
> > Hi Karl
> >
> > Not sure id do a MavenIT annotation - test is enough probably - but i
> > like jupiter style.
>
> MavenIT[1] annotation contains more information like global/local cache,
> the default goals which are used for the build, debugging or not (just
> to mention some; I think there will be more).
>
> Also so the MavenTest annotation is needed to define specific things for
> Maven like activeProfiles etc.
>
> [1]:
>
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/main/java/org/apache/maven/jupiter/extension/MavenIT.java
>
> See also:
>
>
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/test/java/org/apache/maven/jupiter/it/MavenIntegrationIT.java



You can alias @MavenTest for that no? This is what i had in mind. Like
@ShadeTest decorated with @MavenTest to set defaults.

So you reduce thz number of annotations.

That said it is not a blocker.


>
>
> >
> > Im less exited by assertj but it is probably a habit thing.
>
> I'm not sure if I see your issue with AssertJ. Have you used it? AssertJ
> brings clear expression and readable tests in general and gives me a
> simple way to create custom assertions to support special parts for a
> Maven build like Log file, project structure, target directory contents,
> content of archive files etc. ?
>
> like this:
>
>   assertThat(result)
>     .project()
>       .hasTarget()
>         .withEarFile()
>            .containsOnlyOnce("META-INF/MANIFEST.MF");
>
> I've never seen a assertion framework which makes it possible to write
> tests in a more or less fluent way ...
>

I did use it a few and dropped it since it does not bring much i  most
cases.
Asserts stay simple and chaining them just hit autoformatting limitations
in general.
Also assertj had some dependency conflicts - fixed now IIRC.
So staying minimal was good.

The side note being: do you need it or can you back your dsl with jupiter
assertions? No strong need ;).


> What would be your choice?
>

Just an available model, fluent if you want, but no additional dep.
Default Assertions is enough for most cases, in particular with streams and
lambdas.


>
> >
> > Wonder if you evaluated to run in a fake filesystem like jimfs or so and
> > enable the pom+files to be defined on the test method?
>
> This is exactly where I have decided against at the moment cause
> construction of a full maven project with all it's pom file(s)
> directories source code etc. is based on the things I want to solve to
> complicated...we already have such things[2] also in plexus testcase you
> can do such things or more easier today just use mockito ...
>

Hmm, this is not comparable.
Mockito is like not writing tests, jimfs is launching a real build on a
fake in memory filesystem, no mocking for maven.


> Currently I have gone the path to have a convention where to find the
> projects which are used to be tested like maven-invoker-plugin already
> does ...and can also being executed manually within the directory
> without any change ...helps a lot in case error hunting ...
>
> Can you explain the hint about fake filesystem like jimfs a little bit
> more cause I don't know jimfs etc. and what you have in mind?
>

Jimfs is a FileSystem implementation so if maven uses java.nio.file api
then we can run on an in memory FS and configure it from any metamodel we
want.

It would be close to this sftp testing api
https://github.com/rmannibucau/rule-them-all/blob/master/src/test/java/com/github/rmannibucau/rules/test/sftp/TestSftpRule.java#L32



>
> [2]:
>
> https://maven.apache.org/plugin-testing/maven-plugin-testing-harness/index.html
>
>  > > Goal would be to
> > not split setup and asserts (given and then colocalized, then being the
> > mvn exec).
>
> Maybe I misunderstand that? Can you give an example ?
>
> Kind regards
> Karl Heinz Marbaise
>
>
> >
> > Hope it makes sense.
>
>
> >
> > Le mar. 29 oct. 2019 à 21:47, Karl Heinz Marbaise <[hidden email]
> > <mailto:[hidden email]>> a écrit :
> >
> >     Hi to all,
> >
> >     I've invested some time to get a thing working in a different way
> which
> >     nags me for a long time.
> >
> >     Integration tests for maven plugins and for maven core...
> >
> >     So created a prototype based on a JUnit Jupiter extension.
> >
> >     The following is the JUnit Jupiter extension (currently very hacky
> code;
> >     not intended to win a competition for good code!)
> >
> >     https://github.com/khmarbaise/maven-it-extension
> >
> >     which contains some documentation which is of course not ready yet...
> >     but the implementation and usage (see maven-ear-plugin) gives me at
> the
> >     moment already a very good impression how easy it can be to write
> >     integration tests for a maven plugin etc.
> >
> >     Example from the docs(not 100% working like that yet):
> >
> >     @MavenIT
> >     class FirstMavenIT {
> >
> >         @MavenTest
> >         void the_first_test_case(MavenProjectResult result) {
> >           assertThat(result)
> >             .build()
> >               .isSuccessful()
> >             .and()
> >             .project()
> >               .hasTarget()
> >                 .withEarFile()
> >                   .containsOnlyOnce("META-INF/MANIFEST.MF")
> >               .log()
> >                 .info().contains("Writing data to file")
> >             .cache()
> >                 .withEarFile("G:A:V")
> >                 .withPomFile("G:A:V")
> >                 .withMetadata().contains("xxx");
> >         }
> >     }
> >
> >
> >     I created a branch "maven-it-extension" on Maven EAR Plugin which
> shows
> >     that it can be used in combination with maven-invoker-plugin:install
> >     goal and using maven-failsafe-plugin to run the tests for
> >     maven-ear-plugin (some of them at the moment. Not migrated all of
> >     them yet).
> >
> >     Example which already works:
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> >
> >
> >     WDYT ?
> >
> >     Kind regards
> >     Karl Heinz Marbaise
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

stephenconnolly
On Tue, 29 Oct 2019 at 22:16, Romain Manni-Bucau <[hidden email]>
wrote:

> Le mar. 29 oct. 2019 à 22:58, Karl Heinz Marbaise <[hidden email]> a
> écrit :
>
> > Hi Romain,
> >
> > On 29.10.19 22:40, Romain Manni-Bucau wrote:
> > > Hi Karl
> > >
> > > Not sure id do a MavenIT annotation - test is enough probably - but i
> > > like jupiter style.
> >
> > MavenIT[1] annotation contains more information like global/local cache,
> > the default goals which are used for the build, debugging or not (just
> > to mention some; I think there will be more).
> >
> > Also so the MavenTest annotation is needed to define specific things for
> > Maven like activeProfiles etc.
> >
> > [1]:
> >
> >
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/main/java/org/apache/maven/jupiter/extension/MavenIT.java
> >
> > See also:
> >
> >
> >
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/test/java/org/apache/maven/jupiter/it/MavenIntegrationIT.java
>
>
>
> You can alias @MavenTest for that no? This is what i had in mind. Like
> @ShadeTest decorated with @MavenTest to set defaults.
>
> So you reduce thz number of annotations.
>
> That said it is not a blocker.
>
>
> >
> >
> > >
> > > Im less exited by assertj but it is probably a habit thing.
> >
> > I'm not sure if I see your issue with AssertJ. Have you used it? AssertJ
> > brings clear expression and readable tests in general and gives me a
> > simple way to create custom assertions to support special parts for a
> > Maven build like Log file, project structure, target directory contents,
> > content of archive files etc. ?
> >
> > like this:
> >
> >   assertThat(result)
> >     .project()
> >       .hasTarget()
> >         .withEarFile()
> >            .containsOnlyOnce("META-INF/MANIFEST.MF");
> >
> > I've never seen a assertion framework which makes it possible to write
> > tests in a more or less fluent way ...
> >
>
> I did use it a few and dropped it since it does not bring much i  most
> cases.
> Asserts stay simple and chaining them just hit autoformatting limitations
> in general.
> Also assertj had some dependency conflicts - fixed now IIRC.
> So staying minimal was good.
>
> The side note being: do you need it or can you back your dsl with jupiter
> assertions? No strong need ;).
>
>
> > What would be your choice?
> >
>
> Just an available model, fluent if you want, but no additional dep.
> Default Assertions is enough for most cases, in particular with streams and
> lambdas.
>
>
> >
> > >
> > > Wonder if you evaluated to run in a fake filesystem like jimfs or so
> and
> > > enable the pom+files to be defined on the test method?
> >
> > This is exactly where I have decided against at the moment cause
> > construction of a full maven project with all it's pom file(s)
> > directories source code etc. is based on the things I want to solve to
> > complicated...we already have such things[2] also in plexus testcase you
> > can do such things or more easier today just use mockito ...
> >
>
> Hmm, this is not comparable.
> Mockito is like not writing tests, jimfs is launching a real build on a
> fake in memory filesystem, no mocking for maven.
>
>
> > Currently I have gone the path to have a convention where to find the
> > projects which are used to be tested like maven-invoker-plugin already
> > does ...and can also being executed manually within the directory
> > without any change ...helps a lot in case error hunting ...
> >
> > Can you explain the hint about fake filesystem like jimfs a little bit
> > more cause I don't know jimfs etc. and what you have in mind?
> >
>
> Jimfs is a FileSystem implementation so if maven uses java.nio.file api
> then we can run on an in memory FS and configure it from any metamodel we
> want.
>
> It would be close to this sftp testing api
>
> https://github.com/rmannibucau/rule-them-all/blob/master/src/test/java/com/github/rmannibucau/rules/test/sftp/TestSftpRule.java#L32


That would prevent the tests from running Maven in a different JVM... Not
sure if the JUnit extension supports that, but knowing the libraries likely
used I suspect it does


>
>
>
> >
> > [2]:
> >
> >
> https://maven.apache.org/plugin-testing/maven-plugin-testing-harness/index.html
> >
> >  > > Goal would be to
> > > not split setup and asserts (given and then colocalized, then being the
> > > mvn exec).
> >
> > Maybe I misunderstand that? Can you give an example ?
> >
> > Kind regards
> > Karl Heinz Marbaise
> >
> >
> > >
> > > Hope it makes sense.
> >
> >
> > >
> > > Le mar. 29 oct. 2019 à 21:47, Karl Heinz Marbaise <[hidden email]
> > > <mailto:[hidden email]>> a écrit :
> > >
> > >     Hi to all,
> > >
> > >     I've invested some time to get a thing working in a different way
> > which
> > >     nags me for a long time.
> > >
> > >     Integration tests for maven plugins and for maven core...
> > >
> > >     So created a prototype based on a JUnit Jupiter extension.
> > >
> > >     The following is the JUnit Jupiter extension (currently very hacky
> > code;
> > >     not intended to win a competition for good code!)
> > >
> > >     https://github.com/khmarbaise/maven-it-extension
> > >
> > >     which contains some documentation which is of course not ready
> yet...
> > >     but the implementation and usage (see maven-ear-plugin) gives me at
> > the
> > >     moment already a very good impression how easy it can be to write
> > >     integration tests for a maven plugin etc.
> > >
> > >     Example from the docs(not 100% working like that yet):
> > >
> > >     @MavenIT
> > >     class FirstMavenIT {
> > >
> > >         @MavenTest
> > >         void the_first_test_case(MavenProjectResult result) {
> > >           assertThat(result)
> > >             .build()
> > >               .isSuccessful()
> > >             .and()
> > >             .project()
> > >               .hasTarget()
> > >                 .withEarFile()
> > >                   .containsOnlyOnce("META-INF/MANIFEST.MF")
> > >               .log()
> > >                 .info().contains("Writing data to file")
> > >             .cache()
> > >                 .withEarFile("G:A:V")
> > >                 .withPomFile("G:A:V")
> > >                 .withMetadata().contains("xxx");
> > >         }
> > >     }
> > >
> > >
> > >     I created a branch "maven-it-extension" on Maven EAR Plugin which
> > shows
> > >     that it can be used in combination with
> maven-invoker-plugin:install
> > >     goal and using maven-failsafe-plugin to run the tests for
> > >     maven-ear-plugin (some of them at the moment. Not migrated all of
> > >     them yet).
> > >
> > >     Example which already works:
> > >
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> > >
> > >
> > >     WDYT ?
> > >
> > >     Kind regards
> > >     Karl Heinz Marbaise
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Romain Manni-Bucau
Good point so guess it can be combined with a config (more a system
properties) to run in memory or forked and therefore the fs can be either
in mem or just populated from the spec (annotations).
Was looking to get something more fluent on the full setup than matching
multiple resources which is - at least for me - a but bothering and looks
old school.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mer. 30 oct. 2019 à 15:23, Stephen Connolly <
[hidden email]> a écrit :

> On Tue, 29 Oct 2019 at 22:16, Romain Manni-Bucau <[hidden email]>
> wrote:
>
> > Le mar. 29 oct. 2019 à 22:58, Karl Heinz Marbaise <[hidden email]> a
> > écrit :
> >
> > > Hi Romain,
> > >
> > > On 29.10.19 22:40, Romain Manni-Bucau wrote:
> > > > Hi Karl
> > > >
> > > > Not sure id do a MavenIT annotation - test is enough probably - but i
> > > > like jupiter style.
> > >
> > > MavenIT[1] annotation contains more information like global/local
> cache,
> > > the default goals which are used for the build, debugging or not (just
> > > to mention some; I think there will be more).
> > >
> > > Also so the MavenTest annotation is needed to define specific things
> for
> > > Maven like activeProfiles etc.
> > >
> > > [1]:
> > >
> > >
> >
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/main/java/org/apache/maven/jupiter/extension/MavenIT.java
> > >
> > > See also:
> > >
> > >
> > >
> >
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/test/java/org/apache/maven/jupiter/it/MavenIntegrationIT.java
> >
> >
> >
> > You can alias @MavenTest for that no? This is what i had in mind. Like
> > @ShadeTest decorated with @MavenTest to set defaults.
> >
> > So you reduce thz number of annotations.
> >
> > That said it is not a blocker.
> >
> >
> > >
> > >
> > > >
> > > > Im less exited by assertj but it is probably a habit thing.
> > >
> > > I'm not sure if I see your issue with AssertJ. Have you used it?
> AssertJ
> > > brings clear expression and readable tests in general and gives me a
> > > simple way to create custom assertions to support special parts for a
> > > Maven build like Log file, project structure, target directory
> contents,
> > > content of archive files etc. ?
> > >
> > > like this:
> > >
> > >   assertThat(result)
> > >     .project()
> > >       .hasTarget()
> > >         .withEarFile()
> > >            .containsOnlyOnce("META-INF/MANIFEST.MF");
> > >
> > > I've never seen a assertion framework which makes it possible to write
> > > tests in a more or less fluent way ...
> > >
> >
> > I did use it a few and dropped it since it does not bring much i  most
> > cases.
> > Asserts stay simple and chaining them just hit autoformatting limitations
> > in general.
> > Also assertj had some dependency conflicts - fixed now IIRC.
> > So staying minimal was good.
> >
> > The side note being: do you need it or can you back your dsl with jupiter
> > assertions? No strong need ;).
> >
> >
> > > What would be your choice?
> > >
> >
> > Just an available model, fluent if you want, but no additional dep.
> > Default Assertions is enough for most cases, in particular with streams
> and
> > lambdas.
> >
> >
> > >
> > > >
> > > > Wonder if you evaluated to run in a fake filesystem like jimfs or so
> > and
> > > > enable the pom+files to be defined on the test method?
> > >
> > > This is exactly where I have decided against at the moment cause
> > > construction of a full maven project with all it's pom file(s)
> > > directories source code etc. is based on the things I want to solve to
> > > complicated...we already have such things[2] also in plexus testcase
> you
> > > can do such things or more easier today just use mockito ...
> > >
> >
> > Hmm, this is not comparable.
> > Mockito is like not writing tests, jimfs is launching a real build on a
> > fake in memory filesystem, no mocking for maven.
> >
> >
> > > Currently I have gone the path to have a convention where to find the
> > > projects which are used to be tested like maven-invoker-plugin already
> > > does ...and can also being executed manually within the directory
> > > without any change ...helps a lot in case error hunting ...
> > >
> > > Can you explain the hint about fake filesystem like jimfs a little bit
> > > more cause I don't know jimfs etc. and what you have in mind?
> > >
> >
> > Jimfs is a FileSystem implementation so if maven uses java.nio.file api
> > then we can run on an in memory FS and configure it from any metamodel we
> > want.
> >
> > It would be close to this sftp testing api
> >
> >
> https://github.com/rmannibucau/rule-them-all/blob/master/src/test/java/com/github/rmannibucau/rules/test/sftp/TestSftpRule.java#L32
>
>
> That would prevent the tests from running Maven in a different JVM... Not
> sure if the JUnit extension supports that, but knowing the libraries likely
> used I suspect it does
>
>
> >
> >
> >
> > >
> > > [2]:
> > >
> > >
> >
> https://maven.apache.org/plugin-testing/maven-plugin-testing-harness/index.html
> > >
> > >  > > Goal would be to
> > > > not split setup and asserts (given and then colocalized, then being
> the
> > > > mvn exec).
> > >
> > > Maybe I misunderstand that? Can you give an example ?
> > >
> > > Kind regards
> > > Karl Heinz Marbaise
> > >
> > >
> > > >
> > > > Hope it makes sense.
> > >
> > >
> > > >
> > > > Le mar. 29 oct. 2019 à 21:47, Karl Heinz Marbaise <[hidden email]
> > > > <mailto:[hidden email]>> a écrit :
> > > >
> > > >     Hi to all,
> > > >
> > > >     I've invested some time to get a thing working in a different way
> > > which
> > > >     nags me for a long time.
> > > >
> > > >     Integration tests for maven plugins and for maven core...
> > > >
> > > >     So created a prototype based on a JUnit Jupiter extension.
> > > >
> > > >     The following is the JUnit Jupiter extension (currently very
> hacky
> > > code;
> > > >     not intended to win a competition for good code!)
> > > >
> > > >     https://github.com/khmarbaise/maven-it-extension
> > > >
> > > >     which contains some documentation which is of course not ready
> > yet...
> > > >     but the implementation and usage (see maven-ear-plugin) gives me
> at
> > > the
> > > >     moment already a very good impression how easy it can be to write
> > > >     integration tests for a maven plugin etc.
> > > >
> > > >     Example from the docs(not 100% working like that yet):
> > > >
> > > >     @MavenIT
> > > >     class FirstMavenIT {
> > > >
> > > >         @MavenTest
> > > >         void the_first_test_case(MavenProjectResult result) {
> > > >           assertThat(result)
> > > >             .build()
> > > >               .isSuccessful()
> > > >             .and()
> > > >             .project()
> > > >               .hasTarget()
> > > >                 .withEarFile()
> > > >                   .containsOnlyOnce("META-INF/MANIFEST.MF")
> > > >               .log()
> > > >                 .info().contains("Writing data to file")
> > > >             .cache()
> > > >                 .withEarFile("G:A:V")
> > > >                 .withPomFile("G:A:V")
> > > >                 .withMetadata().contains("xxx");
> > > >         }
> > > >     }
> > > >
> > > >
> > > >     I created a branch "maven-it-extension" on Maven EAR Plugin which
> > > shows
> > > >     that it can be used in combination with
> > maven-invoker-plugin:install
> > > >     goal and using maven-failsafe-plugin to run the tests for
> > > >     maven-ear-plugin (some of them at the moment. Not migrated all of
> > > >     them yet).
> > > >
> > > >     Example which already works:
> > > >
> > >
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> > > >
> > > >
> > > >     WDYT ?
> > > >
> > > >     Kind regards
> > > >     Karl Heinz Marbaise
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Tibor Digana
In reply to this post by Karl Heinz Marbaise-3
Karl, where you define CLI command in each test?
Regarding the f/w you have selected. If I had to decide between JUnit5 or
Groovy/Spock, I would decide for Spock.

On Tue, Oct 29, 2019 at 9:47 PM Karl Heinz Marbaise <[hidden email]>
wrote:

> Hi to all,
>
> I've invested some time to get a thing working in a different way which
> nags me for a long time.
>
> Integration tests for maven plugins and for maven core...
>
> So created a prototype based on a JUnit Jupiter extension.
>
> The following is the JUnit Jupiter extension (currently very hacky code;
> not intended to win a competition for good code!)
>
> https://github.com/khmarbaise/maven-it-extension
>
> which contains some documentation which is of course not ready yet...
> but the implementation and usage (see maven-ear-plugin) gives me at the
> moment already a very good impression how easy it can be to write
> integration tests for a maven plugin etc.
>
> Example from the docs(not 100% working like that yet):
>
> @MavenIT
> class FirstMavenIT {
>
>    @MavenTest
>    void the_first_test_case(MavenProjectResult result) {
>      assertThat(result)
>        .build()
>          .isSuccessful()
>        .and()
>        .project()
>          .hasTarget()
>            .withEarFile()
>              .containsOnlyOnce("META-INF/MANIFEST.MF")
>          .log()
>            .info().contains("Writing data to file")
>        .cache()
>            .withEarFile("G:A:V")
>            .withPomFile("G:A:V")
>            .withMetadata().contains("xxx");
>    }
> }
>
>
> I created a branch "maven-it-extension" on Maven EAR Plugin which shows
> that it can be used in combination with maven-invoker-plugin:install
> goal and using maven-failsafe-plugin to run the tests for
> maven-ear-plugin (some of them at the moment. Not migrated all of them
> yet).
>
> Example which already works:
>
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
>
>
> WDYT ?
>
> Kind regards
> Karl Heinz Marbaise
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Romain Manni-Bucau
@Tibor: one goal we should focus on on any new feature is to enable us to
attract more new contributors, spock has the disadvantage to not be
mainstream at all + to be on groovy which has some issues to support recent
java version so it will not help it to be more adopted, therefore I guess
jupiter is not really a question these days whatever we can think about it.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mer. 30 oct. 2019 à 16:16, Tibor Digana <[hidden email]> a
écrit :

> Karl, where you define CLI command in each test?
> Regarding the f/w you have selected. If I had to decide between JUnit5 or
> Groovy/Spock, I would decide for Spock.
>
> On Tue, Oct 29, 2019 at 9:47 PM Karl Heinz Marbaise <[hidden email]>
> wrote:
>
> > Hi to all,
> >
> > I've invested some time to get a thing working in a different way which
> > nags me for a long time.
> >
> > Integration tests for maven plugins and for maven core...
> >
> > So created a prototype based on a JUnit Jupiter extension.
> >
> > The following is the JUnit Jupiter extension (currently very hacky code;
> > not intended to win a competition for good code!)
> >
> > https://github.com/khmarbaise/maven-it-extension
> >
> > which contains some documentation which is of course not ready yet...
> > but the implementation and usage (see maven-ear-plugin) gives me at the
> > moment already a very good impression how easy it can be to write
> > integration tests for a maven plugin etc.
> >
> > Example from the docs(not 100% working like that yet):
> >
> > @MavenIT
> > class FirstMavenIT {
> >
> >    @MavenTest
> >    void the_first_test_case(MavenProjectResult result) {
> >      assertThat(result)
> >        .build()
> >          .isSuccessful()
> >        .and()
> >        .project()
> >          .hasTarget()
> >            .withEarFile()
> >              .containsOnlyOnce("META-INF/MANIFEST.MF")
> >          .log()
> >            .info().contains("Writing data to file")
> >        .cache()
> >            .withEarFile("G:A:V")
> >            .withPomFile("G:A:V")
> >            .withMetadata().contains("xxx");
> >    }
> > }
> >
> >
> > I created a branch "maven-it-extension" on Maven EAR Plugin which shows
> > that it can be used in combination with maven-invoker-plugin:install
> > goal and using maven-failsafe-plugin to run the tests for
> > maven-ear-plugin (some of them at the moment. Not migrated all of them
> > yet).
> >
> > Example which already works:
> >
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> >
> >
> > WDYT ?
> >
> > Kind regards
> > Karl Heinz Marbaise
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Tibor Digana
Romain, I am glad that you are with me.
Attracting the contributors!

I hope we all voted for Java 8 sources in Maven Core.
And Spock is the same story.

Java is the like C++ old style.
Lambda makes this language more moderns a bit.

Regarding issues with Java 14, all can be fixed, just give the Spock the
chance!



On Wed, Oct 30, 2019 at 4:52 PM Romain Manni-Bucau <[hidden email]>
wrote:

> @Tibor: one goal we should focus on on any new feature is to enable us to
> attract more new contributors, spock has the disadvantage to not be
> mainstream at all + to be on groovy which has some issues to support recent
> java version so it will not help it to be more adopted, therefore I guess
> jupiter is not really a question these days whatever we can think about it.
>
> Romain Manni-Bucau
> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> <https://rmannibucau.metawerx.net/> | Old Blog
> <http://rmannibucau.wordpress.com> | Github <
> https://github.com/rmannibucau> |
> LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> <
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> >
>
>
> Le mer. 30 oct. 2019 à 16:16, Tibor Digana <[hidden email]> a
> écrit :
>
> > Karl, where you define CLI command in each test?
> > Regarding the f/w you have selected. If I had to decide between JUnit5 or
> > Groovy/Spock, I would decide for Spock.
> >
> > On Tue, Oct 29, 2019 at 9:47 PM Karl Heinz Marbaise <[hidden email]>
> > wrote:
> >
> > > Hi to all,
> > >
> > > I've invested some time to get a thing working in a different way which
> > > nags me for a long time.
> > >
> > > Integration tests for maven plugins and for maven core...
> > >
> > > So created a prototype based on a JUnit Jupiter extension.
> > >
> > > The following is the JUnit Jupiter extension (currently very hacky
> code;
> > > not intended to win a competition for good code!)
> > >
> > > https://github.com/khmarbaise/maven-it-extension
> > >
> > > which contains some documentation which is of course not ready yet...
> > > but the implementation and usage (see maven-ear-plugin) gives me at the
> > > moment already a very good impression how easy it can be to write
> > > integration tests for a maven plugin etc.
> > >
> > > Example from the docs(not 100% working like that yet):
> > >
> > > @MavenIT
> > > class FirstMavenIT {
> > >
> > >    @MavenTest
> > >    void the_first_test_case(MavenProjectResult result) {
> > >      assertThat(result)
> > >        .build()
> > >          .isSuccessful()
> > >        .and()
> > >        .project()
> > >          .hasTarget()
> > >            .withEarFile()
> > >              .containsOnlyOnce("META-INF/MANIFEST.MF")
> > >          .log()
> > >            .info().contains("Writing data to file")
> > >        .cache()
> > >            .withEarFile("G:A:V")
> > >            .withPomFile("G:A:V")
> > >            .withMetadata().contains("xxx");
> > >    }
> > > }
> > >
> > >
> > > I created a branch "maven-it-extension" on Maven EAR Plugin which shows
> > > that it can be used in combination with maven-invoker-plugin:install
> > > goal and using maven-failsafe-plugin to run the tests for
> > > maven-ear-plugin (some of them at the moment. Not migrated all of them
> > > yet).
> > >
> > > Example which already works:
> > >
> > >
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> > >
> > >
> > > WDYT ?
> > >
> > > Kind regards
> > > Karl Heinz Marbaise
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: [hidden email]
> > > For additional commands, e-mail: [hidden email]
> > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Tibor Digana
Romain, the Java has not made any significant progress in language after
Java 9. Yes, some JVM features in J13 were really great but not in language.
All fixes about switch-case in several versions, strings, preliminary
feature. Nothing very progressive for developers! And I think the frequent
releases/6months were too big hammer.
The Java should think about these frequent releases because f/w do not
adapt fast and the developer has no benefit from dropped or preliminary
features.
For instance for me J13 has sense oly because of JVM feature JEP351 but not
because of language and all these dances around ASM versions are just
because of too frequent Java release with minimum value.

IMO these developers in Maven can use multiple languages and this will be
then attractive project for the contributors.
Here Karl developed some new f/w and it will take some time to complete it
and use it in the Maven ITs. Until then the Spock will change and Java as
well.
So he can decide what to use and maybe he will do it so well that JUnit5
and Spock can be used together.

On Wed, Oct 30, 2019 at 5:03 PM Romain Manni-Bucau <[hidden email]>
wrote:

> @Tibor: do you agree we write the tests with chai (js)? It is the same to
> use groovy for a java dev today since java caught up its lateness. Not
> stacking layers and avoiding useless abstractions is the best way to enable
> people to contribute from my experience. As soon as you add a layer which
> has a step people can find blockers to go upon it will not work (to be
> explicit here you add a language blocker + an API blocker + a change in
> native maven conventions which is likely bad for us).
> Lastly, my point about the java issues is not that it can't be fixed,
> people there are good enough to fix them all releases after releases, it is
> just that it delays the availability and adoption of spock which means it
> is not mainstream so contributor friendly.
> So picking spock woudnt hit our goal.
>
> Le mer. 30 oct. 2019 à 16:58, Tibor Digana <[hidden email]> a
> écrit :
>
> > Romain, I am glad that you are with me.
> > Attracting the contributors!
> >
> > I hope we all voted for Java 8 sources in Maven Core.
> > And Spock is the same story.
> >
> > Java is the like C++ old style.
> > Lambda makes this language more moderns a bit.
> >
> > Regarding issues with Java 14, all can be fixed, just give the Spock the
> > chance!
> >
> >
> >
> > On Wed, Oct 30, 2019 at 4:52 PM Romain Manni-Bucau <
> [hidden email]>
> > wrote:
> >
> > > @Tibor: one goal we should focus on on any new feature is to enable us
> to
> > > attract more new contributors, spock has the disadvantage to not be
> > > mainstream at all + to be on groovy which has some issues to support
> > recent
> > > java version so it will not help it to be more adopted, therefore I
> guess
> > > jupiter is not really a question these days whatever we can think about
> > it.
> > >
> > > Romain Manni-Bucau
> > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > <http://rmannibucau.wordpress.com> | Github <
> > > https://github.com/rmannibucau> |
> > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > <
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > >
> > >
> > >
> > > Le mer. 30 oct. 2019 à 16:16, Tibor Digana <[hidden email]> a
> > > écrit :
> > >
> > > > Karl, where you define CLI command in each test?
> > > > Regarding the f/w you have selected. If I had to decide between
> JUnit5
> > or
> > > > Groovy/Spock, I would decide for Spock.
> > > >
> > > > On Tue, Oct 29, 2019 at 9:47 PM Karl Heinz Marbaise <
> [hidden email]
> > >
> > > > wrote:
> > > >
> > > > > Hi to all,
> > > > >
> > > > > I've invested some time to get a thing working in a different way
> > which
> > > > > nags me for a long time.
> > > > >
> > > > > Integration tests for maven plugins and for maven core...
> > > > >
> > > > > So created a prototype based on a JUnit Jupiter extension.
> > > > >
> > > > > The following is the JUnit Jupiter extension (currently very hacky
> > > code;
> > > > > not intended to win a competition for good code!)
> > > > >
> > > > > https://github.com/khmarbaise/maven-it-extension
> > > > >
> > > > > which contains some documentation which is of course not ready
> yet...
> > > > > but the implementation and usage (see maven-ear-plugin) gives me at
> > the
> > > > > moment already a very good impression how easy it can be to write
> > > > > integration tests for a maven plugin etc.
> > > > >
> > > > > Example from the docs(not 100% working like that yet):
> > > > >
> > > > > @MavenIT
> > > > > class FirstMavenIT {
> > > > >
> > > > >    @MavenTest
> > > > >    void the_first_test_case(MavenProjectResult result) {
> > > > >      assertThat(result)
> > > > >        .build()
> > > > >          .isSuccessful()
> > > > >        .and()
> > > > >        .project()
> > > > >          .hasTarget()
> > > > >            .withEarFile()
> > > > >              .containsOnlyOnce("META-INF/MANIFEST.MF")
> > > > >          .log()
> > > > >            .info().contains("Writing data to file")
> > > > >        .cache()
> > > > >            .withEarFile("G:A:V")
> > > > >            .withPomFile("G:A:V")
> > > > >            .withMetadata().contains("xxx");
> > > > >    }
> > > > > }
> > > > >
> > > > >
> > > > > I created a branch "maven-it-extension" on Maven EAR Plugin which
> > shows
> > > > > that it can be used in combination with
> maven-invoker-plugin:install
> > > > > goal and using maven-failsafe-plugin to run the tests for
> > > > > maven-ear-plugin (some of them at the moment. Not migrated all of
> > them
> > > > > yet).
> > > > >
> > > > > Example which already works:
> > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> > > > >
> > > > >
> > > > > WDYT ?
> > > > >
> > > > > Kind regards
> > > > > Karl Heinz Marbaise
> > > > >
> > > > >
> ---------------------------------------------------------------------
> > > > > To unsubscribe, e-mail: [hidden email]
> > > > > For additional commands, e-mail: [hidden email]
> > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Romain Manni-Bucau
Not sure I understand it well Tibor, do you encourage to multiply the
number of potential solutions to request more time to contributor to see
how to do things?
Don't think it is good, a single simple solution sounds more promising -
once again from my past experience.

Once again Tibor, all I'm writing is about current state and my experience,
I don't want to block anything but I don't want we do choices on some
"potential" gain and not "validable" hypothesis, if we have a doubt we
should stay simple IMHO.

When I first wrote a maven plugin I checked out the solution to test it and
there was several options, inconsistent between plugins and it was not
helping me to move forward.
Here we have the opportunity to solve that kind of issues for new
contributors and I think it is taking a great direction.
It does not enforce to use @MavenIT but it does not document how to do it 5
different ways which is good.

That said I guess once we have it feature complete we can just do a vote
with a "majority wins" rules and it would be good enough for such a
solution, we don't need to all fully agree on such low level topic I think.

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mer. 30 oct. 2019 à 17:26, Tibor Digana <[hidden email]> a
écrit :

> Romain, the Java has not made any significant progress in language after
> Java 9. Yes, some JVM features in J13 were really great but not in
> language.
> All fixes about switch-case in several versions, strings, preliminary
> feature. Nothing very progressive for developers! And I think the frequent
> releases/6months were too big hammer.
> The Java should think about these frequent releases because f/w do not
> adapt fast and the developer has no benefit from dropped or preliminary
> features.
> For instance for me J13 has sense oly because of JVM feature JEP351 but not
> because of language and all these dances around ASM versions are just
> because of too frequent Java release with minimum value.
>
> IMO these developers in Maven can use multiple languages and this will be
> then attractive project for the contributors.
> Here Karl developed some new f/w and it will take some time to complete it
> and use it in the Maven ITs. Until then the Spock will change and Java as
> well.
> So he can decide what to use and maybe he will do it so well that JUnit5
> and Spock can be used together.
>
> On Wed, Oct 30, 2019 at 5:03 PM Romain Manni-Bucau <[hidden email]>
> wrote:
>
> > @Tibor: do you agree we write the tests with chai (js)? It is the same to
> > use groovy for a java dev today since java caught up its lateness. Not
> > stacking layers and avoiding useless abstractions is the best way to
> enable
> > people to contribute from my experience. As soon as you add a layer which
> > has a step people can find blockers to go upon it will not work (to be
> > explicit here you add a language blocker + an API blocker + a change in
> > native maven conventions which is likely bad for us).
> > Lastly, my point about the java issues is not that it can't be fixed,
> > people there are good enough to fix them all releases after releases, it
> is
> > just that it delays the availability and adoption of spock which means it
> > is not mainstream so contributor friendly.
> > So picking spock woudnt hit our goal.
> >
> > Le mer. 30 oct. 2019 à 16:58, Tibor Digana <[hidden email]> a
> > écrit :
> >
> > > Romain, I am glad that you are with me.
> > > Attracting the contributors!
> > >
> > > I hope we all voted for Java 8 sources in Maven Core.
> > > And Spock is the same story.
> > >
> > > Java is the like C++ old style.
> > > Lambda makes this language more moderns a bit.
> > >
> > > Regarding issues with Java 14, all can be fixed, just give the Spock
> the
> > > chance!
> > >
> > >
> > >
> > > On Wed, Oct 30, 2019 at 4:52 PM Romain Manni-Bucau <
> > [hidden email]>
> > > wrote:
> > >
> > > > @Tibor: one goal we should focus on on any new feature is to enable
> us
> > to
> > > > attract more new contributors, spock has the disadvantage to not be
> > > > mainstream at all + to be on groovy which has some issues to support
> > > recent
> > > > java version so it will not help it to be more adopted, therefore I
> > guess
> > > > jupiter is not really a question these days whatever we can think
> about
> > > it.
> > > >
> > > > Romain Manni-Bucau
> > > > @rmannibucau <https://twitter.com/rmannibucau> |  Blog
> > > > <https://rmannibucau.metawerx.net/> | Old Blog
> > > > <http://rmannibucau.wordpress.com> | Github <
> > > > https://github.com/rmannibucau> |
> > > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
> > > > <
> > > >
> > >
> >
> https://www.packtpub.com/application-development/java-ee-8-high-performance
> > > > >
> > > >
> > > >
> > > > Le mer. 30 oct. 2019 à 16:16, Tibor Digana <[hidden email]>
> a
> > > > écrit :
> > > >
> > > > > Karl, where you define CLI command in each test?
> > > > > Regarding the f/w you have selected. If I had to decide between
> > JUnit5
> > > or
> > > > > Groovy/Spock, I would decide for Spock.
> > > > >
> > > > > On Tue, Oct 29, 2019 at 9:47 PM Karl Heinz Marbaise <
> > [hidden email]
> > > >
> > > > > wrote:
> > > > >
> > > > > > Hi to all,
> > > > > >
> > > > > > I've invested some time to get a thing working in a different way
> > > which
> > > > > > nags me for a long time.
> > > > > >
> > > > > > Integration tests for maven plugins and for maven core...
> > > > > >
> > > > > > So created a prototype based on a JUnit Jupiter extension.
> > > > > >
> > > > > > The following is the JUnit Jupiter extension (currently very
> hacky
> > > > code;
> > > > > > not intended to win a competition for good code!)
> > > > > >
> > > > > > https://github.com/khmarbaise/maven-it-extension
> > > > > >
> > > > > > which contains some documentation which is of course not ready
> > yet...
> > > > > > but the implementation and usage (see maven-ear-plugin) gives me
> at
> > > the
> > > > > > moment already a very good impression how easy it can be to write
> > > > > > integration tests for a maven plugin etc.
> > > > > >
> > > > > > Example from the docs(not 100% working like that yet):
> > > > > >
> > > > > > @MavenIT
> > > > > > class FirstMavenIT {
> > > > > >
> > > > > >    @MavenTest
> > > > > >    void the_first_test_case(MavenProjectResult result) {
> > > > > >      assertThat(result)
> > > > > >        .build()
> > > > > >          .isSuccessful()
> > > > > >        .and()
> > > > > >        .project()
> > > > > >          .hasTarget()
> > > > > >            .withEarFile()
> > > > > >              .containsOnlyOnce("META-INF/MANIFEST.MF")
> > > > > >          .log()
> > > > > >            .info().contains("Writing data to file")
> > > > > >        .cache()
> > > > > >            .withEarFile("G:A:V")
> > > > > >            .withPomFile("G:A:V")
> > > > > >            .withMetadata().contains("xxx");
> > > > > >    }
> > > > > > }
> > > > > >
> > > > > >
> > > > > > I created a branch "maven-it-extension" on Maven EAR Plugin which
> > > shows
> > > > > > that it can be used in combination with
> > maven-invoker-plugin:install
> > > > > > goal and using maven-failsafe-plugin to run the tests for
> > > > > > maven-ear-plugin (some of them at the moment. Not migrated all of
> > > them
> > > > > > yet).
> > > > > >
> > > > > > Example which already works:
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> > > > > >
> > > > > >
> > > > > > WDYT ?
> > > > > >
> > > > > > Kind regards
> > > > > > Karl Heinz Marbaise
> > > > > >
> > > > > >
> > ---------------------------------------------------------------------
> > > > > > To unsubscribe, e-mail: [hidden email]
> > > > > > For additional commands, e-mail: [hidden email]
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Karl Heinz Marbaise-3
In reply to this post by Tibor Digana
Hi,

let me conclude some of the things together:

The decision which I have made against Spock was based on several
reasons:

  * People often tend to write Java code (which is valid), cause
    they don't know Groovy or don't want to learn a new language
    just to write tests.
    This means in the end: Why Groovy? Just use Java.

  * It's much easier for new contributors/devs to get into the
    project if you only need to know Java to write plugins, unit
    tests(also using JUnit5) and integration tests. So removing
    a supplemental hurdle will help.

  * Support for most recent Java versions which is a complete
    blocker for the Apache Maven project, cause we are running builds
    in a very early stage which would block us (see our builds).
    Currently spock is not yet tested/build against JDK11+ ?
    So having a Testing framework which might not work on most
    recent versions is a complete blocker.

  * In earlier days I would have argued to use Spock based
    on the language features but since JDK8 I don't see any advantage
    in using Groovy over Java anymore.

  * Spock does not support parallelizing of tests (full blocker for me)

  * Good IDE Support for Groovy is only given at the moment in
    IDEA IntelliJ and DSL support for Spock also.

    That would block many people. This blocker based on the usage
    of a particular IDE is not acceptable for an open source project
    like the Apache Maven Project and with my PMC hat on I would never
    do that.

The command line for executing the tests is at the moment here:
https://github.com/khmarbaise/maven-it-extension/blob/c0823c2afa56b29df1c09f38fe5d73ebcbe4eec3/src/main/java/org/apache/maven/jupiter/extension/ApplicationExecutor.java#L64

which is called from here:
https://github.com/khmarbaise/maven-it-extension/blob/c0823c2afa56b29df1c09f38fe5d73ebcbe4eec3/src/main/java/org/apache/maven/jupiter/extension/MavenITExtension.java#L179


Kind regards
Karl Heinz Marbaise

On 30.10.19 16:15, Tibor Digana wrote:

> Karl, where you define CLI command in each test?
> Regarding the f/w you have selected. If I had to decide between JUnit5 or
> Groovy/Spock, I would decide for Spock.
>
> On Tue, Oct 29, 2019 at 9:47 PM Karl Heinz Marbaise <[hidden email]>
> wrote:
>
>> Hi to all,
>>
>> I've invested some time to get a thing working in a different way which
>> nags me for a long time.
>>
>> Integration tests for maven plugins and for maven core...
>>
>> So created a prototype based on a JUnit Jupiter extension.
>>
>> The following is the JUnit Jupiter extension (currently very hacky code;
>> not intended to win a competition for good code!)
>>
>> https://github.com/khmarbaise/maven-it-extension
>>
>> which contains some documentation which is of course not ready yet...
>> but the implementation and usage (see maven-ear-plugin) gives me at the
>> moment already a very good impression how easy it can be to write
>> integration tests for a maven plugin etc.
>>
>> Example from the docs(not 100% working like that yet):
>>
>> @MavenIT
>> class FirstMavenIT {
>>
>>     @MavenTest
>>     void the_first_test_case(MavenProjectResult result) {
>>       assertThat(result)
>>         .build()
>>           .isSuccessful()
>>         .and()
>>         .project()
>>           .hasTarget()
>>             .withEarFile()
>>               .containsOnlyOnce("META-INF/MANIFEST.MF")
>>           .log()
>>             .info().contains("Writing data to file")
>>         .cache()
>>             .withEarFile("G:A:V")
>>             .withPomFile("G:A:V")
>>             .withMetadata().contains("xxx");
>>     }
>> }
>>
>>
>> I created a branch "maven-it-extension" on Maven EAR Plugin which shows
>> that it can be used in combination with maven-invoker-plugin:install
>> goal and using maven-failsafe-plugin to run the tests for
>> maven-ear-plugin (some of them at the moment. Not migrated all of them
>> yet).
>>
>> Example which already works:
>>
>> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
>>
>>
>> WDYT ?
>>
>> Kind regards
>> Karl Heinz Marbaise
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Enrico Olivelli
Karl
(Sorry for top posting)
Thank you very much for moving this forward.
In my personal experience one real blocker in contributions to Maven,
expecially plugins, is to write integration tests.
So having a nice way to create tests is very welcome.
Having a way to run tests as simple unit tests from an IDE allows us to
make a great step forward.

About Groovy vs Java: totally +1 for Java

Just my 2 cents
Enrico

Il mer 30 ott 2019, 21:17 Karl Heinz Marbaise <[hidden email]> ha
scritto:

> Hi,
>
> let me conclude some of the things together:
>
> The decision which I have made against Spock was based on several
> reasons:
>
>   * People often tend to write Java code (which is valid), cause
>     they don't know Groovy or don't want to learn a new language
>     just to write tests.
>     This means in the end: Why Groovy? Just use Java.
>
>   * It's much easier for new contributors/devs to get into the
>     project if you only need to know Java to write plugins, unit
>     tests(also using JUnit5) and integration tests. So removing
>     a supplemental hurdle will help.
>
>   * Support for most recent Java versions which is a complete
>     blocker for the Apache Maven project, cause we are running builds
>     in a very early stage which would block us (see our builds).
>     Currently spock is not yet tested/build against JDK11+ ?
>     So having a Testing framework which might not work on most
>     recent versions is a complete blocker.
>
>   * In earlier days I would have argued to use Spock based
>     on the language features but since JDK8 I don't see any advantage
>     in using Groovy over Java anymore.
>
>   * Spock does not support parallelizing of tests (full blocker for me)
>
>   * Good IDE Support for Groovy is only given at the moment in
>     IDEA IntelliJ and DSL support for Spock also.
>
>     That would block many people. This blocker based on the usage
>     of a particular IDE is not acceptable for an open source project
>     like the Apache Maven Project and with my PMC hat on I would never
>     do that.
>
> The command line for executing the tests is at the moment here:
>
> https://github.com/khmarbaise/maven-it-extension/blob/c0823c2afa56b29df1c09f38fe5d73ebcbe4eec3/src/main/java/org/apache/maven/jupiter/extension/ApplicationExecutor.java#L64
>
> which is called from here:
>
> https://github.com/khmarbaise/maven-it-extension/blob/c0823c2afa56b29df1c09f38fe5d73ebcbe4eec3/src/main/java/org/apache/maven/jupiter/extension/MavenITExtension.java#L179
>
>
> Kind regards
> Karl Heinz Marbaise
>
> On 30.10.19 16:15, Tibor Digana wrote:
> > Karl, where you define CLI command in each test?
> > Regarding the f/w you have selected. If I had to decide between JUnit5 or
> > Groovy/Spock, I would decide for Spock.
> >
> > On Tue, Oct 29, 2019 at 9:47 PM Karl Heinz Marbaise <[hidden email]>
> > wrote:
> >
> >> Hi to all,
> >>
> >> I've invested some time to get a thing working in a different way which
> >> nags me for a long time.
> >>
> >> Integration tests for maven plugins and for maven core...
> >>
> >> So created a prototype based on a JUnit Jupiter extension.
> >>
> >> The following is the JUnit Jupiter extension (currently very hacky code;
> >> not intended to win a competition for good code!)
> >>
> >> https://github.com/khmarbaise/maven-it-extension
> >>
> >> which contains some documentation which is of course not ready yet...
> >> but the implementation and usage (see maven-ear-plugin) gives me at the
> >> moment already a very good impression how easy it can be to write
> >> integration tests for a maven plugin etc.
> >>
> >> Example from the docs(not 100% working like that yet):
> >>
> >> @MavenIT
> >> class FirstMavenIT {
> >>
> >>     @MavenTest
> >>     void the_first_test_case(MavenProjectResult result) {
> >>       assertThat(result)
> >>         .build()
> >>           .isSuccessful()
> >>         .and()
> >>         .project()
> >>           .hasTarget()
> >>             .withEarFile()
> >>               .containsOnlyOnce("META-INF/MANIFEST.MF")
> >>           .log()
> >>             .info().contains("Writing data to file")
> >>         .cache()
> >>             .withEarFile("G:A:V")
> >>             .withPomFile("G:A:V")
> >>             .withMetadata().contains("xxx");
> >>     }
> >> }
> >>
> >>
> >> I created a branch "maven-it-extension" on Maven EAR Plugin which shows
> >> that it can be used in combination with maven-invoker-plugin:install
> >> goal and using maven-failsafe-plugin to run the tests for
> >> maven-ear-plugin (some of them at the moment. Not migrated all of them
> >> yet).
> >>
> >> Example which already works:
> >>
> >>
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> >>
> >>
> >> WDYT ?
> >>
> >> Kind regards
> >> Karl Heinz Marbaise
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Paul Hammant
Integration test choices include the excellent Spock as mentioned. I've
used it and it's very solid. Two more choices include:

Another choice is Cuppa - https://github.com/cuppa-framework/cuppa
I've used this too, and it's great - no right-click-run-this-one in
Intellij though. I wish it had more forward momentum, and I'd use the heck
out of it if it did.

Also, there is a segmented use of otherwise *vanilla surefire*:
https://github.com/BuildRadiator/BuildRadiator/blob/master/radiator/pom.xml#L158
is an example.

In that pom, there's three executions of surefire. First unit tests (no
threads, no IO, no FS), Second a few service tests (RestAssured poking a
per-test stood up Jooby web app), Third are a few WebDriver tests with the
same per-test bounce of the Jooby server.

When I first made the above, I made a quick build that shows it all
running. If target/classes/ is full already "mcn install" was 16 seconds
(compile and all three of those test stages). On my old MacBookAir with the
video recorder on, that slowed to 30s. then I refactored it to multi-module
and it is longer still. I'm still pleased with the 16s version though and
roll back the repo to demo that when needed.

I'm quite sure I'll never use Failsafe again.
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Paul Hammant