Re: Next Generation Integration Testing for Plugins/Core

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Tibor Digana
well, you have use different JVMs if you expect different env vars.

On Wed, Oct 30, 2019 at 3:23 PM Stephen Connolly <
[hidden email]> wrote:

> On Tue, 29 Oct 2019 at 22:16, Romain Manni-Bucau <[hidden email]>
> wrote:
>
> > Le mar. 29 oct. 2019 à 22:58, Karl Heinz Marbaise <[hidden email]> a
> > écrit :
> >
> > > Hi Romain,
> > >
> > > On 29.10.19 22:40, Romain Manni-Bucau wrote:
> > > > Hi Karl
> > > >
> > > > Not sure id do a MavenIT annotation - test is enough probably - but i
> > > > like jupiter style.
> > >
> > > MavenIT[1] annotation contains more information like global/local
> cache,
> > > the default goals which are used for the build, debugging or not (just
> > > to mention some; I think there will be more).
> > >
> > > Also so the MavenTest annotation is needed to define specific things
> for
> > > Maven like activeProfiles etc.
> > >
> > > [1]:
> > >
> > >
> >
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/main/java/org/apache/maven/jupiter/extension/MavenIT.java
> > >
> > > See also:
> > >
> > >
> > >
> >
> https://github.com/khmarbaise/maven-it-extension/blob/master/src/test/java/org/apache/maven/jupiter/it/MavenIntegrationIT.java
> >
> >
> >
> > You can alias @MavenTest for that no? This is what i had in mind. Like
> > @ShadeTest decorated with @MavenTest to set defaults.
> >
> > So you reduce thz number of annotations.
> >
> > That said it is not a blocker.
> >
> >
> > >
> > >
> > > >
> > > > Im less exited by assertj but it is probably a habit thing.
> > >
> > > I'm not sure if I see your issue with AssertJ. Have you used it?
> AssertJ
> > > brings clear expression and readable tests in general and gives me a
> > > simple way to create custom assertions to support special parts for a
> > > Maven build like Log file, project structure, target directory
> contents,
> > > content of archive files etc. ?
> > >
> > > like this:
> > >
> > >   assertThat(result)
> > >     .project()
> > >       .hasTarget()
> > >         .withEarFile()
> > >            .containsOnlyOnce("META-INF/MANIFEST.MF");
> > >
> > > I've never seen a assertion framework which makes it possible to write
> > > tests in a more or less fluent way ...
> > >
> >
> > I did use it a few and dropped it since it does not bring much i  most
> > cases.
> > Asserts stay simple and chaining them just hit autoformatting limitations
> > in general.
> > Also assertj had some dependency conflicts - fixed now IIRC.
> > So staying minimal was good.
> >
> > The side note being: do you need it or can you back your dsl with jupiter
> > assertions? No strong need ;).
> >
> >
> > > What would be your choice?
> > >
> >
> > Just an available model, fluent if you want, but no additional dep.
> > Default Assertions is enough for most cases, in particular with streams
> and
> > lambdas.
> >
> >
> > >
> > > >
> > > > Wonder if you evaluated to run in a fake filesystem like jimfs or so
> > and
> > > > enable the pom+files to be defined on the test method?
> > >
> > > This is exactly where I have decided against at the moment cause
> > > construction of a full maven project with all it's pom file(s)
> > > directories source code etc. is based on the things I want to solve to
> > > complicated...we already have such things[2] also in plexus testcase
> you
> > > can do such things or more easier today just use mockito ...
> > >
> >
> > Hmm, this is not comparable.
> > Mockito is like not writing tests, jimfs is launching a real build on a
> > fake in memory filesystem, no mocking for maven.
> >
> >
> > > Currently I have gone the path to have a convention where to find the
> > > projects which are used to be tested like maven-invoker-plugin already
> > > does ...and can also being executed manually within the directory
> > > without any change ...helps a lot in case error hunting ...
> > >
> > > Can you explain the hint about fake filesystem like jimfs a little bit
> > > more cause I don't know jimfs etc. and what you have in mind?
> > >
> >
> > Jimfs is a FileSystem implementation so if maven uses java.nio.file api
> > then we can run on an in memory FS and configure it from any metamodel we
> > want.
> >
> > It would be close to this sftp testing api
> >
> >
> https://github.com/rmannibucau/rule-them-all/blob/master/src/test/java/com/github/rmannibucau/rules/test/sftp/TestSftpRule.java#L32
>
>
> That would prevent the tests from running Maven in a different JVM... Not
> sure if the JUnit extension supports that, but knowing the libraries likely
> used I suspect it does
>
>
> >
> >
> >
> > >
> > > [2]:
> > >
> > >
> >
> https://maven.apache.org/plugin-testing/maven-plugin-testing-harness/index.html
> > >
> > >  > > Goal would be to
> > > > not split setup and asserts (given and then colocalized, then being
> the
> > > > mvn exec).
> > >
> > > Maybe I misunderstand that? Can you give an example ?
> > >
> > > Kind regards
> > > Karl Heinz Marbaise
> > >
> > >
> > > >
> > > > Hope it makes sense.
> > >
> > >
> > > >
> > > > Le mar. 29 oct. 2019 à 21:47, Karl Heinz Marbaise <[hidden email]
> > > > <mailto:[hidden email]>> a écrit :
> > > >
> > > >     Hi to all,
> > > >
> > > >     I've invested some time to get a thing working in a different way
> > > which
> > > >     nags me for a long time.
> > > >
> > > >     Integration tests for maven plugins and for maven core...
> > > >
> > > >     So created a prototype based on a JUnit Jupiter extension.
> > > >
> > > >     The following is the JUnit Jupiter extension (currently very
> hacky
> > > code;
> > > >     not intended to win a competition for good code!)
> > > >
> > > >     https://github.com/khmarbaise/maven-it-extension
> > > >
> > > >     which contains some documentation which is of course not ready
> > yet...
> > > >     but the implementation and usage (see maven-ear-plugin) gives me
> at
> > > the
> > > >     moment already a very good impression how easy it can be to write
> > > >     integration tests for a maven plugin etc.
> > > >
> > > >     Example from the docs(not 100% working like that yet):
> > > >
> > > >     @MavenIT
> > > >     class FirstMavenIT {
> > > >
> > > >         @MavenTest
> > > >         void the_first_test_case(MavenProjectResult result) {
> > > >           assertThat(result)
> > > >             .build()
> > > >               .isSuccessful()
> > > >             .and()
> > > >             .project()
> > > >               .hasTarget()
> > > >                 .withEarFile()
> > > >                   .containsOnlyOnce("META-INF/MANIFEST.MF")
> > > >               .log()
> > > >                 .info().contains("Writing data to file")
> > > >             .cache()
> > > >                 .withEarFile("G:A:V")
> > > >                 .withPomFile("G:A:V")
> > > >                 .withMetadata().contains("xxx");
> > > >         }
> > > >     }
> > > >
> > > >
> > > >     I created a branch "maven-it-extension" on Maven EAR Plugin which
> > > shows
> > > >     that it can be used in combination with
> > maven-invoker-plugin:install
> > > >     goal and using maven-failsafe-plugin to run the tests for
> > > >     maven-ear-plugin (some of them at the moment. Not migrated all of
> > > >     them yet).
> > > >
> > > >     Example which already works:
> > > >
> > >
> >
> https://github.com/apache/maven-ear-plugin/blob/maven-it-extension/src/test/java/org/apache/maven/plugins/ear/it/EARIT.java
> > > >
> > > >
> > > >     WDYT ?
> > > >
> > > >     Kind regards
> > > >     Karl Heinz Marbaise
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Karl Heinz Marbaise-3
Hi,

On 30.10.19 15:23, Stephen Connolly wrote:

> On Tue, 29 Oct 2019 at 22:16, Romain Manni-Bucau <[hidden email]>
> wrote:
>
>> Le mar. 29 oct. 2019 à 22:58, Karl Heinz Marbaise <[hidden email]> a
>> écrit :
>>
>>> Hi Romain,
>>>
>>> On 29.10.19 22:40, Romain Manni-Bucau wrote:
>>>> Hi Karl
>>>>
>>>> Not sure id do a MavenIT annotation - test is enough probably - but i
>>>> like jupiter style.
>>>
>>> MavenIT[1] annotation contains more information like global/local cache,
>>> the default goals which are used for the build, debugging or not (just
>>> to mention some; I think there will be more).
>>>
>>> Also so the MavenTest annotation is needed to define specific things for
>>> Maven like activeProfiles etc.
>>>
>>> [1]:
>>>
>>>
>> https://github.com/khmarbaise/maven-it-extension/blob/master/src/main/java/org/apache/maven/jupiter/extension/MavenIT.java
>>>
>>> See also:
>>>
>>>
>>>
>> https://github.com/khmarbaise/maven-it-extension/blob/master/src/test/java/org/apache/maven/jupiter/it/MavenIntegrationIT.java
>>
>>
>>
>> You can alias @MavenTest for that no? This is what i had in mind. Like
>> @ShadeTest decorated with @MavenTest to set defaults.
>>
>> So you reduce thz number of annotations.
>>
>> That said it is not a blocker.
>>
>>
>>>
>>>
>>>>
>>>> Im less exited by assertj but it is probably a habit thing.
>>>
>>> I'm not sure if I see your issue with AssertJ. Have you used it? AssertJ
>>> brings clear expression and readable tests in general and gives me a
>>> simple way to create custom assertions to support special parts for a
>>> Maven build like Log file, project structure, target directory contents,
>>> content of archive files etc. ?
>>>
>>> like this:
>>>
>>>    assertThat(result)
>>>      .project()
>>>        .hasTarget()
>>>          .withEarFile()
>>>             .containsOnlyOnce("META-INF/MANIFEST.MF");
>>>
>>> I've never seen a assertion framework which makes it possible to write
>>> tests in a more or less fluent way ...
>>>
>>
>> I did use it a few and dropped it since it does not bring much i  most
>> cases.
>> Asserts stay simple and chaining them just hit autoformatting limitations
>> in general.
>> Also assertj had some dependency conflicts - fixed now IIRC.
>> So staying minimal was good.
>>
>> The side note being: do you need it or can you back your dsl with jupiter
>> assertions? No strong need ;).
>>
>>
>>> What would be your choice?
>>>
>>
>> Just an available model, fluent if you want, but no additional dep.
>> Default Assertions is enough for most cases, in particular with streams and
>> lambdas.
>>
>>
>>>
>>>>
>>>> Wonder if you evaluated to run in a fake filesystem like jimfs or so
>> and
>>>> enable the pom+files to be defined on the test method?
>>>
>>> This is exactly where I have decided against at the moment cause
>>> construction of a full maven project with all it's pom file(s)
>>> directories source code etc. is based on the things I want to solve to
>>> complicated...we already have such things[2] also in plexus testcase you
>>> can do such things or more easier today just use mockito ...
>>>
>>
>> Hmm, this is not comparable.
>> Mockito is like not writing tests, jimfs is launching a real build on a
>> fake in memory filesystem, no mocking for maven.
>>
>>
>>> Currently I have gone the path to have a convention where to find the
>>> projects which are used to be tested like maven-invoker-plugin already
>>> does ...and can also being executed manually within the directory
>>> without any change ...helps a lot in case error hunting ...
>>>
>>> Can you explain the hint about fake filesystem like jimfs a little bit
>>> more cause I don't know jimfs etc. and what you have in mind?
>>>
>>
>> Jimfs is a FileSystem implementation so if maven uses java.nio.file api
>> then we can run on an in memory FS and configure it from any metamodel we
>> want.
>>
>> It would be close to this sftp testing api
>>
>> https://github.com/rmannibucau/rule-them-all/blob/master/src/test/java/com/github/rmannibucau/rules/test/sftp/TestSftpRule.java#L32
>
>
> That would prevent the tests from running Maven in a different JVM... Not
> sure if the JUnit extension supports that, but knowing the libraries likely
> used I suspect it does


Currently it is not really implemented but prepared
https://github.com/khmarbaise/maven-it-extension/blob/master/src/main/java/org/apache/maven/jupiter/extension/ApplicationExecutor.java

There I can give a parameter for the JVM which will be used..The
MavenITExtension at the moment has not implemented that but it's
prepared to do so...

At the moment it's prototype...


Kind regards
Karl Heinz Marbaise

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Next Generation Integration Testing for Plugins/Core

Romain Manni-Bucau
In reply to this post by Tibor Digana
Hi everyone,

Yesterday I needed to test a maven plugin around graal so wrote a junit5
extension relying on testcontainers. Think it is close to this thread so
sharing the idea/code:
https://gitbox.apache.org/repos/asf?p=geronimo-arthur.git;a=blob;f=integration-test/src/test/java/org/apache/geronimo/arthur/integrationtests/MavenTest.java;h=4372204eaa24513b4c9ecaf6b4a17c22339892bc;hb=19d8c093008ca4c03ced1858aed0f884a60d7220

Le ven. 1 nov. 2019 à 01:08, Tibor Digana <[hidden email]> a écrit :

> am, programming language is our toy ;-)
> everybody has some preferences, so i respect them and i understand that
> even the Lambda would be a big jump for us nevertheless the Groovy or
> Kotlin.
> i saw the parameterized tests, re-runs in Groovy, log result of assertion
> statements, and I spoke with Benedikt and we agreed that Spock is very
> special and we like it.
> i do not want to push Karl. Maybe one advice is to think about the
> programming approach where these annotations and code would be easily used
> in another languages too.
> that's basically all from my side.
>
> Enjoy!
>
>
> On Thu, Oct 31, 2019 at 11:56 PM Vladimir Sitnikov <
> [hidden email]> wrote:
>
> > Karl>on the language features but since JDK8 I don't see any advantage
> >
> > What about Kotlin?
> >
> > There are nice things there: the language is statically compiled, great
> > Java interop, there are extension functions, multiline strings, helpful
> > standard library, default parameters.
> > Kotlin is great for creating DSLs.
> >
> > It can be used with JUnit or other frameworks (e.g.
> > https://github.com/kotlintest/kotlintest )
> >
> > Vladimir
> >
>