maven failsafe plugin & POJO tests

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

maven failsafe plugin & POJO tests

Nigel Jones
I have some 'integration' tests that have been written as POJOs. They run manually

I'm looking at refactoring to automate, making use of the failsafe plugin, together with a few other plugins to launch a server process, and configure the server against which the integration tests run.

I'm struggling to get failsafe to call any tests...

We have a 300+ multi module maven project. A significant amount of those projects contribute to a spring boot application - this is all built/assembled through maven.

When the pojo tests were written, they were added as a regular maven module. As such the source is in the usual source folders, not test ... so I have added configuration like

                           <testSourceDirectory>${project.build.sourceDirectory}</testSourceDirectory>
                            <testClassesDirectory>${project.build.outputDirectory}</testClassesDirectory>

The classes didn't follow the failsafe naming convention, so I added:

                           <includes>
                                 <include>**/*.java</include>
                                 <include>**/*.java</include>
                             </includes>

This isn't where I finally want to get to (refactoring to naming conventions will be clearer for the future) but as a demonstration/first pass, I thought this would be a sensible evolutionary approach.

The tests themselves are not written using any framework like junit/testNG, and require input/parms, but in a class which begins:

 package org.odpi.openmetadata.accessservices.subjectarea.fvt;
// imports
 public class GlossaryFVT {

I could add a method like:

    public static void testGlossary()
     {
         try {
             runIt("https://localhost:10443", "fvtserver", "garygeeke");
         } catch (Exception e) {
             // Test method - so ok to do this
             Thread.dumpStack();
         }
     }

(here runIt is an existing method, which is also called via a main() entry point - but that's not of interest in  migrating to failsafe)

Not only is the method not reported by failsafe, but the class isn't either. For the method, I also tried prepending/appending with 'test' , 'Test', 'IT' but couldn't get any combination working. The code on github (for surefire) hinted at Test, but the docs refer to IT

In the actual source directory I have a number of classes:

CategoryFVT.java                      GlossaryFVT.java                      RunAllFVTNow.java
CategoryHierarchyFVT.java             GraphFVT.java                         SubjectAreaDefinitionCategoryFVT.java
EffectiveDatesFVT.java                ProjectFVT.java                       SubjectAreaFVTCheckedException.java
FVTConstants.java                     RelationshipsFVT.java                 TermFVT.java
FVTUtils.java                         RunAllFVT.java

and when I run I actually get some of these classes picked up by surefire for example:

[INFO] --- maven-failsafe-plugin:3.0.0-M5:integration-test (local) @ subject-area-fvt ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

Interesting ... in fact I changed the includes to '**/*.FVT' as these represent the classes I actually will modify for testing (as opposed to utils/runners) - but then I had NONE of the classes listed above. Hence I went back to * for purpose of explanation .

If we look at one of those picked up -

$ cat src/main/java/org/odpi/openmetadata/accessservices/subjectarea/fvt/FVTConstants.java
/* SPDX-License-Identifier: Apache-2.0 */
/* Copyright Contributors to the ODPi Egeria project. */
package org.odpi.openmetadata.accessservices.subjectarea.fvt;
public class FVTConstants
{
    public static final String USERID = "erinoverview";
    public static final String DEFAULT_URL = "https://localhost:9443";
    public static final String SERVER_NAME1  = "cocoMDS4";
    public static final String SERVER_NAME2  = "cocoMDS1";
}

There's nothing of interest there..

So, I am confused over the status of POJO testing, how to specify classes to test, and which methods will be called.

Can anyone explain?

Are pojo tests supported?
What methods are counted as tests?
Can I override the method names used
Why aren't my other classes even picked up when figuring out classes to use? let alone methods?

I should add I'm using the latest 3.0.0-M5 plugin, though also tried backing off to 2.x - same results. Currently java 14 locally, though java 8 is the base build env. MacOS

I can do more refactoring - ie move to test directories, refactor code, follow method naming conventions (once known), and if needed just do more refactoring and move to JUnit etc, but was confused why my first step wasn't working?

Many thanks
Nigel Jones
ODPi Egeria maintainer
https://github.com/odpi/egeria




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Bernd Eckenfels
Hello,

The failsafe IntegrationTestMojo essentially is the surefire plugin (but with a different error lifecycle handled by the verifyMojo) and a different default include filter.

https://github.com/apache/maven-surefire/blob/master/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java#L223

I would have expected it only supports Junit4/5 and TestNG conventions (I.e. @Test annotations)

BTW one option would be to deliver a single regular JUnit annotated Test which dynamiclly introspective or calls the spring test wrapper - I don't know much about spring testing but I would asume such a thing already exists.

Gruss
Bernd
--
http://bernd.eckenfels.net
________________________________
Von: Nigel Jones <[hidden email]>
Gesendet: Thursday, July 2, 2020 6:37:42 PM
An: [hidden email] <[hidden email]>
Betreff: maven failsafe plugin & POJO tests

I have some 'integration' tests that have been written as POJOs. They run manually

I'm looking at refactoring to automate, making use of the failsafe plugin, together with a few other plugins to launch a server process, and configure the server against which the integration tests run.

I'm struggling to get failsafe to call any tests...

We have a 300+ multi module maven project. A significant amount of those projects contribute to a spring boot application - this is all built/assembled through maven.

When the pojo tests were written, they were added as a regular maven module. As such the source is in the usual source folders, not test ... so I have added configuration like

                           <testSourceDirectory>${project.build.sourceDirectory}</testSourceDirectory>
                            <testClassesDirectory>${project.build.outputDirectory}</testClassesDirectory>

The classes didn't follow the failsafe naming convention, so I added:

                           <includes>
                                 <include>**/*.java</include>
                                 <include>**/*.java</include>
                             </includes>

This isn't where I finally want to get to (refactoring to naming conventions will be clearer for the future) but as a demonstration/first pass, I thought this would be a sensible evolutionary approach.

The tests themselves are not written using any framework like junit/testNG, and require input/parms, but in a class which begins:

 package org.odpi.openmetadata.accessservices.subjectarea.fvt;
// imports
 public class GlossaryFVT {

I could add a method like:

    public static void testGlossary()
     {
         try {
             runIt("https://localhost:10443", "fvtserver", "garygeeke");
         } catch (Exception e) {
             // Test method - so ok to do this
             Thread.dumpStack();
         }
     }

(here runIt is an existing method, which is also called via a main() entry point - but that's not of interest in  migrating to failsafe)

Not only is the method not reported by failsafe, but the class isn't either. For the method, I also tried prepending/appending with 'test' , 'Test', 'IT' but couldn't get any combination working. The code on github (for surefire) hinted at Test, but the docs refer to IT

In the actual source directory I have a number of classes:

CategoryFVT.java                      GlossaryFVT.java                      RunAllFVTNow.java
CategoryHierarchyFVT.java             GraphFVT.java                         SubjectAreaDefinitionCategoryFVT.java
EffectiveDatesFVT.java                ProjectFVT.java                       SubjectAreaFVTCheckedException.java
FVTConstants.java                     RelationshipsFVT.java                 TermFVT.java
FVTUtils.java                         RunAllFVT.java

and when I run I actually get some of these classes picked up by surefire for example:

[INFO] --- maven-failsafe-plugin:3.0.0-M5:integration-test (local) @ subject-area-fvt ---
[INFO]
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVT
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.RunAllFVTNow
[INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTUtils
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

Interesting ... in fact I changed the includes to '**/*.FVT' as these represent the classes I actually will modify for testing (as opposed to utils/runners) - but then I had NONE of the classes listed above. Hence I went back to * for purpose of explanation .

If we look at one of those picked up -

$ cat src/main/java/org/odpi/openmetadata/accessservices/subjectarea/fvt/FVTConstants.java
/* SPDX-License-Identifier: Apache-2.0 */
/* Copyright Contributors to the ODPi Egeria project. */
package org.odpi.openmetadata.accessservices.subjectarea.fvt;
public class FVTConstants
{
    public static final String USERID = "erinoverview";
    public static final String DEFAULT_URL = "https://localhost:9443";
    public static final String SERVER_NAME1  = "cocoMDS4";
    public static final String SERVER_NAME2  = "cocoMDS1";
}

There's nothing of interest there..

So, I am confused over the status of POJO testing, how to specify classes to test, and which methods will be called.

Can anyone explain?

Are pojo tests supported?
What methods are counted as tests?
Can I override the method names used
Why aren't my other classes even picked up when figuring out classes to use? let alone methods?

I should add I'm using the latest 3.0.0-M5 plugin, though also tried backing off to 2.x - same results. Currently java 14 locally, though java 8 is the base build env. MacOS

I can do more refactoring - ie move to test directories, refactor code, follow method naming conventions (once known), and if needed just do more refactoring and move to JUnit etc, but was confused why my first step wasn't working?

Many thanks
Nigel Jones
ODPi Egeria maintainer
https://github.com/odpi/egeria




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Nigel Jones


On 2020/07/02 17:10:08, Bernd Eckenfels <[hidden email]> wrote:

> The failsafe IntegrationTestMojo essentially is the surefire plugin (but with a different error lifecycle handled by the verifyMojo) and a different default include filter.
>
> https://github.com/apache/maven-surefire/blob/master/maven-failsafe-plugin/src/main/java/org/apache/maven/plugin/failsafe/IntegrationTestMojo.java#L223

Thanks -- I did look at that file earlier - and saw nothing there that might change the name used for Pojo tests which seems to remain as including 'Test'

> I would have expected it only supports Junit4/5 and TestNG conventions (I.e. @Test annotations)

I'm thinking that the pojo support isn't something most people use, and I only really looked at it to avoid too much refactoring of code to prove a concept.

> BTW one option would be to deliver a single regular JUnit annotated Test which dynamiclly introspective or calls the spring test wrapper - I don't know much about spring testing but I would asume such a thing already exists.

Thanks - though am not so concerned with spring - I perhaps caused some confusion as this is how the server is built. We do have other tests (ie unit tests) but with failsafe I'm looking at integration tests that depend on a remote server (accessed either via REST or a simple client). So spring isn't really a factor for these particular tests

Thanks for the reply. My current thought is to step away from pojo and go down the JUnit5 route as it seems a more natural, and commonly used fit.

That being said the surefire docs do reference POJO tests at https://maven.apache.org/surefire/maven-failsafe-plugin/usage.html hence wondering if there were any useful tips to get that working :-)

Thanks
nigel


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Piotr Żygieło
In reply to this post by Nigel Jones
>
>     public static void testGlossary()
>      {

POJO ITs work in my set up, but my test methods are not static.

--
Piotrek

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Nigel Jones


On 2020/07/02 17:39:25, Piotr Żygieło <[hidden email]> wrote:
> >
> >     public static void testGlossary()
> >      {
>
> POJO ITs work in my set up, but my test methods are not static.


Thanks. Can I check what naming convention you've used for your test methods -- and/or configuration to override this? How about your classes?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Piotr Żygieło
> > >     public static void testGlossary()
> > >      {
> >
> > POJO ITs work in my set up, but my test methods are not static.
>
> Thanks. Can I check what naming convention you've used for your test methods -- and/or configuration to override this? How about your classes?

In plugin I only define executions (i.e. have no redefined includes),
and test classes follow convention *IT (like SimpleIT).

When I change my method to static I get similar result to your log:

> [INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
> [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants

- class is considered by failsafe but no test is discovered.

Did you try to make test method in GlossaryFVT not static and run
maven `verify -Dit.test=GlossaryFVT`? It shall not need/ignore
includes configured.

> For the method, I also tried prepending/appending with 'test' , 'Test', 'IT' but couldn't get any combination working.
> [...]
> naming convention you've used for your test methods

Note, that per http://maven.apache.org/surefire/maven-surefire-plugin/examples/pojo-test.html
- test method has to be named test* and it works fine.

--
Piotrek

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Nigel Jones


On 2020/07/02 18:23:13, Piotr Żygieło <[hidden email]> wrote:

> In plugin I only define executions (i.e. have no redefined includes),
> and test classes follow convention *IT (like SimpleIT).
 Thanks

> When I change my method to static I get similar result to your log:
>
> > [INFO] Running org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
> > [INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.004 s - in org.odpi.openmetadata.accessservices.subjectarea.fvt.FVTConstants
>
> - class is considered by failsafe but no test is discovered.

What is odd though is that the class 'GlossaryFVT' itself isn't showing up in the Tests run line -- even with the includes in place. Yet other files like 'FVTContants' are. This I don't understand - before I get to the methods (though once found, the static methods observation is very valuable. )

I can also rename that method as GlossaryIT - I did this (after the point below) and the class isn't listed in the output - even with 0 tests (which is what I'd expect if just the method name was wrong)
 
> Did you try to make test method in GlossaryFVT not static and run
> maven `verify -Dit.test=GlossaryFVT`? It shall not need/ignore
> includes configured.

Yes I did, but the run just includes

[INFO] Results:
[INFO]
[INFO] Tests run: 0, Failures: 0, Errors: 0, Skipped: 0

So no mention of that class at all

> Note, that per http://maven.apache.org/surefire/maven-surefire-plugin/examples/pojo-test.html
> - test method has to be named test* and it works fine.

Thanks, I did try test*, *test, IT*, and *IT ... since I know the default class names were changed from surefire -> failsafe.. wasn't sure about the methods

However thanks -- I'll look through that example as I'd missed it..|

I also thought to check 'mvn help:effective-pom' but, as expected, I'm not picking up any other failsafe related configuration.


> Piotrek
Thanks for trying :-)


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Nigel Jones


On 2020/07/02 19:08:26, Nigel Jones <[hidden email]> wrote:

> What is odd though is that the class 'GlossaryFVT' itself isn't showing up in the Tests run line -- even with the includes in place. Yet other files like 'FVTContants' are. This I don't understand - before I get to the methods (though once found, the static methods observation is very valuable. )

From a quick test, it does seem that if I convert to a JUnit5 test I don't have an issue in getting tests run - seems specific to the pojo tests. I may work on that tomorrow.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Tibor Digana
Hi Nigel,

I wrote a project for you
https://github.com/Tibor17/pojo-testing

It is related to the JUnit POJO testing with Maven Failsafe Plugin.
I guess you did not use the expected postfix. That's why there were 0 tests
to run.
Additionally, README file shows the command line example with filtering the
test.
btw, The postfix can be customized by the configuration.

Cheers
Tibor




On Thu, Jul 2, 2020 at 9:44 PM Nigel Jones <[hidden email]> wrote:

>
>
> On 2020/07/02 19:08:26, Nigel Jones <[hidden email]> wrote:
>
> > What is odd though is that the class 'GlossaryFVT' itself isn't showing
> up in the Tests run line -- even with the includes in place. Yet other
> files like 'FVTContants' are. This I don't understand - before I get to the
> methods (though once found, the static methods observation is very
> valuable. )
>
> From a quick test, it does seem that if I convert to a JUnit5 test I don't
> have an issue in getting tests run - seems specific to the pojo tests. I
> may work on that tomorrow.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Nigel Jones
In reply to this post by Nigel Jones
Thanks for all the replies

I did try the test project from @tibordigana which worked well. I was able to change names etc and still have the tests working, but oddly could still not figure out why my main project was failing to find all the correct classes & methods.

I ended up abandoning trying to get the pojos working & just created some JUnit5 tests to wrap the existing tests. Not perfect as it's a small handful of large tests and should be decomposed, but that is independent of the framework. The key thing was with JUnit5 there was no issue locating and running the tests - and in any case it's a better model for future tests.

Regards
Nigel.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: maven failsafe plugin & POJO tests

Nigel Jones
In reply to this post by Tibor Digana


Just to close out on this issue -- we have tests working now. The intent was always to refactor into Junit tests in any case -- and these are getting fired correctly.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]