My maven module dependencies got wipe out randomly

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

My maven module dependencies got wipe out randomly

Dan Tran
Hello to all,

I have a 300+ java modules [0] which takes about 2 min to build without
tests.  The build randomly failing once a while(%5) where I am able to
chase it to the root cause mentioned in the title

Further guessing[2] takes me to the removal of one module which
uses openapi-generator-maven-plugin-4.3.1[1] and the issue completely
disappears.

Looking at the plugin source, iI could not figure out anything standing out.

Looking for more set of eyes to see if that plugin is thread-safe

Appreciate all advice and helps

-D

[0] error is reproducible with maven 3.6.3 and latest 3.7.0-SNAPSHOT

[1]
https://github.com/OpenAPITools/openapi-generator/blob/v4.3.1/modules/openapi-generator-maven-plugin/src/main/java/org/openapitools/codegen/plugin/CodeGenMojo.java

[2] I used to face this type of issue before where a none threadsafe plugin
can corrupted classpath in reactor mode ( for example:
build-number-maven-plugin)
Reply | Threaded
Open this post in threaded view
|

Re: My maven module dependencies got wipe out randomly

Tomo Suzuki
Hi Dan,

(I don't use openapi-generator-maven-plugin)
I had a similar situation where my Maven plugin (Linkage Checker enforcer
rule
<https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/Linkage-Checker-Enforcer-Rule>,
which reads dependencies of a project) stopped working when I started to
use Maven's "-T" option. The root cause was that, with multi-threading, JAR
files in ~/.m2/repository were deleted and recreated while the plugin was
reading JAR files as another thread was resolving dependencies of another
project. I figured out that when I checked the timestamp of the error and
the JAR files. I ended up not running the plugin with multi-thread.

So if your error is about missing files and the files' timestamps were
close to the time of the error, that might be caused by another project
being built simultaneously.

Regards,
Tomo



On Tue, Oct 13, 2020 at 2:30 AM Dan Tran <[hidden email]> wrote:

> Hello to all,
>
> I have a 300+ java modules [0] which takes about 2 min to build without
> tests.  The build randomly failing once a while(%5) where I am able to
> chase it to the root cause mentioned in the title
>
> Further guessing[2] takes me to the removal of one module which
> uses openapi-generator-maven-plugin-4.3.1[1] and the issue completely
> disappears.
>
> Looking at the plugin source, iI could not figure out anything standing
> out.
>
> Looking for more set of eyes to see if that plugin is thread-safe
>
> Appreciate all advice and helps
>
> -D
>
> [0] error is reproducible with maven 3.6.3 and latest 3.7.0-SNAPSHOT
>
> [1]
>
> https://github.com/OpenAPITools/openapi-generator/blob/v4.3.1/modules/openapi-generator-maven-plugin/src/main/java/org/openapitools/codegen/plugin/CodeGenMojo.java
>
> [2] I used to face this type of issue before where a none threadsafe plugin
> can corrupted classpath in reactor mode ( for example:
> build-number-maven-plugin)
>


--
Regards,
Tomo
Reply | Threaded
Open this post in threaded view
|

Re: My maven module dependencies got wipe out randomly

Dan Tran
Hi Tomo

Thanks for a very good hint.

As I recalled we used to have multithreaded issue with this plugin's where
I had to ensure all modules using this plugin to run in sequence to prevent
the multithreaded issues.  The latest version indicates it is now
threadsafe, and using my trick to sequence the build not working either

Now I need to looking into local repo to see any sign of download
corruption.  it would be a challenge due to its randomness

Love to hear more similar stories from this list

Thanks

-D


On Tue, Oct 13, 2020 at 10:24 AM Tomo Suzuki <[hidden email]>
wrote:

> Hi Dan,
>
> (I don't use openapi-generator-maven-plugin)
> I had a similar situation where my Maven plugin (Linkage Checker enforcer
> rule
> <
> https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/Linkage-Checker-Enforcer-Rule
> >,
> which reads dependencies of a project) stopped working when I started to
> use Maven's "-T" option. The root cause was that, with multi-threading, JAR
> files in ~/.m2/repository were deleted and recreated while the plugin was
> reading JAR files as another thread was resolving dependencies of another
> project. I figured out that when I checked the timestamp of the error and
> the JAR files. I ended up not running the plugin with multi-thread.
>
> So if your error is about missing files and the files' timestamps were
> close to the time of the error, that might be caused by another project
> being built simultaneously.
>
> Regards,
> Tomo
>
>
>
> On Tue, Oct 13, 2020 at 2:30 AM Dan Tran <[hidden email]> wrote:
>
> > Hello to all,
> >
> > I have a 300+ java modules [0] which takes about 2 min to build without
> > tests.  The build randomly failing once a while(%5) where I am able to
> > chase it to the root cause mentioned in the title
> >
> > Further guessing[2] takes me to the removal of one module which
> > uses openapi-generator-maven-plugin-4.3.1[1] and the issue completely
> > disappears.
> >
> > Looking at the plugin source, iI could not figure out anything standing
> > out.
> >
> > Looking for more set of eyes to see if that plugin is thread-safe
> >
> > Appreciate all advice and helps
> >
> > -D
> >
> > [0] error is reproducible with maven 3.6.3 and latest 3.7.0-SNAPSHOT
> >
> > [1]
> >
> >
> https://github.com/OpenAPITools/openapi-generator/blob/v4.3.1/modules/openapi-generator-maven-plugin/src/main/java/org/openapitools/codegen/plugin/CodeGenMojo.java
> >
> > [2] I used to face this type of issue before where a none threadsafe
> plugin
> > can corrupted classpath in reactor mode ( for example:
> > build-number-maven-plugin)
> >
>
>
> --
> Regards,
> Tomo
>