How to avoid forking in a maven build?

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

How to avoid forking in a maven build?

Steinar Bang
This project https://github.com/steinarb/authservice has a lot of
messages like this in the build output:

[INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[INFO] Forking Authentication webapp 1.8.0-SNAPSHOT
[INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (filter-resources) @ authservice ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[INFO] Forking Authentication webapp definitions bundle 1.8.0-SNAPSHOT
[INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (filter-resources) @ authservice.definitions ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/sb/workspaces/ws03/authservice/authservice.definitions/src/main/filtered-resources
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ authservice.definitions ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /home/sb/workspaces/ws03/authservice/authservice.definitions/src/main/resources
...

A lot of the forking messages to be repeated, so I worry that it does
some operations (e.g. the npm build of the frontend) way more times than
it has to (ie. only once).

Is the forking something that makes the build take longer than it has
to?  Or is it harmless and nothing worth spending time to fix?

I have tried to figure out who the culprit is. Maybe there are more than
one culprit...?

I thought maybe my use of the maven-resources-plugin in the parent was
the cause of the forking.  So I moved the build of a master karaf
feature repository out of the parent POM and into a module.  But that
only got rid of the first forking in the quoted text abote.

The full output from "mvn clean install" can be seen at:
 https://gist.github.com/steinarb/169d179abec47f50d3aa5574ad8d7585

Thanks!


- Steinar


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to avoid forking in a maven build?

Karl Heinz Marbaise-3
Hi,

based on what I can identify in your pom files/output is that in your
currenty configuration the maven-javadoc-plugin is configured to use the
goal: aggregate[1] which is a goal which forkes the lifecycle. There is
an equivalent "aggregate-no-fork"[2] which could be used instead...

The question is also why running javadoc everytime?

Apart from that you have configured the maven-source-plugin with the
goal "jar" which also forkes the lifecycle[3] this could be replaced by
using the "jar-no-fork" goal[4].

Furthermore your configuration can be simplyfied by using the junit-bom
instead of each dependency separately to import the dependencies for
junit-jupiter ...


The question is: Why have you bound the maven-resources-plugin to
validate life cycle phase?

One thing: Why do you have configured wagon-ftp as extension?

Kind regards
Karl Heinz Marbaise

[1]:
https://maven.apache.org/plugins/maven-javadoc-plugin/aggregate-mojo.html
[2]:
https://maven.apache.org/plugins/maven-javadoc-plugin/aggregate-no-fork-mojo.html
[3]: https://maven.apache.org/plugins/maven-source-plugin/jar-mojo.html
[4]:
https://maven.apache.org/plugins/maven-source-plugin/jar-no-fork-mojo.html

On 31.12.19 16:04, Steinar Bang wrote:

> This project https://github.com/steinarb/authservice has a lot of
> messages like this in the build output:
>
> [INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO] Forking Authentication webapp 1.8.0-SNAPSHOT
> [INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:resources (filter-resources) @ authservice ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 1 resource
> [INFO]
> [INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO] Forking Authentication webapp definitions bundle 1.8.0-SNAPSHOT
> [INFO] >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:resources (filter-resources) @ authservice.definitions ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/sb/workspaces/ws03/authservice/authservice.definitions/src/main/filtered-resources
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ authservice.definitions ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory /home/sb/workspaces/ws03/authservice/authservice.definitions/src/main/resources
> ...
>
> A lot of the forking messages to be repeated, so I worry that it does
> some operations (e.g. the npm build of the frontend) way more times than
> it has to (ie. only once).
>
> Is the forking something that makes the build take longer than it has
> to?  Or is it harmless and nothing worth spending time to fix?
>
> I have tried to figure out who the culprit is. Maybe there are more than
> one culprit...?
>
> I thought maybe my use of the maven-resources-plugin in the parent was
> the cause of the forking.  So I moved the build of a master karaf
> feature repository out of the parent POM and into a module.  But that
> only got rid of the first forking in the quoted text abote.
>
> The full output from "mvn clean install" can be seen at:
>   https://gist.github.com/steinarb/169d179abec47f50d3aa5574ad8d7585
>
> Thanks!

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to avoid forking in a maven build?

Thomas Broyer-2
In reply to this post by Steinar Bang
In this specific case, what causes the forking is the javadoc:aggregate
goal. That one is meant to be used on the command line. If you want to bind
it to a build phase, you should use javadoc:aggregate-no-fork.
Also, javadoc:aggregate forks a "mvn compile", which will trigger the
frontend:webpack (et al.) because they're bound by default to the
generate-resources phase (which is odd if you ask me). You may want to bind
the frontend plugin's goals to another phase, such as prepare-package.

On Tue, Dec 31, 2019 at 4:04 PM Steinar Bang <[hidden email]> wrote:

> This project https://github.com/steinarb/authservice has a lot of
> messages like this in the build output:
>
> [INFO]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO] Forking Authentication webapp 1.8.0-SNAPSHOT
> [INFO]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:resources (filter-resources) @
> authservice ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] Copying 1 resource
> [INFO]
> [INFO]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO] Forking Authentication webapp definitions bundle 1.8.0-SNAPSHOT
> [INFO]
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:resources (filter-resources) @
> authservice.definitions ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory
> /home/sb/workspaces/ws03/authservice/authservice.definitions/src/main/filtered-resources
> [INFO]
> [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @
> authservice.definitions ---
> [INFO] Using 'UTF-8' encoding to copy filtered resources.
> [INFO] skip non existing resourceDirectory
> /home/sb/workspaces/ws03/authservice/authservice.definitions/src/main/resources
> ...
>
> A lot of the forking messages to be repeated, so I worry that it does
> some operations (e.g. the npm build of the frontend) way more times than
> it has to (ie. only once).
>
> Is the forking something that makes the build take longer than it has
> to?  Or is it harmless and nothing worth spending time to fix?
>
> I have tried to figure out who the culprit is. Maybe there are more than
> one culprit...?
>
> I thought maybe my use of the maven-resources-plugin in the parent was
> the cause of the forking.  So I moved the build of a master karaf
> feature repository out of the parent POM and into a module.  But that
> only got rid of the first forking in the quoted text abote.
>
> The full output from "mvn clean install" can be seen at:
>  https://gist.github.com/steinarb/169d179abec47f50d3aa5574ad8d7585
>
> Thanks!
>
>
> - Steinar
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

--
Thomas Broyer
/tɔ.ma.bʁwa.je/ <http://xn--nna.ma.xn--bwa-xxb.je/>
Reply | Threaded
Open this post in threaded view
|

Re: How to avoid forking in a maven build?

Steinar Bang
In reply to this post by Karl Heinz Marbaise-3
>>>>> Karl Heinz Marbaise <[hidden email]>:

> based on what I can identify in your pom files/output is that in your
> currenty configuration the maven-javadoc-plugin is configured to use the
> goal: aggregate[1] which is a goal which forkes the lifecycle. There is
> an equivalent "aggregate-no-fork"[2] which could be used instead...

Thanks! I will try that.

(The purpose of the javadoc aggregate is to get a javadoc jar
attached to the parent pom artifact, that I can link to with a javadoc
banner in the README.)

> The question is also why running javadoc everytime?

Hm... I don't understand...? Could you clarify?

> Apart from that you have configured the maven-source-plugin with the
> goal "jar" which also forkes the lifecycle[3] this could be replaced by
> using the "jar-no-fork" goal[4].

Ok, Thanks! Will try.

> Furthermore your configuration can be simplyfied by using the junit-bom
> instead of each dependency separately to import the dependencies for
> junit-jupiter ...

Thanks! I didn't know that. I will try this as well! (lots of
improvements here)

> The question is: Why have you bound the maven-resources-plugin to
> validate life cycle phase?

I guess because I needed the stuff copied in early in the build.  I
probably arrived there by experimentation. What phase should I use?

> One thing: Why do you have configured wagon-ftp as extension?

Copy/paste leftover from an earlier project. I wanted travis-ci to
deploy to my own snapshot repo and there I initially used ftp, but have
now transitioned to ssh, so this can be deleted (and well... this
particular project is released to maven central so it doesn't need to
put snapshots in a repo).

I will remove it. Thanks again!


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to avoid forking in a maven build?

Steinar Bang
In reply to this post by Thomas Broyer-2
>>>>> Thomas Broyer <[hidden email]>:

> In this specific case, what causes the forking is the javadoc:aggregate
> goal. That one is meant to be used on the command line. If you want to bind
> it to a build phase, you should use javadoc:aggregate-no-fork.

Thanks! I will switch goal here!

> Also, javadoc:aggregate forks a "mvn compile", which will trigger the
> frontend:webpack (et al.) because they're bound by default to the
> generate-resources phase

Right! That's where the rebuilds comes from. Thanks for the explanation.

>(which is odd if you ask me). You may want to bind the frontend
>plugin's goals to another phase, such as prepare-package.

Thanks! I will try!


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to avoid forking in a maven build?

Steinar Bang
Thanks to Thomas Broyer and Karl Heinz Marbaise!

Replacing aggregate-jar in the parent POM with aggregate-no-fork (and an
extra jar:jar) cut down the build time from around 2:50 min to around
50s on a 3.30GHz i5 with openjdk11.

The maven output contained much less clutter avoided re-running npm and
the resource plugin a lot of times.
 https://github.com/steinarb/authservice/commit/22e3314d084d108ce27a27addf0ae404756eb67f


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: How to avoid forking in a maven build?

Stanimir Stamenkov-2
In reply to this post by Steinar Bang
Wed, 01 Jan 2020 03:14:25 +0100, /Steinar Bang/:
>> Karl Heinz Marbaise:
>
> (The purpose of the javadoc aggregate is to get a javadoc jar
> attached to the parent pom artifact, that I can link to with a javadoc
> banner in the README.)
>
>> The question is also why running javadoc everytime?
>
> Hm... I don't understand...? Could you clarify?

I guess it is more common to have the javadoc jar (and sources jar)
packaged just with a "release" profile, speeding up development builds:

mvn package/verify/install (local build w/o javadoc and sources artifacts)

mvn -P release install (local build w/ javadoc and sources artifacts)

mvn -P release deploy (CI build w/ javadoc and sources artifacts)

--
Stanimir

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]