Re: Speed of Maven build

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Speed of Maven build

John Patrick
Are you planning to create a baseline project or selecting a range of
projects to be used as a baseline, so that perceived improvements can
be monitored? So that anyone wanting to help out or give feedback can
submit their own build performance.

i.e.
1. Equipment OS, Ram, CPU, physical, virtual, docker, openshift, other
2. Java version
3. Maven version
4. Speedtest results
5. Direct Internet Connection or via Http Proxy or via Nexus/Artifactory
6. Clean/Fresh Local Repo Execution Time
7. 2nd Execution Time, after everything downloaded

As using Maven since 2005, I've found each new release has gotten
faster and faster, and most performance issues have been around what
OS I'm using, SSD vs HDD and also do you have enough free RAM etc.

As I'm surprised how quickly my builds are running at the moment, the
only issue is when I see maven perform internet connections
downloading new dependencies or say the versions plugin to check. Any
thoughts about adding a HTTP/2 Server Push support so if it's Maven
Aware and you request the pom it can also push back the hashes and
maybe the jar too.

Regarding a "zombie" maven instance, it should be opt in so i need to
explicitly enable it as often i'm jumping around between lots of
projects so don't want each having a "zombie" progress and i might not
be building that project again for another week maybe.

John

On Tue, 25 Aug 2020 at 12:27, Jeff Jensen
<[hidden email]> wrote:

>
> In case this helps, Jason Dillon has a "Maven Shell" that does what you
> seek for CLI - launches a Maven instance and runs interactive commands with
> it, saving the startup time.
> https://github.com/jdillon/mvnsh
>
>
> On Tue, Aug 25, 2020 at 12:27 AM Jaroslav Tulach <[hidden email]>
> wrote:
>
> > > And it's Apache Maven, over the corner at https://maven.apache.org/ so
> > > I suppose that community would be happy to get such contributions.
> > >
> > > -Bertrand
> >
> > You are right, Bertrand. Why not ask!
> >
> > Hello Maven guys,
> > we had a discussion on the NetBeans mailing list recently and here is a
> > summary:
> > * Apache NetBeans IDE is delegating most of its work directly to Maven
> > * Users however complain that the speed isn't great
> > * One of the ideas was to launch a "zombie" instance of Maven in advance
> > * then actions like build, exec or test would be faster
> >
> > Have you thought about something like this already? Any advices?
> >
> > Best regards.
> > Jaroslav Tulach
> > NetBeans Platform Architect
> >
> > ne 23. 8. 2020 v 9:06 odesílatel Jaroslav Tulach <
> > [hidden email]>
> > napsal:
> >
> > > > I agree with others, Ant is much faster day to day. But the pom.xml has
> > > > become the universal project file for Java,
> > >
> > > Thank you all for sharing your thoughts. I know Maven start is slower,
> > but
> > > I
> > > learned to live with it. It is interesting to hear that some of you
> > > maintain a
> > > dual Ant based copy of your project metadata. Once we were trying a
> > > different
> > > approach:
> > >
> > > There is a way to speed Maven in the IDE. Launch Maven, let it read all
> > > XML &
> > > co. files and stop it. As soon as we need to build/run/test, wake up this
> > > zombie Maven process, tell it what to do and let it continue. If the XML
> > > files
> > > are modified, throw the process away and initialize it again. Tomáš
> > Stupka
> > > implemented a prototype of this and there were no issues, as far as I
> > know
> > > (nobody tested it thoroughly however).
> > >
> > > Maybe the support is even in and there is a property to turn it on. If
> > the
> > > Maven startup is the biggest problem for you guys, we shall investigate
> > > how to
> > > turn Tomáš's work on...
> > >
> > > -jt
> > >
> > >
> > >
> > >
> > >
> > >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Maven build

Jaroslav Tulach
Dne středa 26. srpna 2020 10:01:33 CEST, Davide Grandi napsal(a):
> Sorry to be late, but
>      there is a project test case for Maven "slowlyness" ?

This is a great question, thanks Davide!

I've just tried to create such project:
https://issues.apache.org/jira/browse/NETBEANS-4746
and yes, Maven is slower (see the instructions in the issue):

```bash
justtest$ mvn -quiet -Dtest=MainTest process-test-classes surefire:test
justtest$ ant -q test-single
```

However only 2.6s vs. 1.9s and that cannot be the problem NetBeans users
complain about.

I guess the real problem is when people open the projects in the NetBeans IDE.

Try to remove `pom.xml` and open the folder in NetBeans IDE 12.x - it opens as
an Ant project. Select `MainTest.java` class and press Ctrl+F6 - repeat - at
the end it takes less then 1s!

Repeat again from scratch, but remove `build.xml` file and `nbproject`
directory. Then the project opens as Maven. Select `MainTest.java` class and
press Ctrl+F6 - takes more than 2s - repeat - takes more than 2s again.

What cases the difference?

Hypothesis: Probably different Ant and Maven integration into the NetBeans IDE.
While Ant is executed in the same JVM as the IDE in an isolated classloader,
Maven is probably always started as a separate process. Given a regular JDK
doesn't do anything useful for first 1-1.5s of execution we might have found
the difference. Ant code is reused again and again and warms up over time.
Maven always starts from scratch and JDK's warming delay causes the wrong
impression.

Consider verifying my claims and observations.
-jt


>
> I mean : a project for which there's a time gap between
>
> - nb/command line mvn invocations
> - or between ant / mavn builds (I know : how many projects have TWO
> build files ? ...)
>
> for routinely rebuilds of big projects I often activate the "quiet" and
> "parallel" build params
> and keep an eye at mvn java memory params.
> Usual network and I/O tricks may apply.
>
> bye,
>
>      Davide
>
> On 25/08/2020 07:27, Jaroslav Tulach wrote:
> >> And it's Apache Maven, over the corner at https://maven.apache.org/ so
> >> I suppose that community would be happy to get such contributions.
> >>
> >> -Bertrand
> >
> > You are right, Bertrand. Why not ask!
> >
> > Hello Maven guys,
> > we had a discussion on the NetBeans mailing list recently and here is a
> > summary:
> > * Apache NetBeans IDE is delegating most of its work directly to Maven
> > * Users however complain that the speed isn't great
> > * One of the ideas was to launch a "zombie" instance of Maven in advance
> > * then actions like build, exec or test would be faster
> >
> > Have you thought about something like this already? Any advices?
> >
> > Best regards.
> > Jaroslav Tulach
> > NetBeans Platform Architect
> >
> > ne 23. 8. 2020 v 9:06 odesílatel Jaroslav Tulach
> > <[hidden email]>>
> > napsal:
> >>> I agree with others, Ant is much faster day to day. But the pom.xml has
> >>> become the universal project file for Java,
> >>
> >> Thank you all for sharing your thoughts. I know Maven start is slower,
> >> but
> >> I
> >> learned to live with it. It is interesting to hear that some of you
> >> maintain a
> >> dual Ant based copy of your project metadata. Once we were trying a
> >> different
> >> approach:
> >>
> >> There is a way to speed Maven in the IDE. Launch Maven, let it read all
> >> XML &
> >> co. files and stop it. As soon as we need to build/run/test, wake up this
> >> zombie Maven process, tell it what to do and let it continue. If the XML
> >> files
> >> are modified, throw the process away and initialize it again. Tomáš
> >> Stupka
> >> implemented a prototype of this and there were no issues, as far as I
> >> know
> >> (nobody tested it thoroughly however).
> >>
> >> Maybe the support is even in and there is a property to turn it on. If
> >> the
> >> Maven startup is the biggest problem for you guys, we shall investigate
> >> how to
> >> turn Tomáš's work on...
> >>
> >> -jt





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Speed of Maven build

Martin Kanters
Hi Jaroslav, nice analysis. In the maven-integration-testing repository [1]
Maven can be started fully independently or embedded in the same JDK. The
latter is activated with the Maven profile "embedded".
This also gives a huge performance boost, running tests in tenths of a
second instead of seconds.

Martin

[1] https://github.com/apache/maven-integration-testing/

Op do 27 aug. 2020 om 07:30 schreef Jaroslav Tulach <
[hidden email]>:

> Dne středa 26. srpna 2020 10:01:33 CEST, Davide Grandi napsal(a):
> > Sorry to be late, but
> >      there is a project test case for Maven "slowlyness" ?
>
> This is a great question, thanks Davide!
>
> I've just tried to create such project:
> https://issues.apache.org/jira/browse/NETBEANS-4746
> and yes, Maven is slower (see the instructions in the issue):
>
> ```bash
> justtest$ mvn -quiet -Dtest=MainTest process-test-classes surefire:test
> justtest$ ant -q test-single
> ```
>
> However only 2.6s vs. 1.9s and that cannot be the problem NetBeans users
> complain about.
>
> I guess the real problem is when people open the projects in the NetBeans
> IDE.
>
> Try to remove `pom.xml` and open the folder in NetBeans IDE 12.x - it
> opens as
> an Ant project. Select `MainTest.java` class and press Ctrl+F6 - repeat -
> at
> the end it takes less then 1s!
>
> Repeat again from scratch, but remove `build.xml` file and `nbproject`
> directory. Then the project opens as Maven. Select `MainTest.java` class
> and
> press Ctrl+F6 - takes more than 2s - repeat - takes more than 2s again.
>
> What cases the difference?
>
> Hypothesis: Probably different Ant and Maven integration into the NetBeans
> IDE.
> While Ant is executed in the same JVM as the IDE in an isolated
> classloader,
> Maven is probably always started as a separate process. Given a regular
> JDK
> doesn't do anything useful for first 1-1.5s of execution we might have
> found
> the difference. Ant code is reused again and again and warms up over time.
> Maven always starts from scratch and JDK's warming delay causes the wrong
> impression.
>
> Consider verifying my claims and observations.
> -jt
>
>
> >
> > I mean : a project for which there's a time gap between
> >
> > - nb/command line mvn invocations
> > - or between ant / mavn builds (I know : how many projects have TWO
> > build files ? ...)
> >
> > for routinely rebuilds of big projects I often activate the "quiet" and
> > "parallel" build params
> > and keep an eye at mvn java memory params.
> > Usual network and I/O tricks may apply.
> >
> > bye,
> >
> >      Davide
> >
> > On 25/08/2020 07:27, Jaroslav Tulach wrote:
> > >> And it's Apache Maven, over the corner at https://maven.apache.org/
> so
> > >> I suppose that community would be happy to get such contributions.
> > >>
> > >> -Bertrand
> > >
> > > You are right, Bertrand. Why not ask!
> > >
> > > Hello Maven guys,
> > > we had a discussion on the NetBeans mailing list recently and here is a
> > > summary:
> > > * Apache NetBeans IDE is delegating most of its work directly to Maven
> > > * Users however complain that the speed isn't great
> > > * One of the ideas was to launch a "zombie" instance of Maven in
> advance
> > > * then actions like build, exec or test would be faster
> > >
> > > Have you thought about something like this already? Any advices?
> > >
> > > Best regards.
> > > Jaroslav Tulach
> > > NetBeans Platform Architect
> > >
> > > ne 23. 8. 2020 v 9:06 odesílatel Jaroslav Tulach
> > > <[hidden email]>>
> > > napsal:
> > >>> I agree with others, Ant is much faster day to day. But the pom.xml
> has
> > >>> become the universal project file for Java,
> > >>
> > >> Thank you all for sharing your thoughts. I know Maven start is slower,
> > >> but
> > >> I
> > >> learned to live with it. It is interesting to hear that some of you
> > >> maintain a
> > >> dual Ant based copy of your project metadata. Once we were trying a
> > >> different
> > >> approach:
> > >>
> > >> There is a way to speed Maven in the IDE. Launch Maven, let it read
> all
> > >> XML &
> > >> co. files and stop it. As soon as we need to build/run/test, wake up
> this
> > >> zombie Maven process, tell it what to do and let it continue. If the
> XML
> > >> files
> > >> are modified, throw the process away and initialize it again. Tomáš
> > >> Stupka
> > >> implemented a prototype of this and there were no issues, as far as I
> > >> know
> > >> (nobody tested it thoroughly however).
> > >>
> > >> Maybe the support is even in and there is a property to turn it on. If
> > >> the
> > >> Maven startup is the biggest problem for you guys, we shall
> investigate
> > >> how to
> > >> turn Tomáš's work on...
> > >>
> > >> -jt
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>