Re: Speeding up Maven

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Speeding up Maven

Karl Heinz Marbaise-3
Hi Mark,


On 13/04/18 21:20, Mark Raynsford wrote:

First it would be good to know which Maven version are you using? And on
which JDK you are running?


> Hello.
>
> Is there any way I might speed up Maven? I've been using it for years
> and, for the first time, I'm actually executing it from an IDE rather
> than using the IDE's built-in compiler. It seems like there's a rather
> large and apparently fixed amount of overhead per invocation. Take the
> following "could not be simpler" project:
>
>    https://github.com/io7m/maven-minimal-20180413


Maybe I misunderstand you ...are you running it from plain command line
or from inside your IDE ?

What are you memory settings? How is your JDK configured ? Which command
line options ? (MAVEN_OPTS?):??



>
> I piped the output through tai64n [1] to give a better indication of
> exactly how long everything is taking. That's a project that compiles a
> single Java file and produces a jar. It doesn't run any tests, it
> doesn't include any resources. 16 seconds total execution [2]!


Unfortunately I don't have tai64n on my Mac but based on simple tests
with your project I can run the whole project in about 1 Second for the
whole build...


>
> Now for an infrequent command-line build, that's not *too* obnoxious.
> For a "click a button in the IDE because I want to try this bit of code
> now" build, that's a world of suffering.
>
> It seems like Maven first spends ten entire seconds doing not very
> much. That cannot be attributed to JVM startup time (Hello World on my
> system takes about 100ms to reach user code, and even the largest Java
> application I still takes less than a second to reach user code after
> all the classes have loaded).

Depending on your JVM and the configuration that can be quite large
different time on startup...there are several configurations which can
be used to improve that...

>
> Is there anything I can do to cut this down? If not, is this something
> that's intended to be addressed in the near future?

I'm doing from time to time analyzes based on performance / Memory...but
I'm interested to do more here...


>
> [1] http://cr.yp.to/daemontools/tai64n.html
> [2] The Maven output claims 6 seconds, but that doesn't include the
>      time it took for the build to start.
>


Kind regards
Karl Heinz Marbaise

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Speeding up Maven

Paul Hammant
Mark,

Assuming a pre-filled Maven local cache, I've a 16 second build that's only
longer when I run a screen recorder in order to make the video show here -
https://paulhammant.com/2017/02/05/a-16-second-java-webapp-build-including-webdriver-tests/

That one runs three WebDriver tests too after the unit and service tests -
meaning (implicitly) it's testing JavaScript as well as Java in that time.

In order to make it fast I've abandoned the fail-safe plugin and do all
parts of the test-pyramid in surefire (see the pom).

- Paul

On Fri, Apr 13, 2018 at 3:20 PM, Mark Raynsford <
[hidden email]> wrote:

> Hello.
>
> Is there any way I might speed up Maven? I've been using it for years
> and, for the first time, I'm actually executing it from an IDE rather
> than using the IDE's built-in compiler. It seems like there's a rather
> large and apparently fixed amount of overhead per invocation. Take the
> following "could not be simpler" project:
>
>   https://github.com/io7m/maven-minimal-20180413
>
> I piped the output through tai64n [1] to give a better indication of
> exactly how long everything is taking. That's a project that compiles a
> single Java file and produces a jar. It doesn't run any tests, it
> doesn't include any resources. 16 seconds total execution [2]!
>
> Now for an infrequent command-line build, that's not *too* obnoxious.
> For a "click a button in the IDE because I want to try this bit of code
> now" build, that's a world of suffering.
>
> It seems like Maven first spends ten entire seconds doing not very
> much. That cannot be attributed to JVM startup time (Hello World on my
> system takes about 100ms to reach user code, and even the largest Java
> application I still takes less than a second to reach user code after
> all the classes have loaded).
>
> Is there anything I can do to cut this down? If not, is this something
> that's intended to be addressed in the near future?
>
> [1] http://cr.yp.to/daemontools/tai64n.html
> [2] The Maven output claims 6 seconds, but that doesn't include the
>     time it took for the build to start.
>
> --
> Mark Raynsford | http://www.io7m.com
>
>


--
Paul Hammant DevOps <https://devops.paulhammant.com> Let me give your
enterprise a step by step plan to get out of the hell of crazy branching
models (ClearCase maybe?) and into the world of high-throughput CD on
DevOps foundations.
Reply | Threaded
Open this post in threaded view
|

Re: Speeding up Maven

Mark Raynsford
On 2018-04-14T09:02:55 +0000
Bernd Eckenfels <[hidden email]> wrote:

> Hello,
>
> Maven is not the fastest, but in your case it sounds unusual slow. What are your machine specs? Any SSD? Do you happen to have a on-access malware scanner active? They react very bad to Java scanning large number of JAR files.

Intel(R) Core(TM) i5-2410M CPU @ 2.30GHz, 8GB RAM, 7200 RPM non-SSD
drive.

No malware scanner (I'm on Linux).

Going to try to set up another workstation today to see if I can
duplicate the problem there.

--
Mark Raynsford | http://www.io7m.com


attachment0 (235 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Speeding up Maven

Tibor Digana
In reply to this post by Karl Heinz Marbaise-3
This is interesting discussion and so I decided to build your project with
Java 9 and Maven 3.5.3.
Completed within 1.5 sec.
I have noticed that Maven 3.3.9 was slower, cca 2.5 sec.

mvn -V test
Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297;
2018-02-24T20:49:05+01:00)
Maven home: D:\apache-maven-3.5.3
Java version: 9.0.1, vendor: Oracle Corporation
Java home: e:\Program Files\Java\jdk-9.0.1
Default locale: en_US, platform encoding: Cp1250
OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< com.io7m.minimal:com.io7m.minimal
>------------------
[INFO] Building com.io7m.minimal 0.0.1
[INFO] --------------------------------[ jar
]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
com.io7m.minimal ---
[WARNING] Using platform encoding (Cp1250 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
d:\vcs\github\maven-minimal-20180413\src\main\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
com.io7m.minimal ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
@ com.io7m.minimal ---
[WARNING] Using platform encoding (Cp1250 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
d:\vcs\github\maven-minimal-20180413\src\test\resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @
com.io7m.minimal ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @
com.io7m.minimal ---
[INFO] No tests to run.
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 1.450 s
[INFO] Finished at: 2018-04-14T11:36:37+02:00
[INFO]
------------------------------------------------------------------------

On Fri, Apr 13, 2018 at 9:20 PM, Mark Raynsford <
[hidden email]> wrote:

> Hello.
>
> Is there any way I might speed up Maven? I've been using it for years
> and, for the first time, I'm actually executing it from an IDE rather
> than using the IDE's built-in compiler. It seems like there's a rather
> large and apparently fixed amount of overhead per invocation. Take the
> following "could not be simpler" project:
>
>   https://github.com/io7m/maven-minimal-20180413
>
> I piped the output through tai64n [1] to give a better indication of
> exactly how long everything is taking. That's a project that compiles a
> single Java file and produces a jar. It doesn't run any tests, it
> doesn't include any resources. 16 seconds total execution [2]!
>
> Now for an infrequent command-line build, that's not *too* obnoxious.
> For a "click a button in the IDE because I want to try this bit of code
> now" build, that's a world of suffering.
>
> It seems like Maven first spends ten entire seconds doing not very
> much. That cannot be attributed to JVM startup time (Hello World on my
> system takes about 100ms to reach user code, and even the largest Java
> application I still takes less than a second to reach user code after
> all the classes have loaded).
>
> Is there anything I can do to cut this down? If not, is this something
> that's intended to be addressed in the near future?
>
> [1] http://cr.yp.to/daemontools/tai64n.html
> [2] The Maven output claims 6 seconds, but that doesn't include the
>     time it took for the build to start.
>
> --
> Mark Raynsford | http://www.io7m.com
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Speeding up Maven

John Patrick
Mark,
Your project on mine machine takes on average 2.5 second and that is
also doing a clean package each time, not just package.

Spec:
MacBook Pro (Retina, 13-inch, Late 2013)
Processor 2.8 GHz Intel Core i7
Memory 16 GB 1600 MHz DDR3
Disk 512GB SSD

$ java -version
java version "9.0.4"
Java(TM) SE Runtime Environment (build 9.0.4+11)
Java HotSpot(TM) 64-Bit Server VM (build 9.0.4+11, mixed mode)
$ mvn -version
Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297;
2018-02-24T19:49:05Z)
Maven home: /opt/local/share/java/maven3
Java version: 9.0.4, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk-9.0.4.jdk/Contents/Home
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.13.4", arch: "x86_64", family: "mac"
$ mvn -Dmaven.tests.skip=true -DskipTests=true clean package 2>&1
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< com.io7m.minimal:com.io7m.minimal >------------------
[INFO] Building com.io7m.minimal 0.0.1
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ com.io7m.minimal ---
[INFO] Deleting /Users/john/projects/tmp/maven-minimal-20180413/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
com.io7m.minimal ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/Users/john/projects/tmp/maven-minimal-20180413/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
com.io7m.minimal ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding
UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to
/Users/john/projects/tmp/maven-minimal-20180413/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources
(default-testResources) @ com.io7m.minimal ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/Users/john/projects/tmp/maven-minimal-20180413/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile
(default-testCompile) @ com.io7m.minimal ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @
com.io7m.minimal ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ com.io7m.minimal ---
[INFO] Building jar:
/Users/john/projects/tmp/maven-minimal-20180413/target/com.io7m.minimal-0.0.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.515 s
[INFO] Finished at: 2018-04-14T11:13:18+01:00
[INFO] ------------------------------------------------------------------------
$ mvn -Dmaven.tests.skip=true -DskipTests=true clean package 2>&1
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< com.io7m.minimal:com.io7m.minimal >------------------
[INFO] Building com.io7m.minimal 0.0.1
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ com.io7m.minimal ---
[INFO] Deleting /Users/john/projects/tmp/maven-minimal-20180413/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
com.io7m.minimal ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/Users/john/projects/tmp/maven-minimal-20180413/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
com.io7m.minimal ---
[INFO] Changes detected - recompiling the module!
[WARNING] File encoding has not been set, using platform encoding
UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file to
/Users/john/projects/tmp/maven-minimal-20180413/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources
(default-testResources) @ com.io7m.minimal ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered
resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory
/Users/john/projects/tmp/maven-minimal-20180413/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.7.0:testCompile
(default-testCompile) @ com.io7m.minimal ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @
com.io7m.minimal ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ com.io7m.minimal ---
[INFO] Building jar:
/Users/john/projects/tmp/maven-minimal-20180413/target/com.io7m.minimal-0.0.1.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.576 s
[INFO] Finished at: 2018-04-14T11:13:23+01:00
[INFO] ------------------------------------------------------------------------
$ mvn -Dmaven.tests.skip=true -DskipTests=true clean package 2>&1

In my view maven can always be made faster, once all of maven and it's
plugins are using java modules, what is loaded into memory will be
reduced and it will naturally get faster.

John


On 14 April 2018 at 10:42, Tibor Digana <[hidden email]> wrote:

> This is interesting discussion and so I decided to build your project with
> Java 9 and Maven 3.5.3.
> Completed within 1.5 sec.
> I have noticed that Maven 3.3.9 was slower, cca 2.5 sec.
>
> mvn -V test
> Apache Maven 3.5.3 (3383c37e1f9e9b3bc3df5050c29c8aff9f295297;
> 2018-02-24T20:49:05+01:00)
> Maven home: D:\apache-maven-3.5.3
> Java version: 9.0.1, vendor: Oracle Corporation
> Java home: e:\Program Files\Java\jdk-9.0.1
> Default locale: en_US, platform encoding: Cp1250
> OS name: "windows 7", version: "6.1", arch: "amd64", family: "windows"
> [INFO] Scanning for projects...
> [INFO]
> [INFO] -----------------< com.io7m.minimal:com.io7m.minimal
>>------------------
> [INFO] Building com.io7m.minimal 0.0.1
> [INFO] --------------------------------[ jar
> ]---------------------------------
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
> com.io7m.minimal ---
> [WARNING] Using platform encoding (Cp1250 actually) to copy filtered
> resources, i.e. build is platform dependent!
> [INFO] skip non existing resourceDirectory
> d:\vcs\github\maven-minimal-20180413\src\main\resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @
> com.io7m.minimal ---
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-resources-plugin:2.6:testResources (default-testResources)
> @ com.io7m.minimal ---
> [WARNING] Using platform encoding (Cp1250 actually) to copy filtered
> resources, i.e. build is platform dependent!
> [INFO] skip non existing resourceDirectory
> d:\vcs\github\maven-minimal-20180413\src\test\resources
> [INFO]
> [INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @
> com.io7m.minimal ---
> [INFO] No sources to compile
> [INFO]
> [INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @
> com.io7m.minimal ---
> [INFO] No tests to run.
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 1.450 s
> [INFO] Finished at: 2018-04-14T11:36:37+02:00
> [INFO]
> ------------------------------------------------------------------------
>
> On Fri, Apr 13, 2018 at 9:20 PM, Mark Raynsford <
> [hidden email]> wrote:
>
>> Hello.
>>
>> Is there any way I might speed up Maven? I've been using it for years
>> and, for the first time, I'm actually executing it from an IDE rather
>> than using the IDE's built-in compiler. It seems like there's a rather
>> large and apparently fixed amount of overhead per invocation. Take the
>> following "could not be simpler" project:
>>
>>   https://github.com/io7m/maven-minimal-20180413
>>
>> I piped the output through tai64n [1] to give a better indication of
>> exactly how long everything is taking. That's a project that compiles a
>> single Java file and produces a jar. It doesn't run any tests, it
>> doesn't include any resources. 16 seconds total execution [2]!
>>
>> Now for an infrequent command-line build, that's not *too* obnoxious.
>> For a "click a button in the IDE because I want to try this bit of code
>> now" build, that's a world of suffering.
>>
>> It seems like Maven first spends ten entire seconds doing not very
>> much. That cannot be attributed to JVM startup time (Hello World on my
>> system takes about 100ms to reach user code, and even the largest Java
>> application I still takes less than a second to reach user code after
>> all the classes have loaded).
>>
>> Is there anything I can do to cut this down? If not, is this something
>> that's intended to be addressed in the near future?
>>
>> [1] http://cr.yp.to/daemontools/tai64n.html
>> [2] The Maven output claims 6 seconds, but that doesn't include the
>>     time it took for the build to start.
>>
>> --
>> Mark Raynsford | http://www.io7m.com
>>
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]