Why no mvn daemon?
Locked
 
|
Hi guys,
Wonder why maven doesnt have any daemon and din't find anything satisfying to answer this question with our preferred search engine. Idea would be - a bit like gradle I guess - to cache computations and classloaders. Asking cause: - computing the reactor could be long on big projects and if the pom didnt' change between 2 builds it is useless - creating a classloader can be long but "filling" it - loadClass - is really longer for several plugins - big classloaders or dynamic languages - and doing it again and again is slow So the idea would be to run maven as a long running process (transparently through a -D in mvn opts if possible) and: - cache all is cacheable like the reactor etc while pom last modified date didnt change (OS needs to support it or natives could be used for OS where the JVM doesnt help) - cache classloaders and reuse them when re-executing a mojo - the communication between "mvn" which would then be a client to the daemon could just use a socket Of course if the pom changed the cache would just be invaildated but we would still gain the maven boot itself and we can still keep the classloader cache for plugins while their configuration is the sae (dependencies). Last thing missing in my description is two more configurations: a -Dxxx to kill the daemon and one to skip it for one build. Any technical/politics reason I miss to not have it? Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber <http://www.tomitribe.com> |
|
|
Maven Shell?
https://github.com/jdillon/mvnsh On Thu, Mar 3, 2016 at 2:13 PM Romain Manni-Bucau <[hidden email]> wrote: > Hi guys, > > Wonder why maven doesnt have any daemon and din't find anything satisfying > to answer this question with our preferred search engine. > > Idea would be - a bit like gradle I guess - to cache computations and > classloaders. > > Asking cause: > - computing the reactor could be long on big projects and if the pom didnt' > change between 2 builds it is useless > - creating a classloader can be long but "filling" it - loadClass - is > really longer for several plugins - big classloaders or dynamic languages - > and doing it again and again is slow > > So the idea would be to run maven as a long running process (transparently > through a -D in mvn opts if possible) and: > - cache all is cacheable like the reactor etc while pom last modified date > didnt change (OS needs to support it or natives could be used for OS where > the JVM doesnt help) > - cache classloaders and reuse them when re-executing a mojo > - the communication between "mvn" which would then be a client to the > daemon could just use a socket > > Of course if the pom changed the cache would just be invaildated but we > would still gain the maven boot itself and we can still keep the > classloader cache for plugins while their configuration is the sae > (dependencies). > > Last thing missing in my description is two more configurations: a -Dxxx to > kill the daemon and one to skip it for one build. > > Any technical/politics reason I miss to not have it? > > Romain Manni-Bucau > @rmannibucau <https://twitter.com/rmannibucau> | Blog > <http://rmannibucau.wordpress.com> | Github < > https://github.com/rmannibucau> | > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber > <http://www.tomitribe.com> > |
|
|
Good catch but something not changing user shell would be great and part of
maven distribution would be awesome. Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber <http://www.tomitribe.com> 2016-03-03 14:42 GMT+01:00 Tamás Cservenák <[hidden email]>: > Maven Shell? > > https://github.com/jdillon/mvnsh > > On Thu, Mar 3, 2016 at 2:13 PM Romain Manni-Bucau <[hidden email]> > wrote: > > > Hi guys, > > > > Wonder why maven doesnt have any daemon and din't find anything > satisfying > > to answer this question with our preferred search engine. > > > > Idea would be - a bit like gradle I guess - to cache computations and > > classloaders. > > > > Asking cause: > > - computing the reactor could be long on big projects and if the pom > didnt' > > change between 2 builds it is useless > > - creating a classloader can be long but "filling" it - loadClass - is > > really longer for several plugins - big classloaders or dynamic > languages - > > and doing it again and again is slow > > > > So the idea would be to run maven as a long running process > (transparently > > through a -D in mvn opts if possible) and: > > - cache all is cacheable like the reactor etc while pom last modified > date > > didnt change (OS needs to support it or natives could be used for OS > where > > the JVM doesnt help) > > - cache classloaders and reuse them when re-executing a mojo > > - the communication between "mvn" which would then be a client to the > > daemon could just use a socket > > > > Of course if the pom changed the cache would just be invaildated but we > > would still gain the maven boot itself and we can still keep the > > classloader cache for plugins while their configuration is the sae > > (dependencies). > > > > Last thing missing in my description is two more configurations: a -Dxxx > to > > kill the daemon and one to skip it for one build. > > > > Any technical/politics reason I miss to not have it? > > > > Romain Manni-Bucau > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > <http://rmannibucau.wordpress.com> | Github < > > https://github.com/rmannibucau> | > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Tomitriber > > <http://www.tomitribe.com> > > > |
|
|
In reply to this post by Tamás Cservenák
On March 3, 2016 at 5:42:56 AM, Tamás Cservenák ([hidden email]) wrote:
Maven Shell? https://github.com/jdillon/mvnsh Does anyone actually use this? —jason |
|
|
Some months ago I was looking for something like this. Due to this thread,
I will soon set it up and try it. (Thanks for sharing Tamás!) On Sat, Mar 5, 2016 at 10:31 PM, Jason Dillon <[hidden email]> wrote: > On March 3, 2016 at 5:42:56 AM, Tamás Cservenák ([hidden email]) > wrote: > Maven Shell? > > https://github.com/jdillon/mvnsh > > Does anyone actually use this? > > —jason > > |
|
|
FYI Just cloned and tried build but Jansi 1.2 is not in Central: "Could not
find artifact org.fusesource.jansi:jansi:jar:1.2". http://search.maven.org/#search|gav|1|g%3A%22org.fusesource.jansi%22%20AND%20a%3A%22jansi%22 I noted the issue tracker no longer exists so replying here to start. jansi is not declared in the POMs, so assuming transitive. Found it is only referenced by "mvnsh-commands/mvnsh-maven/src/main/java/org/sonatype/maven/shell/commands/maven/ColorizingStream.java" in mvnsh-commands/mvnsh-maven. I'm out of time to look further at the moment. Jason, if you have a built version, do you mind adding it as a download to the release files? Do you have interest in supporting further or prefer not to continue with it? On Sat, Mar 5, 2016 at 10:40 PM, Jeff Jensen < [hidden email]> wrote: > Some months ago I was looking for something like this. Due to this > thread, I will soon set it up and try it. (Thanks for sharing Tamás!) > > > On Sat, Mar 5, 2016 at 10:31 PM, Jason Dillon <[hidden email]> > wrote: > >> On March 3, 2016 at 5:42:56 AM, Tamás Cservenák ([hidden email]) >> wrote: >> Maven Shell? >> >> https://github.com/jdillon/mvnsh >> >> Does anyone actually use this? >> >> —jason >> >> > |
|
|
On March 8, 2016 at 7:45:53 AM, Jeff Jensen ([hidden email]) wrote:
FYI Just cloned and tried build but Jansi 1.2 is not in Central: "Could not find artifact org.fusesource.jansi:jansi:jar:1.2". http://search.maven.org/#search|gav|1|g%3A%22org.fusesource.jansi%22%20AND%20a%3A%22jansi%22 I noted the issue tracker no longer exists so replying here to start. jansi is not declared in the POMs, so assuming transitive. Found it is only referenced by "mvnsh-commands/mvnsh-maven/src/main/java/org/sonatype/maven/shell/commands/maven/ColorizingStream.java" in mvnsh-commands/mvnsh-maven. I'm out of time to look further at the moment. I’m aware of this. I took a brief look at upgrading to the latest jline but its a bit more work. There is another sonatype version of jline that may work better in the short-term until moving to the official jline2 stuff can be done. Jason, if you have a built version, do you mind adding it as a download to the release files? I can make a binary of this, though I do plan on fixing it up so that folks can build it in the near future. Do you have interest in supporting further or prefer not to continue with it? I’ve recently put in a bit more effort to bring this stuff up to speed w/changes to maven. I’m not sure I’ll be spending a ton of effort around it but I’m not sure how much effort is required. Certainly though if folks are actually using this then I’ll do what I can to support its continued use. There is another maven shell which may have more legs in terms of longer term community support, but I’m not yet sure where that is in terms of public availability. —jason |
|
|
Jason, if you have a built version, do you mind adding it as a download to
the release files? I can make a binary of this, though I do plan on fixing it up so that folks can build it in the near future. Build up here for the moment: https://www.dropbox.com/sh/yvt1g43r2pr2scj/AADqM__VhJaFf_x57OiUEZXva?dl=0 gshell:master should be buildable with just central now, dangling ref to older version of jline for classifer=tests which was unused and polluting the build dependencies. —jason |
|
|
In reply to this post by Jason Dillon-2
On Wed, Mar 9, 2016 at 3:59 PM, Jason Dillon <[hidden email]> wrote:
> On March 8, 2016 at 7:45:53 AM, Jeff Jensen ( > [hidden email]) wrote: > > FYI Just cloned and tried build but Jansi 1.2 is not in Central: "Could not > > find artifact org.fusesource.jansi:jansi:jar:1.2". > > http://search.maven.org/#search|gav|1|g%3A%22org.fusesource.jansi%22%20AND%20a%3A%22jansi%22 > <http://search.maven.org/#search%7Cgav%7C1%7Cg%3A%22org.fusesource.jansi%22%20AND%20a%3A%22jansi%22> > > > I noted the issue tracker no longer exists so replying here to start. > > jansi is not declared in the POMs, so assuming transitive. Found it is > only referenced by > > "mvnsh-commands/mvnsh-maven/src/main/java/org/sonatype/maven/shell/commands/maven/ColorizingStream.java" > > in mvnsh-commands/mvnsh-maven. > I'm out of time to look further at the moment. > > I’m aware of this. I took a brief look at upgrading to the latest jline > but its a bit more work. There is another sonatype version of jline that > may work better in the short-term until moving to the official jline2 stuff > can be done. > > > Jason, if you have a built version, do you mind adding it as a download to > > the release files? > > I can make a binary of this, though I do plan on fixing it up so that > folks can build it in the near future. > > Do you have interest in supporting further or prefer not to continue with > it? > > I’ve recently put in a bit more effort to bring this stuff up to speed > w/changes to maven. I’m not sure I’ll be spending a ton of effort around > it but I’m not sure how much effort is required. Certainly though if folks > are actually using this then I’ll do what I can to support its continued > use. > Thanks - my curiosity was around if this was "abandon-ware" or you may make it buildable again and do a few updates over the years... if not, then I would keep looking for one! There is another maven shell which may have more legs in terms of longer > term community support, but I’m not yet sure where that is in terms of > public availability. > Interesting, is it an internal Sonatype one or from somewhere else? |
|
|
In reply to this post by Jason Dillon-2
Thanks! It's running just fine. :-) It's very cool. Really nice
directory traversal and completion, colors, and commands/features I don't know yet! Very interesting timing diffs (for each casual test, I ran the command for each multiple times to seed infra caches): * on some asciidoc gen with "mvn generate-resources", it was about the same duration as CLI but after running each about 10 times, mvnshell saved about 20% consistently (possibly due to JIT? besides directory traversal, this was the first things I did). This was my key use case for wanting a "mvn server" - handling situations like this with repeated runs (asciidoc, site, etc.). * on a simple "mvn clean", mvnshell was about 2x faster than CLI. * on a small module build, "mvn install" was about 20% faster over CLI (after a mvn clean for each). I look forward to trying more things. Nice to have, thank you Jason! On Wed, Mar 9, 2016 at 4:17 PM, Jason Dillon <[hidden email]> wrote: > Jason, if you have a built version, do you mind adding it as a download to > > the release files? > > I can make a binary of this, though I do plan on fixing it up so that > folks can build it in the near future. > > Build up here for the moment: > > https://www.dropbox.com/sh/yvt1g43r2pr2scj/AADqM__VhJaFf_x57OiUEZXva?dl=0 > > gshell:master should be buildable with just central now, dangling ref to > older version of jline for classifer=tests which was unused and polluting > the build dependencies. > > —jason > > > |
|
|
The class cache is also available since Java 8 in the OpenJ9 VM, which also
has faster startup times in most cases (for me). Not sure if the scc is portable when using different Java language levels, though. But you can give it a try: https://www.eclipse.org/openj9/docs/shrc/ Other than that, I use mvnd at work and we never had problems on any project. We consistently save time without the need of adding the -T parameter manually. Another big question is probably: is there enough community demand to merge it into core? Tbh, I expected more mails on this thread. On Mon, 29 Mar 2021, 20:49 Romain Manni-Bucau, <[hidden email]> wrote: > Le lun. 29 mars 2021 à 19:26, Markus KARG <[hidden email]> a > écrit : > > > Looking at the impressively reduced bootstrap times of modern JDKs with > > features like Automatic AppCDS, I wonder if in the year 2021 this still > > makes sense? In rather near future there will be native precompilation > > available in most JDKs, effectively reducing bootstrap time to few ms. So > > is it worth investing this time and complexity? In the past, yes, up to > JDK > > 8, it definitively was. But NOW? > > > > Yep it makes sense more than ever IMHO because nothing actually changed for > maven: > > 1. CDS impact on maven is more or less 0 since maven bootstrap everything > in subclassloaders of the app loader and it is the only one able to benefit > from it (+ it is not the main benefit of a daemon, it also bypasses > resolution and all other steps the JVM will never do since it must be maven > aware) > 2. Making it GraalVM - ignoring the effort required to do it, run it on CI > (in time) and maintain it - will not help much for the same daemon goal > reason > > So is it worth doing a daemon -> clearly, this is what mvnd proves > (leveraging a mvnd client in native mode when possible btw). > > > > > > -Markus > > > > > > -----Ursprüngliche Nachricht----- > > Von: Romain Manni-Bucau [mailto:[hidden email]] > > Gesendet: Montag, 29. März 2021 12:21 > > An: Maven Developers List > > Cc: Jason Dillon > > Betreff: Re: Why no mvn daemon? > > > > Up 4 years later ;) > > > > Now mvnd exists and proves it is very interesting to have such a feature, > > should it be something which can fit maven standard delivery? > > If overall yes we can start by asking mvnd if it can be contributed with > > main codebase and if not we can either decide to do our own (hope not ;)) > > or that maven does not care about caching/optimizations in its "core" and > > that it is only done in extensions (I know 3 "main" ones as of today). > > > > Wdyt? > > > > Romain Manni-Bucau > > @rmannibucau <https://twitter.com/rmannibucau> | Blog > > <https://rmannibucau.metawerx.net/> | Old Blog > > <http://rmannibucau.wordpress.com> | Github < > > https://github.com/rmannibucau> | > > LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book > > < > > > https://www.packtpub.com/application-development/java-ee-8-high-performance > > > > > > > > > Le mer. 9 mars 2016 à 23:58, Jeff Jensen < > > [hidden email]> > > a écrit : > > > > > Thanks! It's running just fine. :-) It's very cool. Really nice > > > directory traversal and completion, colors, and commands/features I > don't > > > know yet! > > > > > > Very interesting timing diffs (for each casual test, I ran the command > > for > > > each multiple times to seed infra caches): > > > * on some asciidoc gen with "mvn generate-resources", it was about the > > > same duration as CLI but after running each about 10 times, mvnshell > > saved > > > about 20% consistently (possibly due to JIT? besides directory > traversal, > > > this was the first things I did). This was my key use case for > wanting a > > > "mvn server" - handling situations like this with repeated runs > > (asciidoc, > > > site, etc.). > > > > > > * on a simple "mvn clean", mvnshell was about 2x faster than CLI. > > > > > > * on a small module build, "mvn install" was about 20% faster over CLI > > > (after a mvn clean for each). > > > > > > I look forward to trying more things. > > > > > > Nice to have, thank you Jason! > > > > > > > > > On Wed, Mar 9, 2016 at 4:17 PM, Jason Dillon <[hidden email]> > > > wrote: > > > > > > > Jason, if you have a built version, do you mind adding it as a > download > > > to > > > > > > > > the release files? > > > > > > > > I can make a binary of this, though I do plan on fixing it up so that > > > > folks can build it in the near future. > > > > > > > > Build up here for the moment: > > > > > > > > > > > > > > https://www.dropbox.com/sh/yvt1g43r2pr2scj/AADqM__VhJaFf_x57OiUEZXva?dl=0 > > > > > > > > gshell:master should be buildable with just central now, dangling ref > > to > > > > older version of jline for classifer=tests which was unused and > > polluting > > > > the build dependencies. > > > > > > > > —jason > > > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [hidden email] > > For additional commands, e-mail: [hidden email] > > > > > |
«
Return to Maven Developers
|
1 view|%1 views
| Free forum by Nabble | Edit this page |