Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

Romain Manni-Bucau
(just cause I think it was not mentionned in this thread whereas it is very
relevant)

Here are some links converging with the overall idea:

-
https://cwiki.apache.org/confluence/display/MAVEN/Incremental+Builds?focusedCommentId=30149618
-
http://maven.40175.n5.nabble.com/Re-parallelise-not-overlapping-tasks-td5922723.html
- https://github.com/sudhanwadindorkar/incremental-maven-builder (which is
likely incomplete too but has some overall good idea around a potential
extension impl)

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le mer. 18 sept. 2019 à 09:53, Maximilian Novikov <[hidden email]>
a écrit :

> Classification: For internal use only
>
> Hi Falko,
>
> I saw this project.
> It can help in some cases, but to build fast you need:
>   1. Incremental build
>   2. Remote cache(shared cache)
>
> gitflow-incremental-builder helps to cover #1. BTW I still see limitations
> here:
>   1. It creates coupling with GIT
>   2. No IDE integration
>   3. Further advanced optimizations don't look possible
>
> Our idea was to move from workaround solutions(as Tibor classified this)
> to native solution.
>
> Thanks for sharing this.
>
> Kind regards,
> Max
>
>
> -----Original Message-----
> From: Falko Modler [mailto:[hidden email]]
> Sent: Wednesday, September 18, 2019 1:15 AM
> To: [hidden email]
> Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with
> local and remote caching
>
> Hi there,
>
> I must admit that I did not read everything but in case you are using Git
> this extension might help:
>
> https://github.com/vackosar/gitflow-incremental-builder
>
> It is a Maven extension and it is _not_ limited to Gitflow setups!
>
> Disclaimer: I am not the owner of this project but I am a "Collaborator"
> (I can cut releases etc.).
>
> Feedback is very much appreciated.
>
>
> PS: I hope this works, never posted to a mailing list before. :-O
>
>
> Best regards,
>
> Falko
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email] For additional
> commands, e-mail: [hidden email]
>
>
> ---
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and delete this e-mail. Any
> unauthorized copying, disclosure or distribution of the material in this
> e-mail is strictly forbidden.
>
> Please refer to https://www.db.com/disclosures for additional EU
> corporate and regulatory disclosures and to
> http://www.db.com/unitedkingdom/content/privacy.htm for information about
> privacy.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
Reply | Threaded
Open this post in threaded view
|

RE: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

Maximilian Novikov
Classification: For internal use only

Hi Enrico,

It's not open sourced for now. We need to spend much efforts according with DB policies/procedures to make it open source.
That's why we started this discussion, we don't want to waste time if change conceptually wrong(from Maven Dev Community perspective) and will be rejected.

To sum up discussion: no go for this change.

In some time we will wrap caching module in extension.

Regards,
Max

-----Original Message-----
From: Enrico Olivelli [mailto:[hidden email]]
Sent: Saturday, September 21, 2019 9:50 AM
To: Maven Developers List <[hidden email]>
Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

Hi Maximilian,
is there anyway to see this work ? is it already open source? (I am sorry, maybe I missed some email with links)

Enrico

Il giorno ven 20 set 2019 alle ore 19:30 Alexander Ashitkin < [hidden email]> ha scritto:

> Hi Martijn
> thanks for positive feedback.
>
> Regarding IDE part, yes you're right on integration part, but still
> there important cases when cache helps:
> 1) you need to navigate less in project as top level targets fast
> enough to not drill down
> 2) if you need to build a part of project (say only rest of wicket)
> you need to provide up-to-date rest dependencies which are not active
> in the subproject - and caches restores missing pieces for you without
> rebuilding remaining part of the project
> 3) If you need to test project and invoke test - cache saves your time
> (as gradle does) on unchanged pieces
> 4) and because tests run faster you can try run slow tests which often
> too expensive in rapid development
>
> So maven integration in Intellij works nice. There is nothing super
> smart here, just sharing how i benefit from the cache in everyday ide
> work
>
> Thank you!
>
> On 2019/09/19 11:28:48, Martijn Dashorst <[hidden email]>
> wrote:
> > On Thu, Sep 19, 2019 at 7:48 AM Alexander Ashitkin
> > <[hidden email]> wrote:
> > > Configuration:
> > > * verify -T4 -P default,all-shapshots-repos
> > > * my project config (might be suboptimal for wicket)
> > > * scala tests disabled in 2 modules (caused bytecode version
> > > conflict
> on my machine)
> > >
> > > Results
> > > Clean state (cache disabled):                           15:58 min
> > > Second run, target up to date (cache disabled):      10:20 min
> > > Fully cached (no changes):                                      17.507
> s
> > > wicketstuff-jwicket-tooltip-wtooltips changed:          34.936 s
> > > wicketstuff-rest-utils changed:                                 54.040
> s
> > >
> > > If you want to try other modules - please let me know.
> >
> > Nice results!
> >
> > > regarding ide - it's a usual maven installation, so any ide with
> > > maven
> integration should benefit from cache them maven action invoked
> >
> > My instinct says that an IDE as Eclipse won't benefit much from it,
> > as it has its own build lifecycle. Only when you invoke a
> > commandline Maven action (such as generate-sources) one might have a benefit.
> >
> > So in the day-to-day life the caching might not be as beneficial for
> > developers, but commandline builds happen often enough to make this
> > matter.
> >
> > Martijn
> >
> > --------------------------------------------------------------------
> > - To unsubscribe, e-mail: [hidden email] For
> > additional commands, e-mail: [hidden email]
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email] For
> additional commands, e-mail: [hidden email]
>
>


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]