Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

Romain Manni-Bucau
Le sam. 14 sept. 2019 à 19:34, Maximilian Novikov <[hidden email]>
a écrit :

> Classification: For internal use only
>
> Ok, looks like this is the only option for us: create extension and
> override MojoExecutor.
>
> > The only challenge is an exhaustive test suite since your current impl
> can easily fake a passing build (as gradle does today if you don't disable
> the daemon and state cache on the CI).
> I assume we are safe here. Our solution provides incremental build with
> per project granularity. So there is no need in smart "test relationship
> discovery" within a project.
>

Well, for you I trust you, but not as a generic core maven feature so if it
stays like that it can't exit extension state IMHO.



> -----Original Message-----
> From: Romain Manni-Bucau [mailto:[hidden email]]
> Sent: Saturday, September 14, 2019 11:48 AM
> To: Maven Developers List <[hidden email]>
> Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with
> local and remote caching
>
> Le sam. 14 sept. 2019 à 08:00, Alexander Ashitkin <[hidden email]>
> a écrit :
>
> > Indeed we have a kind of the option 2 with variations. Current
> > implementation is opt-in feature driven by configuration with some
> > metadata of required cache behavior and hints.
> >
> > Maven extensions is the option, but we would love to have it in maven
> > itself which is in interest of maven community i believe. Extension is
> > a way we are trying to avoid and even not sure it could be implemented
> > as extension as it requires changes in maven core.
> >
>
> No real change required in maven core here since guice enables to override
> any bean or even just to rewrite the pom to remove modules to just rebuild
> the minimum set (keeping downstream project).
>
> The only challenge is an exhaustive test suite since your current impl can
> easily fake a passing build (as gradle does today if you dont disable the
> daemon and state cache on the CI).
>
> Side note: test relationship discovery is close to AOT in terms of impl
> and very very slow so can be worse than doing the full suite in simple
> projects and it still asks the IT question.
>
> So due to the numerous "?" of a core solution, extension is the way to go.
> Now if a guice bean in core can help to write your extension, it can
> surely be reviewed more easily IMHO.
>
> Hope it helps.
>
>
> > Thanks in advance, Aleks
> >
> > On 2019/09/13 21:37:15, Romain Manni-Bucau <[hidden email]>
> wrote:
> > > There are multiple possible incremental support:
> > >
> > > 1. Scm related: do a status and rebuild downstream reactor 2. Full
> > > and module build graph: seems it is the one you target, ie bypass
> > > modules without change. Note that it only works if upstream graph is
> > taken
> > > into account.
> > > 3. Full build: each mojo has incremental support so the full build
> > > gets
> > it.
> > > Issue is that it requires each mojo to know if it needs to be
> > > executed or give enough info to the mojo executor to do so (gradle
> > > requires all inputs/outputs to assume this state - which is still
> > > just an heuristic
> > and
> > > not 100% reliable).
> > >
> > > In current state, 2. sounds like a good option since 3 can require
> > > a
> > loot
> > > of work for external plugins (today's builds have a lot more of not
> > > maven provide plugins than core plugins).
> > > Now, we should be able to activate it or not so having a
> > > cacheLocation config in settings.xml can be good.
> > >
> > > Side notes:
> > >
> > > 1. having it on by default will break builds - reactor is
> > > deterministic
> > and
> > > bypassing a module can break a build since it can init maven
> > > properties - for ex - for next modules 2. You cant find all in/out
> > > paths from the pom in general so your algo is not generic, a meta
> > > config can be needed in .mvn 3. We should let a mojo be able to
> > > disable that to replace default logic (surefire is a good example
> > > where it must be refined and it can save
> > hours
> > > there ;))
> > > 4. Let's try to impl it as a mvn extension first then if it works
> > > well on multiple big project get it to core?
> > >
> > > Romain
> > >
> > >
> > >
> > > Le ven. 13 sept. 2019 à 23:18, Tibor Digana <[hidden email]>
> > > a écrit :
> > >
> > > > In theory, the incremental compiler would make it faster.
> > > > But this can be told only if you present a demo project with has
> > trivial
> > > > tests taking much less time to complete than the compiler.
> > > >
> > > > In reality the tests in huge projects take significantly longer
> > > > time
> > than
> > > > the compiler.
> > > > Some developers say "switch off all the tests" in the release
> > > > phase but that's wrong because then the quality goes down and
> > > > methodologies are broken.
> > > >
> > > > I can see a big problem that we do not have an interface between
> > Surefire
> > > > and Compiler plugin negotiating which tests have been modified
> > including
> > > > modules and classes in the entire structure.
> > > >
> > > > Having incremental compiler is easy, just use compiler:3.8.1 or
> > > > use the Takari compiler.
> > > > But IMO the biggest benefit in performance would be after having
> > > > the
> > truly
> > > > incremental test executor.
> > > >
> > > > On Fri, Sep 13, 2019 at 10:46 PM Maximilian Novikov <
> > > > [hidden email]> wrote:
> > > >
> > > > > Hi All,
> > > > >
> > > > >
> > > > >
> > > > > *We want to create upstream change to Maven* to support true
> > incremental
> > > > > build for big-sized projects.
> > > > >
> > > > > To raise a pull request we have to pass long chain of Deutsche
> > > > > Bank’s internal procedures. So, *before starting the process we
> > > > > would like
> > to
> > > > > get your feedback regarding this feature*.
> > > > >
> > > > >
> > > > >
> > > > > *Motivation:*
> > > > >
> > > > >
> > > > >
> > > > > Our project is hosted in mono-repo and contains ~600 modules.
> > > > > All
> > modules
> > > > > has the same SNAPSHOT version.
> > > > >
> > > > > There are lot of test automation around this, everything is
> > > > > tested
> > before
> > > > > merge into release branch.
> > > > >
> > > > >
> > > > >
> > > > > Current setup helps us to simplify build/release/dependency
> > management
> > > > for
> > > > > 10+ teams those contribute into codebase. We can release
> > > > > 10+ everything
> > in
> > > > > 1-click.
> > > > >
> > > > > The major drawback of such approach is build time: *full local
> > > > > build
> > took
> > > > > 45-60 min (*-T8)*, CI build ~25min(*-T16*)*.
> > > > >
> > > > >
> > > > >
> > > > > To speed-up our build we needed 2 features: incremental build
> > > > > and
> > shared
> > > > > cache.
> > > > >
> > > > > Initially we started to think about migration to Gradle or
> > > > > Bazel. As migration costs for the mentioned tools were too high,
> > > > > we decided to
> > add
> > > > > similar functionality into Maven.
> > > > >
> > > > >
> > > > >
> > > > > Current results we get: *1-2 mins for local build(*-T8*)* if
> > > > > build
> > was
> > > > > cached by CI*, CI build ~5 mins (*-T16*).*
> > > > >
> > > > >
> > > > >
> > > > > *Feature description:*
> > > > >
> > > > >
> > > > >
> > > > > The idea is to calculate checksum for inputs and save outputs in
> > cache.
> > > > >
> > > > > [image: image2019-8-27_20-0-14.png]
> > > > >
> > > > > Each node checksum calculated with:
> > > > >
> > > > >
> > > > >
> > > > > ·         Effective POM hash
> > > > >
> > > > > ·         Sources hash
> > > > >
> > > > > ·         Dependencies hash (dependencies within multi-module
> > project)
> > > > >
> > > > >
> > > > >
> > > > > Project sources inputs are searched inside project + all paths
> > > > > from plugins configuration:
> > > > >
> > > > > [image: image2019-8-30_10-28-56.png]
> > > > >
> > > > > How does it work in practice:
> > > > >
> > > > >
> > > > >
> > > > > 1.       CI: runs builds and stores outputs in shared cache
> > > > >
> > > > > 2.       CI: reuse outputs for same inputs, so time is decreasing
> > > > >
> > > > > 3.       Locally: when I checkout branch and run ‘install’ for
> whole
> > > > > project, I get all actual snapshots from remote cache for this
> > > > > branch
> > > > >
> > > > > 4.       Locally: if I change multiple modules in tree, only
> changed
> > > > > subtree is rebuilt
> > > > >
> > > > >
> > > > >
> > > > > Impact on current Maven codebase is very localized
> > > > > (MojoExecutor,
> > where
> > > > we
> > > > > injected cache controller).
> > > > >
> > > > > Caching can be activated/deactivated by property, so current
> > > > > maven
> > flow
> > > > > will work as is.
> > > > >
> > > > >
> > > > >
> > > > > And the big plus is that you don’t need to re-work your current
> > project.
> > > > > Caching should work out of box, just need to add config in .mvn
> > folder.
> > > > >
> > > > >
> > > > >
> > > > > Please let us know what do you think. We are ready to invest in
> > > > > this feature and address any further feedback.
> > > > >
> > > > >
> > > > >
> > > > > Kind regards,
> > > > >
> > > > > Max
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > ---
> > > > > This e-mail may contain confidential and/or privileged information.
> > If
> > > > you
> > > > > are not the intended recipient (or have received this e-mail in
> > error)
> > > > > please notify the sender immediately and delete this e-mail. Any
> > > > > unauthorized copying, disclosure or distribution of the material
> > > > > in
> > this
> > > > > e-mail is strictly forbidden.
> > > > >
> > > > > Please refer to https://www.db.com/disclosures for additional EU
> > > > > corporate and regulatory disclosures and to
> > > > > http://www.db.com/unitedkingdom/content/privacy.htm for
> > > > > information
> > > > about
> > > > > privacy.
> > > > >
> > > >
> > >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email] For
> > additional commands, e-mail: [hidden email]
> >
> >
>
>
> ---
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and delete this e-mail. Any
> unauthorized copying, disclosure or distribution of the material in this
> e-mail is strictly forbidden.
>
> Please refer to https://www.db.com/disclosures for additional EU
> corporate and regulatory disclosures and to
> http://www.db.com/unitedkingdom/content/privacy.htm for information about
> privacy.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
Reply | Threaded
Open this post in threaded view
|

Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

Romain Manni-Bucau
Le sam. 14 sept. 2019 à 19:11, Maximilian Novikov <[hidden email]>
a écrit :

> Classification: For internal use only
>
> It's moving to off-topic.
>
> Monorepo != single build
>

Agree but then how your figures relates to your repo? You exposed a single
build (or equivalent with downstream jobs) state.


Monorepo != monolithic application
>

Ack



> We moved from poly-repo to monorepo 3 years ago and see value in this.
> Let's say that this approach exists. It has own benefits/drawbacks
> comparing to poly-repo.
>

Ack

I am just trying to catch you on your particular case and original figures
which don't represent a monorepo but a monoproject build for now.

Goal is to ensure we fix it right for you too and not aside. Incremental
build is good but monorepo is another story where a quick win is inter
projects build (project vs mojo level to make it clear).



> -----Original Message-----
> From: Romain Manni-Bucau [mailto:[hidden email]]
> Sent: Saturday, September 14, 2019 7:42 PM
> To: Maven Developers List <[hidden email]>
> Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with
> local and remote caching
>
> Hope I didnt miss it but how monorepo=single build?
>
> It is working well to not have a common parent too and is unlinked to
> monorepo which uses local relative paths in general (at least in the
> references you quoted which are also not about java ;)).
>
> Unrelated to making maven better at incremental builds but both tracks can
> help you to get a very fast build feedback.
>
> Le sam. 14 sept. 2019 à 17:35, Robert Scholte <[hidden email]> a
> écrit :
>
> > https://issues.apache.org/jira/browse/MPLUGIN-350 is the issue to
> > start with.
> >
> > Please read all the comments, because my original thought won't work.
> >
> > thanks,
> > Robert
> >
> > On Sat, 14 Sep 2019 17:10:13 +0200, Alexander Ashitkin
> > <[hidden email]> wrote:
> >
> > > We checked and price of 550$ per user makes us think twice of what's
> > the
> > > best way to proceed here :-)
> > > Regarding plugin api - yes, changes are desirable to make maven
> > > model cache-friendly. Both in plugin invocation model and
> > > Mojo#execute input/output apis. But it is possible to work with
> > > current model with declarative approach.
> > >
> > > Thanks in advance
> > >
> > > On 2019/09/14 10:45:24, Tibor Digana <[hidden email]> wrote:
> > >> But I do not understand why the Maven should be responsible for the
> > >> project cahe control/management of "/target" directories.
> > >> It is a responsibility of the build manager which is the Jenkins.
> > >> The Jenkins has the ability to archive files and such property
> > >> already exists in the Jenkins.
> > >>
> > >> So the Jenkins has a full knowledge about:
> > >>
> > >> 1. how long the workspace content retains intact 2. what commit
> > >> hash is for the last build/job/branch 3. and what commit was
> > >> successful
> > >>
> > >> If the target directories retain intact (or renewed from archive)
> > >> in the workspace for very long time and the workspace was reused by
> > >> the next build then I would say that the improvement should work as
> > >> it is on CI level.
> > >>
> > >> Maybe what is necessary is only that improvement in Maven where we
> > >> would obtain the list of modules or directories of changes in the
> > >> current commit.
> > >> Then the Maven can highly optimize its own build steps and build
> > >> only those modules which have been changed and their dependent
> > >> modules.
> > >> So the interface between CI and Maven is needed in a kind of
> > >> extension or the class MavenCli can be extended with some new
> > >> entrypoint.
> > >>
> > >> But I do not hink that Maven has to take care of responsibilities
> > >> of CI (project cache mgmt), that's not our task I would say and we
> > >> as Maven would never know all about the miscellaneous CI specifics
> > >> and therefore we would not cope with CI related troubles.
> > >>
> > >> Cheers
> > >> Tibor17
> > >>
> > >>
> > >>
> > >> On Sat, Sep 14, 2019 at 11:08 AM Robert Scholte
> > >> <[hidden email]>
> > >> wrote:
> > >>
> > >> > On Fri, 13 Sep 2019 23:37:15 +0200, Romain Manni-Bucau
> > >> > <[hidden email]> wrote:
> > >> >
> > >> > > There are multiple possible incremental support:
> > >> > >
> > >> > > 1. Scm related: do a status and rebuild downstream reactor 2.
> > >> > > Full and module build graph: seems it is the one you target, ie
> > >> bypass
> > >> > > modules without change. Note that it only works if upstream
> > >> > > graph is taken into account.
> > >> > > 3. Full build: each mojo has incremental support so the full
> > >> > > build
> > >> gets
> > >> > > it.
> > >> > > Issue is that it requires each mojo to know if it needs to be
> > >> executed or
> > >> > > give enough info to the mojo executor to do so (gradle requires
> > >> > > all inputs/outputs to assume this state - which is still just
> > >> > > an
> > >> heuristic
> > >> > > and
> > >> > > not 100% reliable).
> > >> > >
> > >> > > In current state, 2. sounds like a good option since 3 can
> > >> > > require
> >
> > >> a
> > >> > > loot
> > >> > > of work for external plugins (today's builds have a lot more of
> > not
> > >> maven
> > >> > > provide plugins than core plugins).
> > >> > > Now, we should be able to activate it or not so having a
> > >> cacheLocation
> > >> > > config in settings.xml can be good.
> > >> > >
> > >> > > Side notes:
> > >> > >
> > >> > > 1. having it on by default will break builds - reactor is
> > >> deterministic
> > >> > > and
> > >> > > bypassing a module can break a build since it can init maven
> > >> properties -
> > >> > > for ex - for next modules
> > >> > > 2. You cant find all in/out paths from the pom in general so
> > >> > > your
> > >> algo is
> > >> > > not generic, a meta config can be needed in .mvn 3. We should
> > >> > > let a mojo be able to disable that to replace default
> > >> logic
> > >> > > (surefire is a good example where it must be refined and it can
> > >> > > save hours there ;)) 4. Let's try to impl it as a mvn extension
> > >> > > first then if it works
> > >> well on
> > >> > > multiple big project get it to core?
> > >> >
> > >> > Did anyone Google for "maven extension build cache"? There are
> > >> > already commercial solutions for it.
> > >> > Even though I would like to see improvements in this area, the
> > >> > old architecture of Maven makes it quite hard to move to that
> situation.
> > >> > First
> > >> > of all it requires changes to the Plugin API (without breaking
> > >> backwards
> > >> > compatibility) to have support out of the box.
> > >> >
> > >> > Robert
> > >> >
> > >> > >
> > >> > > Romain
> > >> > >
> > >> > >
> > >> > >
> > >> > > Le ven. 13 sept. 2019 à 23:18, Tibor Digana
> > >> <[hidden email]> a
> > >> > > écrit :
> > >> > >
> > >> > >> In theory, the incremental compiler would make it faster.
> > >> > >> But this can be told only if you present a demo project with
> > >> > >> has
> > >> trivial
> > >> > >> tests taking much less time to complete than the compiler.
> > >> > >>
> > >> > >> In reality the tests in huge projects take significantly
> > >> > >> longer
> > >> time
> > >> > >> than
> > >> > >> the compiler.
> > >> > >> Some developers say "switch off all the tests" in the release
> > >> phase but
> > >> > >> that's wrong because then the quality goes down and
> > >> > >> methodologies
> > >> are
> > >> > >> broken.
> > >> > >>
> > >> > >> I can see a big problem that we do not have an interface
> > >> > >> between Surefire and Compiler plugin negotiating which tests
> > >> > >> have been modified
> > >> including
> > >> > >> modules and classes in the entire structure.
> > >> > >>
> > >> > >> Having incremental compiler is easy, just use compiler:3.8.1
> > >> > >> or
> > >> use the
> > >> > >> Takari compiler.
> > >> > >> But IMO the biggest benefit in performance would be after
> > >> > >> having
> > >> the
> > >> > >> truly
> > >> > >> incremental test executor.
> > >> > >>
> > >> > >> On Fri, Sep 13, 2019 at 10:46 PM Maximilian Novikov <
> > >> > >> [hidden email]> wrote:
> > >> > >>
> > >> > >> > Hi All,
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > *We want to create upstream change to Maven* to support true
> > >> > >> incremental
> > >> > >> > build for big-sized projects.
> > >> > >> >
> > >> > >> > To raise a pull request we have to pass long chain of
> > >> > >> > Deutsche
> > >> Bank’s
> > >> > >> > internal procedures. So, *before starting the process we
> > >> > >> > would
> > >> like to
> > >> > >> > get your feedback regarding this feature*.
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > *Motivation:*
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > Our project is hosted in mono-repo and contains ~600
> > >> > >> > modules. All
> > >> > >> modules
> > >> > >> > has the same SNAPSHOT version.
> > >> > >> >
> > >> > >> > There are lot of test automation around this, everything is
> > >> tested
> > >> > >> before
> > >> > >> > merge into release branch.
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > Current setup helps us to simplify build/release/dependency
> > >> management
> > >> > >> for
> > >> > >> > 10+ teams those contribute into codebase. We can release
> > >> everything in
> > >> > >> > 1-click.
> > >> > >> >
> > >> > >> > The major drawback of such approach is build time: *full
> > >> > >> > local
> > >> build
> > >> > >> took
> > >> > >> > 45-60 min (*-T8)*, CI build ~25min(*-T16*)*.
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > To speed-up our build we needed 2 features: incremental
> > >> > >> > build and
> > >> > >> shared
> > >> > >> > cache.
> > >> > >> >
> > >> > >> > Initially we started to think about migration to Gradle or
> > >> Bazel. As
> > >> > >> > migration costs for the mentioned tools were too high, we
> > >> decided to
> > >> > >> add
> > >> > >> > similar functionality into Maven.
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > Current results we get: *1-2 mins for local build(*-T8*)* if
> > >> build was
> > >> > >> > cached by CI*, CI build ~5 mins (*-T16*).*
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > *Feature description:*
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > The idea is to calculate checksum for inputs and save
> > >> > >> > outputs in
> > >> > >> cache.
> > >> > >> >
> > >> > >> > [image: image2019-8-27_20-0-14.png]
> > >> > >> >
> > >> > >> > Each node checksum calculated with:
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > ·         Effective POM hash
> > >> > >> >
> > >> > >> > ·         Sources hash
> > >> > >> >
> > >> > >> > ·         Dependencies hash (dependencies within multi-module
> > >> project)
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > Project sources inputs are searched inside project + all
> > >> > >> > paths
> > >> from
> > >> > >> > plugins configuration:
> > >> > >> >
> > >> > >> > [image: image2019-8-30_10-28-56.png]
> > >> > >> >
> > >> > >> > How does it work in practice:
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > 1.       CI: runs builds and stores outputs in shared cache
> > >> > >> >
> > >> > >> > 2.       CI: reuse outputs for same inputs, so time is
> decreasing
> > >> > >> >
> > >> > >> > 3.       Locally: when I checkout branch and run ‘install’ for
> > >> whole
> > >> > >> > project, I get all actual snapshots from remote cache for
> > >> > >> > this
> > >> branch
> > >> > >> >
> > >> > >> > 4.       Locally: if I change multiple modules in tree, only
> > >> changed
> > >> > >> > subtree is rebuilt
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > Impact on current Maven codebase is very localized
> > >> > >> > (MojoExecutor,
> > >> > >> where
> > >> > >> we
> > >> > >> > injected cache controller).
> > >> > >> >
> > >> > >> > Caching can be activated/deactivated by property, so current
> > >> maven
> > >> > >> flow
> > >> > >> > will work as is.
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > And the big plus is that you don’t need to re-work your
> > >> > >> > current
> > >> > >> project.
> > >> > >> > Caching should work out of box, just need to add config in
> > >> > >> > .mvn
> > >> > >> folder.
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > Please let us know what do you think. We are ready to invest
> > >> > >> > in
> > >> this
> > >> > >> > feature and address any further feedback.
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > Kind regards,
> > >> > >> >
> > >> > >> > Max
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> >
> > >> > >> > ---
> > >> > >> > This e-mail may contain confidential and/or privileged
> > >> information. If
> > >> > >> you
> > >> > >> > are not the intended recipient (or have received this e-mail
> > >> > >> > in
> > >> error)
> > >> > >> > please notify the sender immediately and delete this e-mail.
> > >> > >> > Any unauthorized copying, disclosure or distribution of the
> > material
> > >> in
> > >> > >> this
> > >> > >> > e-mail is strictly forbidden.
> > >> > >> >
> > >> > >> > Please refer to https://www.db.com/disclosures for
> > >> > >> > additional EU corporate and regulatory disclosures and to
> > >> > >> > http://www.db.com/unitedkingdom/content/privacy.htm for
> > >> information
> > >> > >> about
> > >> > >> > privacy.
> > >> > >> >
> > >> >
> > >> > -----------------------------------------------------------------
> > >> > ---- To unsubscribe, e-mail: [hidden email] For
> > >> > additional commands, e-mail: [hidden email]
> > >> >
> > >> >
> > >>
> > >
> > > --------------------------------------------------------------------
> > > - To unsubscribe, e-mail: [hidden email] For
> > > additional commands, e-mail: [hidden email]
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email] For
> > additional commands, e-mail: [hidden email]
> >
> >
>
>
> ---
> This e-mail may contain confidential and/or privileged information. If you
> are not the intended recipient (or have received this e-mail in error)
> please notify the sender immediately and delete this e-mail. Any
> unauthorized copying, disclosure or distribution of the material in this
> e-mail is strictly forbidden.
>
> Please refer to https://www.db.com/disclosures for additional EU
> corporate and regulatory disclosures and to
> http://www.db.com/unitedkingdom/content/privacy.htm for information about
> privacy.
>
Reply | Threaded
Open this post in threaded view
|

RE: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

Maximilian Novikov
In reply to this post by Romain Manni-Bucau
Classification: For internal use only

Hi Falko,

I saw this project.
It can help in some cases, but to build fast you need:
  1. Incremental build
  2. Remote cache(shared cache)

gitflow-incremental-builder helps to cover #1. BTW I still see limitations here:
  1. It creates coupling with GIT
  2. No IDE integration
  3. Further advanced optimizations don't look possible

Our idea was to move from workaround solutions(as Tibor classified this) to native solution.

Thanks for sharing this.

Kind regards,
Max


-----Original Message-----
From: Falko Modler [mailto:[hidden email]]
Sent: Wednesday, September 18, 2019 1:15 AM
To: [hidden email]
Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching

Hi there,

I must admit that I did not read everything but in case you are using Git this extension might help:

https://github.com/vackosar/gitflow-incremental-builder

It is a Maven extension and it is _not_ limited to Gitflow setups!

Disclaimer: I am not the owner of this project but I am a "Collaborator"
(I can cut releases etc.).

Feedback is very much appreciated.


PS: I hope this works, never posted to a mailing list before. :-O


Best regards,

Falko


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: [VOTE] Maven incremental build for BIG-sized projects with local and remote caching [I]

Falko Modler
Hi Maximilian,

> 2. No IDE integration

IDEs usually have their own mechanisms to build incrementally. They also execute Maven core in their own special way, often very different from the default command line execution.

> 3. Further advanced optimizations don't look possible

Feature requests are welcome! :-)

Otherwise I wish you good luck on your path.

Best regards,

Falko


Am 18. September 2019 09:52:52 MESZ schrieb Maximilian Novikov <[hidden email]>:

>Classification: For internal use only
>
>Hi Falko,
>
>I saw this project.
>It can help in some cases, but to build fast you need:
>  1. Incremental build
>  2. Remote cache(shared cache)
>
>gitflow-incremental-builder helps to cover #1. BTW I still see
>limitations here:
>  1. It creates coupling with GIT
>  2. No IDE integration
>  3. Further advanced optimizations don't look possible
>
>Our idea was to move from workaround solutions(as Tibor classified
>this) to native solution.
>
>Thanks for sharing this.
>
>Kind regards,
>Max
>
>
>-----Original Message-----
>From: Falko Modler [mailto:[hidden email]]
>Sent: Wednesday, September 18, 2019 1:15 AM
>To: [hidden email]
>Subject: Re: [VOTE] Maven incremental build for BIG-sized projects with
>local and remote caching
>
>Hi there,
>
>I must admit that I did not read everything but in case you are using
>Git this extension might help:
>
>https://github.com/vackosar/gitflow-incremental-builder
>
>It is a Maven extension and it is _not_ limited to Gitflow setups!
>
>Disclaimer: I am not the owner of this project but I am a
>"Collaborator"
>(I can cut releases etc.).
>
>Feedback is very much appreciated.
>
>
>PS: I hope this works, never posted to a mailing list before. :-O
>
>
>Best regards,
>
>Falko
>
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [hidden email] For additional
>commands, e-mail: [hidden email]
>
>
>---
>This e-mail may contain confidential and/or privileged information. If
>you are not the intended recipient (or have received this e-mail in
>error) please notify the sender immediately and delete this e-mail. Any
>unauthorized copying, disclosure or distribution of the material in
>this e-mail is strictly forbidden.
>
>Please refer to https://www.db.com/disclosures for additional EU
>corporate and regulatory disclosures and to
>http://www.db.com/unitedkingdom/content/privacy.htm for information
>about privacy.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [hidden email]
>For additional commands, e-mail: [hidden email]
>>