Re: Plugin dependency resolution

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Plugin dependency resolution

Tamás Cservenák
Just for fun, tried it on another, a bit bigger project (137 modules), and
as 3.6.3 and master was "so close", tried master and patch only:

mvn master (4.0.0-alpha-1, ab20190a1a9fecdc8f85b40e8d03d806f3da4fc6)
total time 1:49 min
total files in local repo: 7679
total bytes in local repo: 652624
count of files having "plexus-container-default" in local repo:  37

patched mvn master
total time 1:33 min
total files in local repo: 6531
total bytes in local repo: 624568
count of files having "plexus-container-default" in local repo: 0


Some preliminary conclusions: plexus-container-default count is SAME as
with Maven build, so it is most probably some "common" plugin(s) pulling it
in...
Again, around 1.1k unnecessary downloads prevented, which surprises me...

So, to me the "gain" looks constant.... so maybe not really worth the
effort?

T


On Wed, Feb 10, 2021 at 9:50 AM Tamás Cservenák <[hidden email]> wrote:

> Howdy,
>
> Some time ago I stumbled upon Robert S. remark in "[DISCUSS] MNG-7020
> Remove Maven 2 WagonExcluder backward compat code" thread: "Why download,
> if they are being removed from the classpath afterwards due to classworld
> config". Similarly, there is a thread "maven-site-plugin and
> sisu-inject-plexus" discussing that plugin pulls down ancient
> plexus-container-default (10+ years ago dropped). And finally, we all know
> about "maven downloads the whole internet" maxim :)
>
> Hence, I wanted to check this out. My "experiment" was not about maven
> "speed" (whatever you mean by it), but more about it's "bandwidth" usage.
>
> My setup:
> - am using (primed) MRM, not interested in bashing Central or measuring my
> ISP network speed
> - am always nuking local repo (hence, starting state is "get everything
> needed for build")
> - my test bed project is maven itself
> (master, ab20190a1a9fecdc8f85b40e8d03d806f3da4fc6)
> - build + tests ARE executed, and succeed OK, but I was really interested
> in local repository post-build state
> - command line am executing from maven checkout root is: rm -r /tmp/repo;
> ~/bin/maven/apache-maven-XXX/bin/mvn clean install
> -Dmaven.repo.local=/tmp/repo
>
> Remarks: times and the whole "experiment' is not scientific, they are just
> there for "rough comparison" sake. I did not repeat builds multiple times
> (to have some "mean time"), as I was not interested in time, but did repeat
> enough times to ensure that local repository state (file count, file sizes)
> are consistent.
>
> Results:
>
> maven 3.6.3 (released):
> total time 1:45 min (as Maven reports)
> total files in local repo: 3743
> total bytes in local repo: 162600
> count of files having "plexus-container-default" in local repo:  37 (18
> POM, 10 JAR)
>
> mvn master (4.0.0-alpha-1, ab20190a1a9fecdc8f85b40e8d03d806f3da4fc6):
> total time 1:08 min (as Maven reports)
> total files in local repo: 3741
> total bytes in local repo: 162428
> count of files having "plexus-container-default" in local repo:  37 (18
> POM, 10 JAR)
>
> so, pretty much the same. Then I went to create a (hacked for now) patch,
> as can be seen here:
>
> https://github.com/apache/maven/compare/master...cstamas:plugin-resolution-hack
>
> results with above patched mvn master:
> total time 1:04 min (as Maven reports)
> total files in local repo: 2992
> total bytes in local repo: 149740
> count of files having "plexus-container-default" in local repo: 0
>
> Basically patch "cut" almost 1k of unnecessary file downloads, 10+MB byte
> downloads in case of Maven being built.
>
> Is there some issue in JIRA covering this behaviour of Maven (I could not
> find any)?
>
> Have fun,
> Tamas
>
Reply | Threaded
Open this post in threaded view
|

Re: Plugin dependency resolution

Tamás Cservenák
Created https://issues.apache.org/jira/browse/MNG-7097

On Thu, Feb 11, 2021 at 8:41 AM Tamás Cservenák <[hidden email]> wrote:

> Hi Robert,
>
> I agree with you: is a really small change.
> Re non-exported packages: will take a look into those a bit more, my POC
> was just at "artifact level"... (but given Maven toss away downloaded
> plugin dependency, is plugin today able to use core artifact's non-exported
> package?)
> Yes, several maven bits, several plexus bits, and so on, it was just
> always annoying me :)
> Yup, will create a JIRA.
>
> Still, I'd really love to have more feedback, have people try the patch on
> more projects than I did (maven itself and one another) ;)
>
> HTH
> Tamas
>
> On Wed, Feb 10, 2021 at 10:33 PM Robert Scholte <[hidden email]>
> wrote:
>
>> Hi Tamas,
>>
>> based on the number of lines you wrote versus the amount of downloads
>> that are prevented (and storage) I think it is worth adding.
>> My biggest worry about this solution was about the non exported packages
>> of the exported artifacts. Would such changes have a negative and
>> inconsistent effect?
>> And I think it would make more sense for Maven users
>> If those people take a closer look at the downloaded file, they should
>> now wonder: my is maven-core 2.2.1, 3.0, 3.1.1 etc downloaded, while I'm
>> using Maven 3.6.3?
>>
>> AFAIK there's no ticket for it yet, these were just the ideas I could
>> think of regarding the WagonExcluder issue.
>> Looks like it is time to create it.
>>
>> thanks,
>> Robert
>>
>> On 10-2-2021 09:50:24, Tamás Cservenák <[hidden email]> wrote:
>> Howdy,
>>
>> Some time ago I stumbled upon Robert S. remark in "[DISCUSS] MNG-7020
>> Remove Maven 2 WagonExcluder backward compat code" thread: "Why download,
>> if they are being removed from the classpath afterwards due to classworld
>> config". Similarly, there is a thread "maven-site-plugin and
>> sisu-inject-plexus" discussing that plugin pulls down ancient
>> plexus-container-default (10+ years ago dropped). And finally, we all know
>> about "maven downloads the whole internet" maxim :)
>>
>> Hence, I wanted to check this out. My "experiment" was not about maven
>> "speed" (whatever you mean by it), but more about it's "bandwidth" usage.
>>
>> My setup:
>> - am using (primed) MRM, not interested in bashing Central or measuring my
>> ISP network speed
>> - am always nuking local repo (hence, starting state is "get everything
>> needed for build")
>> - my test bed project is maven itself
>> (master, ab20190a1a9fecdc8f85b40e8d03d806f3da4fc6)
>> - build + tests ARE executed, and succeed OK, but I was really interested
>> in local repository post-build state
>> - command line am executing from maven checkout root is: rm -r /tmp/repo;
>> ~/bin/maven/apache-maven-XXX/bin/mvn clean install
>> -Dmaven.repo.local=/tmp/repo
>>
>> Remarks: times and the whole "experiment' is not scientific, they are just
>> there for "rough comparison" sake. I did not repeat builds multiple times
>> (to have some "mean time"), as I was not interested in time, but did
>> repeat
>> enough times to ensure that local repository state (file count, file
>> sizes)
>> are consistent.
>>
>> Results:
>>
>> maven 3.6.3 (released):
>> total time 1:45 min (as Maven reports)
>> total files in local repo: 3743
>> total bytes in local repo: 162600
>> count of files having "plexus-container-default" in local repo: 37 (18
>> POM, 10 JAR)
>>
>> mvn master (4.0.0-alpha-1, ab20190a1a9fecdc8f85b40e8d03d806f3da4fc6):
>> total time 1:08 min (as Maven reports)
>> total files in local repo: 3741
>> total bytes in local repo: 162428
>> count of files having "plexus-container-default" in local repo: 37 (18
>> POM, 10 JAR)
>>
>> so, pretty much the same. Then I went to create a (hacked for now) patch,
>> as can be seen here:
>>
>> https://github.com/apache/maven/compare/master...cstamas:plugin-resolution-hack
>>
>> results with above patched mvn master:
>> total time 1:04 min (as Maven reports)
>> total files in local repo: 2992
>> total bytes in local repo: 149740
>> count of files having "plexus-container-default" in local repo: 0
>>
>> Basically patch "cut" almost 1k of unnecessary file downloads, 10+MB byte
>> downloads in case of Maven being built.
>>
>> Is there some issue in JIRA covering this behaviour of Maven (I could not
>> find any)?
>>
>> Have fun,
>> Tamas
>>
>