Why is old IO API used in maven resolver?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Why is old IO API used in maven resolver?

STEFAN REICH
Hi there!

I am working on a very large code base, and build performance issues made me look at the maven-resolver source code. In terms of File usages, there are a lot of InputStreams being copied around using ByteBuffer, instead of using FileChannel.transferTo. Affected classes are DefaultFileProcessor, ChecksumCalculator, WagonTransporter, AbstractTransporter and potentially more. Was it a conscious decision to use this pattern over the more efficient transferTo? Would you accept a PR with more modern NIO API that still works with JDK 7?

Here are the throughput results from a JMH benchmark, copying a 22MB file around using the pattern currently used in maven, and transferTo, as measured on macOS with JDK 11 on an SSD.

Result "MyBenchmark.resolverCopy":
  291.362 ±(99.9%) 5.443 ops/s [Average]
  (min, avg, max) = (276.923, 291.362, 302.911), stdev = 7.266
  CI (99.9%): [285.919, 296.804] (assumes normal distribution)

Result "MyBenchmark.transferTo":
  325.188 ±(99.9%) 8.838 ops/s [Average]
  (min, avg, max) = (306.978, 325.188, 355.784), stdev = 11.799
  CI (99.9%): [316.350, 334.026] (assumes normal distribution)


Thanks!
Stefan
Reply | Threaded
Open this post in threaded view
|

Re: Why is old IO API used in maven resolver?

olamy
On Sun, 16 Aug 2020 at 16:07, STEFAN REICH <[hidden email]> wrote:

> Hi there!
>
> I am working on a very large code base, and build performance issues made
> me look at the maven-resolver source code. In terms of File usages, there
> are a lot of InputStreams being copied around using ByteBuffer, instead of
> using FileChannel.transferTo. Affected classes are DefaultFileProcessor,
> ChecksumCalculator, WagonTransporter, AbstractTransporter and potentially
> more. Was it a conscious decision to use this pattern over the more
> efficient transferTo? Would you accept a PR with more modern NIO API that
> still works with JDK 7?
>

yes please.


> Here are the throughput results from a JMH benchmark, copying a 22MB file
> around using the pattern currently used in maven, and transferTo, as
> measured on macOS with JDK 11 on an SSD.
>
> Result "MyBenchmark.resolverCopy":
>   291.362 ±(99.9%) 5.443 ops/s [Average]
>   (min, avg, max) = (276.923, 291.362, 302.911), stdev = 7.266
>   CI (99.9%): [285.919, 296.804] (assumes normal distribution)
>
> Result "MyBenchmark.transferTo":
>   325.188 ±(99.9%) 8.838 ops/s [Average]
>   (min, avg, max) = (306.978, 325.188, 355.784), stdev = 11.799
>   CI (99.9%): [316.350, 334.026] (assumes normal distribution)
>
>
sounds like a good result.
Will it be the same for all OS? (windows, linux. osx)



>
> Thanks!
> Stefan



--
Olivier Lamy
http://twitter.com/olamy | http://linkedin.com/in/olamy
Reply | Threaded
Open this post in threaded view
|

Re: Why is old IO API used in maven resolver?

jochen-2
In reply to this post by STEFAN REICH
Looking at

    https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6822107

I would very much like us to abstain from every attempt to go into the
direction, that Stefan suggests. The consequence could be, that we
need to bother with exotic corner cases. Let's stay simple, but
reliable.

Jochen



On Sun, Aug 16, 2020 at 10:07 AM STEFAN REICH <[hidden email]> wrote:

>
> Hi there!
>
> I am working on a very large code base, and build performance issues made me look at the maven-resolver source code. In terms of File usages, there are a lot of InputStreams being copied around using ByteBuffer, instead of using FileChannel.transferTo. Affected classes are DefaultFileProcessor, ChecksumCalculator, WagonTransporter, AbstractTransporter and potentially more. Was it a conscious decision to use this pattern over the more efficient transferTo? Would you accept a PR with more modern NIO API that still works with JDK 7?
>
> Here are the throughput results from a JMH benchmark, copying a 22MB file around using the pattern currently used in maven, and transferTo, as measured on macOS with JDK 11 on an SSD.
>
> Result "MyBenchmark.resolverCopy":
>   291.362 ±(99.9%) 5.443 ops/s [Average]
>   (min, avg, max) = (276.923, 291.362, 302.911), stdev = 7.266
>   CI (99.9%): [285.919, 296.804] (assumes normal distribution)
>
> Result "MyBenchmark.transferTo":
>   325.188 ±(99.9%) 8.838 ops/s [Average]
>   (min, avg, max) = (306.978, 325.188, 355.784), stdev = 11.799
>   CI (99.9%): [316.350, 334.026] (assumes normal distribution)
>
>
> Thanks!
> Stefan



--

Look, that's why there's rules, understand? So that you think before
you break 'em.

    -- (Terry Pratchett, Thief of Time)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Why is old IO API used in maven resolver?

jochen-2
On Wed, Aug 19, 2020 at 11:49 AM Michael Osipov <[hidden email]> wrote:

>
> Am 2020-08-19 um 11:27 schrieb Jochen Wiedmann:
> > Looking at
> >
> >      https://bugs.java.com/bugdatabase/view_bug.do?bug_id=6822107
> >
> > I would very much like us to abstain from every attempt to go into the
> > direction, that Stefan suggests. The consequence could be, that we
> > need to bother with exotic corner cases. Let's stay simple, but
> > reliable.
>
>
> You proposal is not to change the code at all?

If it's working fine: Yes.

--

Look, that's why there's rules, understand? So that you think before
you break 'em.

    -- (Terry Pratchett, Thief of Time)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]