Re: Reproducible Builds for Maven

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible Builds for Maven

Hervé BOUTEMY
Le lundi 23 septembre 2019, 05:56:06 CEST Tomo Suzuki a écrit :
> Sounds nice!
don't hesitate to build for yourself, check that you get the same sha512 and
report: this will help me either confirm "it works", or find little remaining
issues.

>
> > The precise result depends only on 2 key facts
>
> When I hear “reproducible builds”, I think of  lock files that remember
> library versions used.
> Gradle’s approach:
> https://docs.gradle.org/current/userguide/dependency_locking.html
>
> Does your approach use such file to record library versions?
no, we don't need such a lock file since we don't use version ranges: the
dependency resolution is already stable

Here, "Reproducible builds are a set of software development practices that
create an independently-verifiable path from source to binary code."
see https://reproducible-builds.org/

For Java, one key non-reproducible aspect for example is the timestamp of zip
entries in jar files.

Regards,

Hervé

>
> Regards,
> Tomo





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Reproducible Builds for Maven

Tomo Suzuki
Hi Mark,

Thank you for response.

> resolve highest org.jetbrains:annotations:[16.0.3,17.0.0) via public;

For reproducible builds, I expected the lock file contains specific
versions, rather than ranges. Would you share the background why your tool
records the ranges?


--
Regards,
Tomo
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible Builds for Maven

Mark Derricutt

On 24 Sep 2019, at 23:37, Tomo Suzuki wrote:

versions, rather than ranges. Would you share the background why your tool
records the ranges?

The full examples is at:

https://github.com/HalBuilder/halbuilder-support-4.x/blob/master/pom.deps

It resolves the locked down versions, but also retains the desired ranges for controlled updates.

We tend to keep ranges between major versions, i.e. [1.0.0,2.0.0) for a semblance of semver.

When I reresolve the bill of materials, I find I'll often look at the git diff and see what new versions of libraries have been updated, and decide which ( and when ) we pull them in to use - often committing those changes individually.


"The ease with which a change can be implemented has no relevance at all to whether it is the right change for the (Java) Platform for all time." — Mark Reinhold.

Mark Derricutt
http://www.theoryinpractice.net
http://www.chaliceofblood.net
http://plus.google.com/+MarkDerricutt
http://twitter.com/talios
http://facebook.com/mderricutt


signature.asc (546 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Reproducible Builds for Maven

Hervé BOUTEMY
In reply to this post by Hervé BOUTEMY
last updates:
- tar.gz archives are now also reproducible (in addition to .zip)
- src archives are also built and reproducible (notice that the result is the same on every JDK version of a platform. Notice 2: if you don't get the same result than CI, check that you don't have IDE configuration files that went into your local source archives...)
- artifacts built on ASF CI are available, for people to download and compare if you get a different result:
https://builds.apache.org/view/M-R/view/Maven/job/maven-box/job/maven/job/reproducible/lastSuccessfulBuild/artifact/org/apache/maven/apache-maven/3.6.3-SNAPSHOT/

I'll share shortly a discussion on a choice we need to do together to define how to configure reproducible builds (property name and value/format of current source-date-epoch defined in PoC)

Once this decision is made, we can start release packaging plugins that support "native" reproducible builds
https://reproducible-builds.org/

Regards,

Hervé

Le lundi 23 septembre 2019, 01:52:48 CEST Hervé BOUTEMY a écrit :

> after a few years of testing, thinking, procrastination and hard work (thank
> you Thomas for your talk at Devoxx France 2016 [1]), I think I achieved a
> key step this week-end toward native Reproducible Builds with Maven [2]:
> Maven core itself can be built in a reproducible way!
>
> It means that if you build "reproducible" branch of Maven core, you'll get
> the same apache-maven-3.6.3-SNAPSHOT-bin.zip than me or the ASF CI server
> [3]. The precise result depends only on 2 key facts:
> - do you build on Windows or any Unix? This impacts newlines...
> - what JDK major version do you use to build? This affects generated .class
> (notice: AFAIK minor JDK version does not have any impact, nor platform)
>
> This branch is only a PoC: it uses unreleased packaging plugins that give
> reproducible results (versions in .RB-SNAPSHOT), and I had to tweak a
> little bit the build for remaining reproduciblity issues with sisu and
> plexus plugins. There are many details to decide before releasing these
> plugins and making every build reproducible by default. But the current
> steps proves that is is feasible.
>
> Interested in joining the effort to bring this feature to releases for end
> users?
>
> Regards,
>
> Hervé
>
>
> [1]
> http://zlika.github.io/presentations/devoxx_fr_2016/reproducible-builds/sli
> des_fr.html
>
> [2]
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> [3]
> https://builds.apache.org/view/M-R/view/Maven/job/maven-box/job/maven/job/r
> eproducible/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]