Reproducible Builds for Maven

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Reproducible Builds for Maven

Hervé BOUTEMY
after a few years of testing, thinking, procrastination and hard work (thank you Thomas for your talk at Devoxx France 2016 [1]), I think I achieved a key step this week-end toward native Reproducible Builds with Maven [2]: Maven core itself can be built in a reproducible way!

It means that if you build "reproducible" branch of Maven core, you'll get the same apache-maven-3.6.3-SNAPSHOT-bin.zip than me or the ASF CI server [3].
The precise result depends only on 2 key facts:
- do you build on Windows or any Unix? This impacts newlines...
- what JDK major version do you use to build? This affects generated .class (notice: AFAIK minor JDK version does not have any impact, nor platform)

This branch is only a PoC: it uses unreleased packaging plugins that give reproducible results (versions in .RB-SNAPSHOT), and I had to tweak a little bit the build for remaining reproduciblity issues with sisu and plexus plugins.
There are many details to decide before releasing these plugins and making every build reproducible by default.
But the current steps proves that is is feasible.

Interested in joining the effort to bring this feature to releases for end users?

Regards,

Hervé


[1] http://zlika.github.io/presentations/devoxx_fr_2016/reproducible-builds/slides_fr.html

[2] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318

[3] https://builds.apache.org/view/M-R/view/Maven/job/maven-box/job/maven/job/reproducible/



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Reproducible Builds for Maven

Hervé BOUTEMY
Le mardi 24 septembre 2019, 02:28:15 CEST Mark Derricutt a écrit :
> Tomo Suzuki wrote on 23/09/19 3:56 PM:
> > Does your approach use such file to record library versions?
>
> I don't know about what Hervé is doing,
I added an "out of scope" paragraph: managing version ranges in a stable way is not in the scope
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318#Reproducible/VerifiableBuilds-Outofscope


> but internally we have a tool I
> wrote for handling this, we have a pom.deps file that looks like:
nice separation of concerns: stable versions chosen vs updates controlled with ranges

with such an approach, version ranges could become something I love :)

Regards,

Hervé

>
>      repository http://nexus.XXXXXX as public;
>
>      import smx3:smx3.upstream.bill-of-materials:1.1.22;
>
>      resolve highest org.jetbrains:annotations:[16.0.3,17.0.0) via public;
>      resolve highest
> org.apache.maven.plugins:maven-jar-plugin:[3.1.2,4.0.0) via public;
>      resolve highest org.apache.cxf:cxf-codegen-plugin:[3.3.3,4.0.0) via
> public;
>
> which when we resolve, will find the highest, snapshot, or lowest
> version in a given range - also allowing filtering out annoying things
> like beta/alpha/CR from central, and rewriting the pom.xml's.
>
> Our tooling also has an 'import' option shown above that lets us
> standardize the versions we resolving, and breaking it up - so we have
> 'upstream.bill-of-materials' and 'upstream.testing.bill-of-materials`.
>
> As part of this we also add in <exclusions> to ban all transitive build
> deps, and [] range all version references.
>
> I keep meaning to push for open sourcing it, but just haven't had the time.
>
> Mark





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]