Re: last review of Reproducible Builds proposal

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Tibor Digana
Hi Herve,

I want to make sure we understand correctly. So. What you want to achieve
with this property is to stick the property to one fixed value when the
user has supposed the archive would have same content. And opposite, means
that the property would be real when the content of the archive should
change.

That's technically ok that the plugins are aware of shared time stamp, but
this looks like the problem with egg and chicken. Practically, the user
would never be sure when is the time that the time stamp should be fixed
value. Not sure if the user would be able to have fully working
reproducible build.
Perhaps the users will have exactly this question too.

Cheers
Tibor17



On Sat, Oct 5, 2019 at 7:52 PM Hervé BOUTEMY <[hidden email]> wrote:

> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with
> project.build.outputTimestamp property, in ISO-8601 format
> <project>
>   <properties>
>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
>   </properties>
> </project>
>
> The shared components, plugins, parent poms and Maven core branches have
> been updated to match this new proposal
>
>
> If no one objects, next week-end, I'll start the (heavy) release train to
> bring (binary) Reproducible Builds plugins to general availability
>
> Regards,
>
> Hervé
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Hervé BOUTEMY
with inheritance from parent, this question disappears: once parent pom has a
timestamp value to have a reproducible release, child poms inherit the value
and the reproducible feature
changing the value in child is just a little improvement to get a timestamp
that has more meaningful value, if anybody cares about the exact value: but
who really looks at the timestamp of entries in release zips/jars/tar.gz
honestly?


Regards,

Hervé

Le samedi 5 octobre 2019, 20:42:04 CEST Tibor Digana a écrit :

> Hi Herve,
>
> I want to make sure we understand correctly. So. What you want to achieve
> with this property is to stick the property to one fixed value when the
> user has supposed the archive would have same content. And opposite, means
> that the property would be real when the content of the archive should
> change.
>
> That's technically ok that the plugins are aware of shared time stamp, but
> this looks like the problem with egg and chicken. Practically, the user
> would never be sure when is the time that the time stamp should be fixed
> value. Not sure if the user would be able to have fully working
> reproducible build.
> Perhaps the users will have exactly this question too.
>
> Cheers
> Tibor17
>
> On Sat, Oct 5, 2019 at 7:52 PM Hervé BOUTEMY <[hidden email]> wrote:
> > based on the feedback I got, I updated the proposal:
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> >
> > The archives entries timestamp is now configured with
> > project.build.outputTimestamp property, in ISO-8601 format
> > <project>
> >
> >   <properties>
> >
> > <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputT
> > imestamp>>
> >   </properties>
> >
> > </project>
> >
> > The shared components, plugins, parent poms and Maven core branches have
> > been updated to match this new proposal
> >
> >
> > If no one objects, next week-end, I'll start the (heavy) release train to
> > bring (binary) Reproducible Builds plugins to general availability
> >
> > Regards,
> >
> > Hervé
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Michael Osipov-2
In reply to this post by Tibor Digana
Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:

> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
>>> based on the feedback I got, I updated the proposal:
>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>>>
>>> The archives entries timestamp is now configured with
>>> project.build.outputTimestamp property, in ISO-8601 format <project>
>>>
>>>     <properties>
>>>    
>>>       <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.ou
>>>       tputTimestamp>>
>>>     </properties>
>>>
>>> </project>
>>>
>>> The shared components, plugins, parent poms and Maven core branches have
>>> been updated to match this new proposal
>>>
>>>
>>> If no one objects, next week-end, I'll start the (heavy) release train to
>>> bring (binary) Reproducible Builds plugins to general availability
>> Really, really nice work..
> thank you, it required a lot of energy for a long period of time...
>
>>
>> I do like <project.build.outputTimestamp>. Why did you make it a String?
>> Why not go directly with Instant? It gives your ISO 8601 for free
> I tried to explain it in https://issues.apache.org/jira/browse/MSHARED-837
> Plexus Date injection support is really limited: could not match the
> requirements

yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
will require to change the converter of course.

Alternatively, you could try
https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html/org/apache/commons/lang3/time/DateFormatUtils.html#line.72 
which I have added long time ago.

Moreover, I'd like these updated plugins be part of MNG-6169.

Michael


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Emmanuel Bourg
In reply to this post by Tibor Digana
Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit :
> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
> <project>
>   <properties>
>     <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
>   </properties>
> </project>

You may want to specify explicitly that the timestamp must be formatted
with the UTC timezone (as in your example).

Would it be possible to prevent this property from being inherited?

Emmanuel Bourg

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Hervé BOUTEMY
In reply to this post by Hervé BOUTEMY
Yes, having the timestamp to 0 is something I wanted generally to avoid.
But this has opened another question: use what value? Can this be automated?

I knew that war files could be a specific use case.
Perhaps this plugin requires a specific way of handling reproducibility, even
more than the standard in the proposal "define a  timestamp in pom.xml"?

Do you want that I create a Reproducible branch in mave-war-plugin like I did
for Jar, Source and Assembly, so we can test?

Regards,

Hervé

Le samedi 5 octobre 2019, 22:19:54 CEST Vladimir Sitnikov a écrit :

> >but
> >who really looks at the timestamp of entries in release zips/jars/tar.gz
> >honestly?
>
> Tomcat when it decides on what to send in the "Last-Modified" header.
> For instance, current Gradle does not allow to configure the timestamp, and
> for reproducible builds it always sets the timestamp to 0 or so.
> It breaks Tomcat's assumptions:
> https://github.com/gradle/gradle/issues/10917
>
> Vladimir





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Hervé BOUTEMY
In reply to this post by Michael Osipov-2
Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :

> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
> > Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
> >> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
> >>> based on the feedback I got, I updated the proposal:
> >>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=7468231
> >>> 8
> >>>
> >>> The archives entries timestamp is now configured with
> >>> project.build.outputTimestamp property, in ISO-8601 format <project>
> >>>
> >>>     <properties>
> >>>    
> >>>       <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build
> >>>       .ou
> >>>       tputTimestamp>>
> >>>    
> >>>     </properties>
> >>>
> >>> </project>
> >>>
> >>> The shared components, plugins, parent poms and Maven core branches have
> >>> been updated to match this new proposal
> >>>
> >>>
> >>> If no one objects, next week-end, I'll start the (heavy) release train
> >>> to
> >>> bring (binary) Reproducible Builds plugins to general availability
> >>
> >> Really, really nice work..
> >
> > thank you, it required a lot of energy for a long period of time...
> >
> >> I do like <project.build.outputTimestamp>. Why did you make it a String?
> >> Why not go directly with Instant? It gives your ISO 8601 for free
> >
> > I tried to explain it in https://issues.apache.org/jira/browse/MSHARED-837
> > Plexus Date injection support is really limited: could not match the
> > requirements
>
> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
> will require to change the converter of course.
>
> Alternatively, you could try
> https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html
> /org/apache/commons/lang3/time/DateFormatUtils.html#line.72 which I have
> added long time ago.
the question is not about coding the date format: it's already done in maven-archiver
https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a46fbd4c78445301

the question, as you perfectly asked initially, is: do you have an idea on how to have the direct Date injection, instead of having a String in the plugin parameter?

>
> Moreover, I'd like these updated plugins be part of MNG-6169.
sure, having native Reproducible Builds by default is a good feature for Maven Core.
Even if there will be a warning since people need to define the plugin version in their pom to avoid dependency on Maven core version: https://issues.apache.org/jira/browse/MNG-6562 

>
> Michael
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Hervé BOUTEMY
In reply to this post by Emmanuel Bourg
Le samedi 5 octobre 2019, 23:15:58 CEST Emmanuel Bourg a écrit :

> Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit :
> > based on the feedback I got, I updated the proposal:
> > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> >
> > The archives entries timestamp is now configured with
> > project.build.outputTimestamp property, in ISO-8601 format <project>
> >
> >   <properties>
> >  
> >     <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.out
> >     putTimestamp>>  
> >   </properties>
> >
> > </project>
>
> You may want to specify explicitly that the timestamp must be formatted
> with the UTC timezone (as in your example).
the current documentation on the plugin parameter is "Timestamp for reproducible archive entries."
https://github.com/apache/maven-source-plugin/commit/1279fe02564d805e21d75cbfc8408178190fe12a
Don't hesitate to propose a better explanation: once ok, I'll add the result to the proposal, to be easily copy/pasted to every packaging plugin.

Notice that you can also express a timezone (as digits), as seen in the unit tests.

>
> Would it be possible to prevent this property from being inherited?
AFAIK no.
And I find that default inheritance from parent to child is a nice feature.
What I added, in specific case where inheritance could cause an issue, is that you can disable the Reproducible Builds configuration by setting a 1 character value, whatever the character is.
This is the only trick I found to have inheritance not ignore child pom's value: tested also in the unit tests

>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Michael Osipov-2
In reply to this post by Hervé BOUTEMY
Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY:

> Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
>> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
>>> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
>>>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
>>>>> based on the feedback I got, I updated the proposal:
>>>>> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=7468231
>>>>> 8
>>>>>
>>>>> The archives entries timestamp is now configured with
>>>>> project.build.outputTimestamp property, in ISO-8601 format <project>
>>>>>
>>>>>      <properties>
>>>>>      
>>>>>        <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build
>>>>>        .ou
>>>>>        tputTimestamp>>
>>>>>      
>>>>>      </properties>
>>>>>
>>>>> </project>
>>>>>
>>>>> The shared components, plugins, parent poms and Maven core branches have
>>>>> been updated to match this new proposal
>>>>>
>>>>>
>>>>> If no one objects, next week-end, I'll start the (heavy) release train
>>>>> to
>>>>> bring (binary) Reproducible Builds plugins to general availability
>>>>
>>>> Really, really nice work..
>>>
>>> thank you, it required a lot of energy for a long period of time...
>>>
>>>> I do like <project.build.outputTimestamp>. Why did you make it a String?
>>>> Why not go directly with Instant? It gives your ISO 8601 for free
>>>
>>> I tried to explain it in https://issues.apache.org/jira/browse/MSHARED-837
>>> Plexus Date injection support is really limited: could not match the
>>> requirements
>>
>> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
>> will require to change the converter of course.
>>
>> Alternatively, you could try
>> https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html
>> /org/apache/commons/lang3/time/DateFormatUtils.html#line.72 which I have
>> added long time ago.
> the question is not about coding the date format: it's already done in maven-archiver
> https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a46fbd4c78445301

The commit contains some flaws. I will leave comments there.

> the question, as you perfectly asked initially, is: do you have an idea on how to have the direct Date injection, instead of having a String in the plugin parameter?

I assumed the DateConverter will do as soon as you say:
@Parameter
Date myDate;

Won't it?


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Tibor Digana
 ISO format was often discussed and this was found as problematic format
because you cannot always compute it to UTC due to GMT offset. The offset
is not enough. What is required for EXACT computing to UTC is Time zome
name but this format does not support it. It is exactly the same problem in
XML.

I prefer using LONG by default because it naturaly represents the time in
UTC.
Format in second property would override but I do not recommend to use it.
Then the plugins can compute LONG/UTC to whatever format they want to or
they prefer.

On Sun, Oct 6, 2019 at 12:25 PM Michael Osipov <[hidden email]> wrote:

> Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY:
> > Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
> >> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
> >>> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
> >>>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
> >>>>> based on the feedback I got, I updated the proposal:
> >>>>>
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=7468231
> >>>>> 8
> >>>>>
> >>>>> The archives entries timestamp is now configured with
> >>>>> project.build.outputTimestamp property, in ISO-8601 format <project>
> >>>>>
> >>>>>      <properties>
> >>>>>
> >>>>>
> <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build
> >>>>>        .ou
> >>>>>        tputTimestamp>>
> >>>>>
> >>>>>      </properties>
> >>>>>
> >>>>> </project>
> >>>>>
> >>>>> The shared components, plugins, parent poms and Maven core branches
> have
> >>>>> been updated to match this new proposal
> >>>>>
> >>>>>
> >>>>> If no one objects, next week-end, I'll start the (heavy) release
> train
> >>>>> to
> >>>>> bring (binary) Reproducible Builds plugins to general availability
> >>>>
> >>>> Really, really nice work..
> >>>
> >>> thank you, it required a lot of energy for a long period of time...
> >>>
> >>>> I do like <project.build.outputTimestamp>. Why did you make it a
> String?
> >>>> Why not go directly with Instant? It gives your ISO 8601 for free
> >>>
> >>> I tried to explain it in
> https://issues.apache.org/jira/browse/MSHARED-837
> >>> Plexus Date injection support is really limited: could not match the
> >>> requirements
> >>
> >> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick. It also
> >> will require to change the converter of course.
> >>
> >> Alternatively, you could try
> >>
> https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-html
> >> /org/apache/commons/lang3/time/DateFormatUtils.html#line.72 which I have
> >> added long time ago.
> > the question is not about coding the date format: it's already done in
> maven-archiver
> >
> https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a46fbd4c78445301
>
> The commit contains some flaws. I will leave comments there.
>
> > the question, as you perfectly asked initially, is: do you have an idea
> on how to have the direct Date injection, instead of having a String in the
> plugin parameter?
>
> I assumed the DateConverter will do as soon as you say:
> @Parameter
> Date myDate;
>
> Won't it?
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Tibor Digana
Michael, it is the problem with summer time. Do you know what i mean?We had
this problem in my company therefore we strictly used Z as UTC and if
somebody sent another timezone we sent back an error from REST.
You cannot say that you disagree if you do not understand. Pls have it
logically explained first!

On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <[hidden email]> wrote:

> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
> >   ISO format was often discussed and this was found as problematic format
> > because you cannot always compute it to UTC due to GMT offset. The offset
> > is not enough. What is required for EXACT computing to UTC is Time zome
> > name but this format does not support it. It is exactly the same problem
> in
> > XML.
>
> I don't understand that and do not agree. Of course, you can do
> normalization. Can you please elaborate?
>
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Michael Osipov-2
I still don't see and issue because the offset is there. If you subtract
or add the offset and you have the Zulu time.

Can you provide this concrete example? I am quite certain that there was
an error on some side.

If you case is true, the entire time logic in Java 8 woudn't be able to
perform conversions from Istants to LocalDateTime.

Am 2019-10-06 um 12:34 schrieb Tibor Digana:

> Michael, it is the problem with summer time. Do you know what i mean?We had
> this problem in my company therefore we strictly used Z as UTC and if
> somebody sent another timezone we sent back an error from REST.
> You cannot say that you disagree if you do not understand. Pls have it
> logically explained first!
>
> On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <[hidden email]> wrote:
>
>> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
>>>    ISO format was often discussed and this was found as problematic format
>>> because you cannot always compute it to UTC due to GMT offset. The offset
>>> is not enough. What is required for EXACT computing to UTC is Time zome
>>> name but this format does not support it. It is exactly the same problem
>> in
>>> XML.
>>
>> I don't understand that and do not agree. Of course, you can do
>> normalization. Can you please elaborate?
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Tibor Digana
The difference between this time and UTC is ( Zone Offset + DST offset ).
I think here this feature does not need to describe the LOCAL time nothing
but the timestamp and that is in UTC. It makes sense that the plugins use
particular FORMAT, forinstance platform format of predefined format by the
Maven. It is save to manipulate with the UTC time even if you use it a
minute before DST or after. The UTC will be always precisely understood
exact clock by Java and Maven too.

On Sun, Oct 6, 2019 at 12:37 PM Michael Osipov <[hidden email]> wrote:

> I still don't see and issue because the offset is there. If you subtract
> or add the offset and you have the Zulu time.
>
> Can you provide this concrete example? I am quite certain that there was
> an error on some side.
>
> If you case is true, the entire time logic in Java 8 woudn't be able to
> perform conversions from Istants to LocalDateTime.
>
> Am 2019-10-06 um 12:34 schrieb Tibor Digana:
> > Michael, it is the problem with summer time. Do you know what i mean?We
> had
> > this problem in my company therefore we strictly used Z as UTC and if
> > somebody sent another timezone we sent back an error from REST.
> > You cannot say that you disagree if you do not understand. Pls have it
> > logically explained first!
> >
> > On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <[hidden email]>
> wrote:
> >
> >> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
> >>>    ISO format was often discussed and this was found as problematic
> format
> >>> because you cannot always compute it to UTC due to GMT offset. The
> offset
> >>> is not enough. What is required for EXACT computing to UTC is Time zome
> >>> name but this format does not support it. It is exactly the same
> problem
> >> in
> >>> XML.
> >>
> >> I don't understand that and do not agree. Of course, you can do
> >> normalization. Can you please elaborate?
> >>
> >
>
>
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Emmanuel Bourg
In reply to this post by Hervé BOUTEMY
Le 06/10/2019 à 09:43, Hervé BOUTEMY a écrit :

> Notice that you can also express a timezone (as digits), as seen in the unit tests.

I know but that's not desirable, otherwise the formatted timestamp would
depend on the timezone of the developer and that would harm the
reproducibility of the pom (I assume the timestamp is added
automatically to the pom during the build, is that right?).


>> Would it be possible to prevent this property from being inherited?
> AFAIK no.
> And I find that default inheritance from parent to child is a nice feature.

Why do you think this is a nice feature? I can only see the downside of
child projects having a timestamp clamped in the past and leaving the
developers clueless about the reason. This would be especially bad if a
widely used parent pom like org.sonatype.oss:oss-parent or
org.apache:apache had it.

Emmanuel Bourg

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Michael Osipov-2
In reply to this post by Tibor Digana
I don't understand that either. ISO 8601 is neither aware of zones or
DSTs, just abstract offsets which is a good thing. The format Hervé has
chosen is almost correct (comments) left. The way the value has to be
provided can *always* be canonicalized to UTC. I don't see here a big
hurdle to solve a problem. When you provide 2019-10-06T14:30:00+02:00 is
clear and equals to 2019-10-06T12:30:00Z. Same applies for every other
offset.

Please keep in mind that java.util.Date is merely a wrapper around Unix
timestamp. SimpleDateFormat will work.

Am 2019-10-06 um 14:24 schrieb Tibor Digana:

> The difference between this time and UTC is ( Zone Offset + DST offset ).
> I think here this feature does not need to describe the LOCAL time nothing
> but the timestamp and that is in UTC. It makes sense that the plugins use
> particular FORMAT, forinstance platform format of predefined format by the
> Maven. It is save to manipulate with the UTC time even if you use it a
> minute before DST or after. The UTC will be always precisely understood
> exact clock by Java and Maven too.
>
> On Sun, Oct 6, 2019 at 12:37 PM Michael Osipov <[hidden email]> wrote:
>
>> I still don't see and issue because the offset is there. If you subtract
>> or add the offset and you have the Zulu time.
>>
>> Can you provide this concrete example? I am quite certain that there was
>> an error on some side.
>>
>> If you case is true, the entire time logic in Java 8 woudn't be able to
>> perform conversions from Istants to LocalDateTime.
>>
>> Am 2019-10-06 um 12:34 schrieb Tibor Digana:
>>> Michael, it is the problem with summer time. Do you know what i mean?We
>> had
>>> this problem in my company therefore we strictly used Z as UTC and if
>>> somebody sent another timezone we sent back an error from REST.
>>> You cannot say that you disagree if you do not understand. Pls have it
>>> logically explained first!
>>>
>>> On Sun, Oct 6, 2019 at 12:32 PM Michael Osipov <[hidden email]>
>> wrote:
>>>
>>>> Am 2019-10-06 um 12:25 schrieb Tibor Digana:
>>>>>     ISO format was often discussed and this was found as problematic
>> format
>>>>> because you cannot always compute it to UTC due to GMT offset. The
>> offset
>>>>> is not enough. What is required for EXACT computing to UTC is Time zome
>>>>> name but this format does not support it. It is exactly the same
>> problem
>>>> in
>>>>> XML.
>>>>
>>>> I don't understand that and do not agree. Of course, you can do
>>>> normalization. Can you please elaborate?
>>>>
>>>
>>
>>
>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Vladimir Sitnikov
>ISO 8601 is neither aware of zones or
>DSTs, just abstract offsets which is a good thing

Well. ISO 8601 allows timestamps without "UTC offset"
For instance, 2019-10-06T12:30:00 is ISO 8601-compatible timestamp, however
it is ambiguous.

So I suggest that the property must require Z or +HH:MM part.

Vladimir
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Hervé BOUTEMY
In reply to this post by Emmanuel Bourg
Le dimanche 6 octobre 2019, 12:19:35 CEST Michael Osipov a écrit :

> Am 2019-10-05 um 23:15 schrieb Emmanuel Bourg:
> > Le 05/10/2019 à 19:52, Hervé BOUTEMY a écrit :
> >> based on the feedback I got, I updated the proposal:
> >> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
> >>
> >> The archives entries timestamp is now configured with
> >> project.build.outputTimestamp property, in ISO-8601 format <project>
> >>
> >>    <properties>
> >>    
> >>      <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.o
> >>      utputTimestamp>>>    
> >>    </properties>
> >>
> >> </project>
> >
> > You may want to specify explicitly that the timestamp must be formatted
> > with the UTC timezone (as in your example).
>
> No, if you have a decent format, like ISO 8601, it is regardless of the
> timezone offset because it is properly parsed to.
wrong reasoning: ISO 8601 format supports many ways of expressing a timestamp.
If you want to express a timestamp with a timezone offset, why not, it's
perfectly legitimate to do so, even if it is not your personal taste

>
> Michael
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Michael Osipov-2
In reply to this post by Michael Osipov-2
Am 2019-10-06 um 20:21 schrieb Hervé BOUTEMY:> Le dimanche 6 octobre
2019 12:24:57 CEST, vous avez écrit :
 >> Am 2019-10-06 um 09:35 schrieb Hervé BOUTEMY:
 >>> Le samedi 5 octobre 2019, 22:46:20 CEST Michael Osipov a écrit :
 >>>> Am 2019-10-05 um 22:10 schrieb Hervé BOUTEMY:
 >>>>> Le samedi 5 octobre 2019 20:41:40 CEST, vous avez écrit :
 >>>>>> Am 2019-10-05 um 19:52 schrieb Hervé BOUTEMY:
 >>>>>>> based on the feedback I got, I updated the proposal:
 >>>>>>>
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682
 >>>>>>> 31
 >>>>>>> 8
 >>>>>>>
 >>>>>>> The archives entries timestamp is now configured with
 >>>>>>> project.build.outputTimestamp property, in ISO-8601 format
<project>
 >>>>>>>
 >>>>>>>       <properties>
 >>>>>>>
 >>>>>>>
<project.build.outputTimestamp>2019-10-02T08:04:00Z</project.bu
 >>>>>>>         ild
 >>>>>>>         .ou
 >>>>>>>         tputTimestamp>>
 >>>>>>>
 >>>>>>>       </properties>
 >>>>>>>
 >>>>>>> </project>
 >>>>>>>
 >>>>>>> The shared components, plugins, parent poms and Maven core branches
 >>>>>>> have
 >>>>>>> been updated to match this new proposal
 >>>>>>>
 >>>>>>>
 >>>>>>> If no one objects, next week-end, I'll start the (heavy)
release train
 >>>>>>> to
 >>>>>>> bring (binary) Reproducible Builds plugins to general availability
 >>>>>>
 >>>>>> Really, really nice work..
 >>>>>
 >>>>> thank you, it required a lot of energy for a long period of time...
 >>>>>
 >>>>>> I do like <project.build.outputTimestamp>. Why did you make it a
 >>>>>> String?
 >>>>>> Why not go directly with Instant? It gives your ISO 8601 for free
 >>>>>
 >>>>> I tried to explain it in
 >>>>> https://issues.apache.org/jira/browse/MSHARED-837
 >>>>> Plexus Date injection support is really limited: could not match the
 >>>>> requirements
 >>>>
 >>>> yyyy-MM-dd'T'HH:mm:ssXXX and SimpleDateFormat will do the trick.
It also
 >>>> will require to change the converter of course.
 >>>>
 >>>> Alternatively, you could try
 >>>>
https://commons.apache.org/proper/commons-lang/javadocs/api-release/src-h
 >>>> tml /org/apache/commons/lang3/time/DateFormatUtils.html#line.72
which I
 >>>> have added long time ago.
 >>>
 >>> the question is not about coding the date format: it's already done in
 >>> maven-archiver
 >>>
https://github.com/apache/maven-archiver/commit/5f07f227aa89e0bb4163c125a
 >>> 46fbd4c78445301
 >> The commit contains some flaws. I will leave comments there.
 >>
 >>> the question, as you perfectly asked initially, is: do you have an
idea on
 >>> how to have the direct Date injection, instead of having a String
in the
 >>> plugin parameter?
 >> I assumed the DateConverter will do as soon as you say:
 >> @Parameter
 >> Date myDate;
 >>
 >> Won't it?
 > yes, it will
 > and if you look at the SimpleDateFormat value, you'll see that it
does not
 > permit any timezone, then uses the local developper timezone, then is not
 > reproducible

No, it won't if you supply and enforce XXX => yyyy-MM-dd'T'HH:mm:ssXXX.
The input has to have the tz offset, thus you'll have it properly
converted. It would perform the very same conversion and your code
currently does in Maven Archiver. You should give it a try.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Tibor Digana
In reply to this post by Emmanuel Bourg
Hi Emmanuel,

>> The point is, two developers may generate a different pom if the local
timezone is used.

very well explained! It could not be better: utc time is the same however
the text is different and that's important for the stream identity.
Karl had a proposal with additional property "format" as a compromise.
Is the geography a localization important information for the behavior of
reproducible build? IMO, it is not!
So why not to have Karl's format NULL by default, means the UTC.

btw, there is whole bunch of problems with time with/out TZ in webapps and
rdbms databases and cloud system and distributed HTTP clients but that's
another story.


>> I suspect that if the jar includes the pom it'll break its
reproducibility too

Actually, there is "pom.properties" in META-INF folder of the JAR archive
artifact and this file containes the comment having the system time
generated by JDK.
There's a PR for this issue. The POM and the properties file are both
embedded in META-INF.


Cheers
Tibor17

On Sun, Oct 6, 2019 at 10:19 PM Emmanuel Bourg <[hidden email]> wrote:

> Le 06/10/2019 à 20:13, Hervé BOUTEMY a écrit :
>
> > no, it does not add any dependency on developer configuration:
> > 2019-10-05T18:37:42Z == 2019-10-05T20:37:42+02:00 ==
> 2019-10-05T16:37:42-02:00
>
> yes but:
>
>  "2019-10-05T18:37:42Z" != "2019-10-05T20:37:42+02:00" !=
> "2019-10-05T16:37:42-02:00"
>
> The point is, two developers may generate a different pom if the local
> timezone is used. A fixed timezone is necessary to ensure the pom itself
> is reproducible.
>
>
> > when will this value be written in the pom.xml is independant:
> currently, in
> > my PoC, I wrote the values by hand. In the future, it will probably be
> updated
> > by maven-release-plugin, and we'll have to choose if the timestamp is
> written
> > in Z or if it is written in local timezone with its offset: both ways of
> > expressing the timestamp are valid and will give reproducible result
>
> The jar generated is reproducible, but not the pom. I suspect that if
> the jar includes the pom it'll break its reproducibility too (this is
> the default for maven-jar-plugin, but I don't know if it embeds the
> original pom without the timestamp, or the generated pom with the
> timestamp).
>
>
> > once again, war files taken apart for web servers, who looks at
> timestamp in
> > zip files?
>
> archive timestamps are just the tip of the iceberg. There are more
> visible timestamps elsewhere, for example in the javadoc headers, in
> .properties files, in OSGi attributes, sometimes in the source files...
>
> Emmanuel Bourg
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Andreas Sewe-2
In reply to this post by Tibor Digana
Hi,

Hervé BOUTEMY wrote:
> based on the feedback I got, I updated the proposal:
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=74682318
>
> The archives entries timestamp is now configured with project.build.outputTimestamp property, in ISO-8601 format
> <project>
>   <properties>
>     <project.build.outputTimestamp>2019-10-02T08:04:00Z</project.build.outputTimestamp>
>   </properties>
> </project>

nice work, Hervé.

Two comments:

- We already have maven.build.timestamp [1], so I wonder whether it
should be maven.build.outputTimestap, too. Although one may argue that
this *output* timestamp is a property of the project being built, not
the Maven execution (in particular if the property becomes a POM element
in POM version 5).

- Can we get something analogous to maven.build.timestamp.format? One
could then write the following to be compatible with env.SOURCE_DATE_EPOCH:

<project>
  <properties>

<project.build.outputTimestamp>${env.SOURCE_DATE_EPOCH}<project.build.outputTimestamp>

<project.build.outputTimestamp.format>...</project.build.outputTimestamp.format>
  </properties>
</project>

Unfortunately, SimpleDateFormat or its java.time equivalents do not seem
to have a format string for "seconds since the epoch", so we may want to
make "seconds since the epoch" the default for
project.build.outputTimestamp's format and only parse according to
project.build.outputTimestamp.format when that property is indeed set.

WDYT?

Best wishes.

Andreas

<https://maven.apache.org/guides/introduction/introduction-to-the-pom.html#Available_Variables>


signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: last review of Reproducible Builds proposal

Hervé BOUTEMY
In reply to this post by Tibor Digana
Le jeudi 10 octobre 2019 17:17:07 CEST, vous avez écrit :

> Hi Hervé,
>
> >> - We already have maven.build.timestamp [1], so I wonder whether it
> >> should be maven.build.outputTimestap, too. Although one may argue that
> >> this *output* timestamp is a property of the project being built, not
> >> the Maven execution (in particular if the property becomes a POM element
> >> in POM version 5).
> >
> > you got it: it's project.* because it will be a POM element in POM v5
>
> Makes sense. Actually, my interpretation of the maven.* prefix (as being
> for the execution itself) seems to be wrong: A lot of core
> maven-*-plugins use this prefix for their properties, e.g.,
> maven.clean.failOnError, maven.install.skip, or maven.jar.forceCreation.
>
> Anyway, project.* makes more sense.
>
> >> Unfortunately, SimpleDateFormat or its java.time equivalents do not seem
> >> to have a format string for "seconds since the epoch", so we may want to
> >> make "seconds since the epoch" the default for
> >> project.build.outputTimestamp's format and only parse according to
> >> project.build.outputTimestamp.format when that property is indeed set.
> >>
> >> WDYT?
> >
> > the parsing from String to Date is done in maven-archiver code [1], then
> > we
> > can have the algorithm we want. We can even code that if the value is only
> > digits, it's parsed as "seconds since the epoch": no need to add the
> > format
> > property, the choice on one format over the other would be based on
> > effective value
>
> Yes, all digits -> "seconds since epoch" would be a reasonable heuristic.
>
> > What I don't get is: why do you absolutely want such a format?
> > And why do you want to use ${env.SOURCE_DATE_EPOCH}?
> > The proposal is about having "native" Reproducible Builds for Maven, then
> > not something added by an external rebuilder: it will be configured by
> > the original release.
>
> OK, I think I have misunderstood your use case:
>
> In the make-style builds I know that use SOURCE_DATE_EPOCH, some build
> step typically extracts a commit timestamp from, say, git and then sets
> SOURCE_DATE_EPOCH. This variable is then read and interpreted by the
> various built steps that follow. In other words, SOURCE_DATE_EPOCH is
> fed into the build from the outside.
>
> You seem to envision project.build.outputTimestamp to be hard-coded in
> the (consumer?) POM, i.e., not added from the outside (e.g., via
> ${env.SOURCE_DATE_EPOCH}). Is this interpretation correct?
yes
the only little mis-interpretation is that it is a pure build information,
then I don't see why this property would appear in a consumer POM

you can test it by building the fully working PoC:
https://github.com/apache/maven/tree/reproducible

>
> This means that updating the value would fall, e.g., to the
> maven-release-plugin (via [1]), so only releases would have a
> project.build.outputTimestamp (matching the release's time, not the
> timestamp of the corresponding commit, I assume).
the release plugin will update the release timestamp, yes.
But SNAPSHOTs would also have a timestamp just because they kept the timestamp
of previous release or they inherited from parent

>
> I would prefer if each commit would get an output timestamp matching the
> commit timestamp. Maybe one can use the buildnumber-maven-plugin [1] to
> achieve this, though. Setting <timestampPropertyName> to
> project.build.outputTimestamp would initialize that property during the
> "initialize" phase. Would that be early enough to be picked up by rest
> of the build.
imagine the commit is about a unit test, or documentation: changing the
timestamp would change the jar that has exactly the same .class content.
I think that updating the timestamp on every commit for SNAPSHOTs can seem
logical, but finally it would be counter-productive: that would break one
benefit of Reproducible Builds we can have for SNAPSHOTS is that we can use jar
hash to know if the code has changed or not = one use case that someone was
interested in when I had discussions with people (not on mailing list)

>
> Also, would a property set this way be part of the consumer POM?
I see this data only useful at build time: I don't know why it could be useful
at consume time

>
> I hope you can see what I am getting at, even if I might not have
> explained it very well.
I understand your solutions that look reasonably feasible, but not really
their objective.
But I appreciate the concrete discussion: maybe you have a use case that was
not defined before

Regards,

Hervé

>
> Best wishes,
>
> Andreas
>
> [1] <<https://issues.apache.org/jira/browse/MRELEASE-1029>
> [2] <https://www.mojohaus.org/buildnumber-maven-plugin/create-mojo.html>





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

12