Re: Updating the apache parent to automatically clean before performing a release?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Updating the apache parent to automatically clean before performing a release?

Gary Gregory-2
You might want to take a peek at what we do over at Apache Commons. We have
attempted to make things easier for ourselves with two custom Maven plugins
but IMO it is still error prone and a pain. Some of the pain is due to
having to perform a "double release" by pushing various files to both Nexus
and the dev/release svn repo.

Gary

On Sun, Jan 5, 2020, 08:41 Christofer Dutz <[hidden email]>
wrote:

> Hi Robert,
>
> I would be more than happy to assist with making releases with maven
> easier for Apache projects ...
> So if you give me a little update on what needs to be done, I might be
> able to spare some time and work on this.
>
> But using the files from the prepare is the issue I'm trying to address
> ... the perform build is a clean checkout of the tagged version.
> The clean checkout ensures the build is 100% clean ... things like packing
> in test-results, packing in maven-wrapper jars
> etc. only happens because the source distribution is not built in a clean
> environment.
>
> I would love if it was possible to extend questions the release-prepare
> plugin asks for ... so if it would ask for a RC number,
> we could add a call to the scm:checkin goal in the deploy phase of a
> release-perform build as part of the "apache-release" profile.
> The only piece of information that’s missing, would be the
> release-candidate number.
>
>
> Chris
>
>
>
> Am 05.01.20, 14:07 schrieb "Robert Scholte" <[hidden email]>:
>
>     ASF deserves a customized release strategy, which is now possible with
> MRELEASE-956
>     My idea is that during "prepare" the plugin should upload several
> files to the ASF dist folder besides the tagging.
>     During "perform" it should use these files instead of the tag in SCM
> (because these files are the official releases, not the tag).
>
>     Just waiting for someone to pick it up.
>
>     thanks,
>     Robert
>
>
>     [1] https://jira.apache.org/jira/browse/MRELEASE-956
>     On 5-1-2020 11:28:22, Christofer Dutz <[hidden email]>
> wrote:
>     Hi all,
>
>     I just wanted to suggest something I have noticed a lot of Apache
> projects were doing wrong. Especially when unexperienced RMs are doing the
> releases.
>
>     Several times now after doing a release, the RMs have uploaded the
> source bundles from “target” to the SVN. However they should have uploaded
> the “target/checkout/target”.
>     It is just too tempting to upload the versions left over from the
> release:prepare step as they too have the release version.
>
>     Usually this wouldn’t be a problem and I guess this has happened quite
> often in the past. The thing is with the adoption of the maven-wrapper we
> can unfortunately see when something’s going wrong.
>     In this case there is a difference. The source bundles from the
> prepare step then usually contain the ./mvn/maven-wrapper.jar … which is
> usually my indicator for instantly knowing what went wrong.
>     The version in the target/checkout simply couldn’t contain this file.
>
>     From the discussions with the reproducible builds I learned that you
> can define pre and post actions to the prepare and perform steps. So how
> about adding a pre-perform action that simply cleans the target directory?
>
>     I guess updating the default assembly to exclude the jar and class
> files in the “./mvn” directory could cure some symptoms, but people would
> still upload the wrong file.
>
>     I think this could prevent a lot of RCs being -1ed by Justin ;-)
>
>
>     Chris
>
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Updating the apache parent to automatically clean before performing a release?

Hervé BOUTEMY
notice that with Reproducible Builds being active by default, the source
archive is reproducible and RM will get the same content in both locations

There is only the .asc signature file that contains a different timestamp

I'm just launching the long overdue Apache Parent 22 vote that will enable
Reproducible Builds by default, we'll see if a new release will be ready soon

Regards,

Hervé

Le dimanche 5 janvier 2020, 14:41:37 CET Christofer Dutz a écrit :
> Hi Robert,
>
> I would be more than happy to assist with making releases with maven easier
> for Apache projects ...
 So if you give me a little update on what needs to
> be done, I might be able to spare some time and work on this.
> But using the files from the prepare is the issue I'm trying to address ...
> the perform build is a clean checkout of the tagged version.
 The clean
> checkout ensures the build is 100% clean ... things like packing in
> test-results, packing in maven-wrapper jars etc. only happens because the
> source distribution is not built in a clean environment.
> I would love if it was possible to extend questions the release-prepare
> plugin asks for ... so if it would ask for a RC number,
 we could add a

> call to the scm:checkin goal in the deploy phase of a release-perform build
> as part of the "apache-release" profile. The only piece of information
> that’s missing, would be the release-candidate number.
>
> Chris
>
>
>
> Am 05.01.20, 14:07 schrieb "Robert Scholte" <[hidden email]>:
>
>     ASF deserves a customized release strategy, which is now possible with
> MRELEASE-956
 My idea is that during "prepare" the plugin should upload

> several files to the ASF dist folder besides the tagging. During "perform"
> it should use these files instead of the tag in SCM (because these files
> are the official releases, not the tag).
>     Just waiting for someone to pick it up.
>    
>     thanks,
>     Robert
>    
>    
>     [1] https://jira.apache.org/jira/browse/MRELEASE-956
>     On 5-1-2020 11:28:22, Christofer Dutz <[hidden email]>
> wrote:
 Hi all,
>    
>     I just wanted to suggest something I have noticed a lot of Apache
> projects were doing wrong. Especially when unexperienced RMs are doing the
> releases.
 
>     Several times now after doing a release, the RMs have uploaded the
> source bundles from “target” to the SVN. However they should have uploaded
> the “target/checkout/target”.
 It is just too tempting to upload the
> versions left over from the release:prepare step as they too have the
> release version.
>     Usually this wouldn’t be a problem and I guess this has happened quite
> often in the past. The thing is with the adoption of the maven-wrapper we
> can unfortunately see when something’s going wrong.
 In this case there is
> a difference. The source bundles from the prepare step then usually contain
> the ./mvn/maven-wrapper.jar … which is usually my indicator for instantly
> knowing what went wrong. The version in the target/checkout simply couldn’t
> contain this file.
>     From the discussions with the reproducible builds I learned that you can
> define pre and post actions to the prepare and perform steps. So how about
> adding a pre-perform action that simply cleans the target directory?
 
>     I guess updating the default assembly to exclude the jar and class files
> in the “./mvn” directory could cure some symptoms, but people would still
> upload the wrong file.
 
>     I think this could prevent a lot of RCs being -1ed by Justin ;-)
>    
>    
>     Chris
>    
>





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Updating the apache parent to automatically clean before performing a release?

Christofer Dutz
Hi Herve,

he'll only get the same content if everything is done right ... unfortunately for example the IoTDB project had to re-do quite a number of RCs as they were still not 100% doing things the maven way.
In PLC4X we just had to cancel a release of a sub-project due to the maven-wrapper.jar being packaged in.

There were tests writing things outside the target directory, which they were manually excluding. The maven-wrapper.jar keeps on being packaged by the default apache source assembly.
Sometimes GIT excluded temp files are packaged. Only by running the release in the target/checkout directory you can be 100% sure it's a clean structure you're packaging.

I know that most problems are due to people not doing things the official "maven way" (TM) ... but I have to admit being involved in a lot of projects,
if we could ensure some of the common pitfalls don't have such a severe impact, we could help a lot of projects.


Chris



Am 05.01.20, 18:33 schrieb "Hervé BOUTEMY" <[hidden email]>:

    notice that with Reproducible Builds being active by default, the source
    archive is reproducible and RM will get the same content in both locations
   
    There is only the .asc signature file that contains a different timestamp
   
    I'm just launching the long overdue Apache Parent 22 vote that will enable
    Reproducible Builds by default, we'll see if a new release will be ready soon
   
    Regards,
   
    Hervé
   
    Le dimanche 5 janvier 2020, 14:41:37 CET Christofer Dutz a écrit :
    > Hi Robert,
    >
    > I would be more than happy to assist with making releases with maven easier
    > for Apache projects ...
     So if you give me a little update on what needs to
    > be done, I might be able to spare some time and work on this.
    > But using the files from the prepare is the issue I'm trying to address ...
    > the perform build is a clean checkout of the tagged version.
     The clean
    > checkout ensures the build is 100% clean ... things like packing in
    > test-results, packing in maven-wrapper jars etc. only happens because the
    > source distribution is not built in a clean environment.
    > I would love if it was possible to extend questions the release-prepare
    > plugin asks for ... so if it would ask for a RC number,
     we could add a
    > call to the scm:checkin goal in the deploy phase of a release-perform build
    > as part of the "apache-release" profile. The only piece of information
    > that’s missing, would be the release-candidate number.
    >
    > Chris
    >
    >
    >
    > Am 05.01.20, 14:07 schrieb "Robert Scholte" <[hidden email]>:
    >
    >     ASF deserves a customized release strategy, which is now possible with
    > MRELEASE-956
     My idea is that during "prepare" the plugin should upload
    > several files to the ASF dist folder besides the tagging. During "perform"
    > it should use these files instead of the tag in SCM (because these files
    > are the official releases, not the tag).
    >     Just waiting for someone to pick it up.
    >    
    >     thanks,
    >     Robert
    >    
    >    
    >     [1] https://jira.apache.org/jira/browse/MRELEASE-956
    >     On 5-1-2020 11:28:22, Christofer Dutz <[hidden email]>
    > wrote:
     Hi all,
    >    
    >     I just wanted to suggest something I have noticed a lot of Apache
    > projects were doing wrong. Especially when unexperienced RMs are doing the
    > releases.
     
    >     Several times now after doing a release, the RMs have uploaded the
    > source bundles from “target” to the SVN. However they should have uploaded
    > the “target/checkout/target”.
     It is just too tempting to upload the
    > versions left over from the release:prepare step as they too have the
    > release version.
    >     Usually this wouldn’t be a problem and I guess this has happened quite
    > often in the past. The thing is with the adoption of the maven-wrapper we
    > can unfortunately see when something’s going wrong.
     In this case there is
    > a difference. The source bundles from the prepare step then usually contain
    > the ./mvn/maven-wrapper.jar … which is usually my indicator for instantly
    > knowing what went wrong. The version in the target/checkout simply couldn’t
    > contain this file.
    >     From the discussions with the reproducible builds I learned that you can
    > define pre and post actions to the prepare and perform steps. So how about
    > adding a pre-perform action that simply cleans the target directory?
     
    >     I guess updating the default assembly to exclude the jar and class files
    > in the “./mvn” directory could cure some symptoms, but people would still
    > upload the wrong file.
     
    >     I think this could prevent a lot of RCs being -1ed by Justin ;-)
    >    
    >    
    >     Chris
    >    
    >
   
   
   
   
   
    ---------------------------------------------------------------------
    To unsubscribe, e-mail: [hidden email]
    For additional commands, e-mail: [hidden email]
   
   


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Updating the apache parent to automatically clean before performing a release?

Hervé BOUTEMY
an issue has been found in Apache Parent POM 22, regarding scm-publish to Git:
see MPOM-236 [1]
it does not affect Maven project itself, since we're publishing our site html
to svn instead of Git, but many Apache projects with small sites publish to
Git
Then we'll need to release version 23 soon

Is there any chance that a solution for your clean idea is added in this
release?

Regards,

Hervé

[1] https://issues.apache.org/jira/browse/MPOM-236

Le dimanche 5 janvier 2020, 18:39:50 CET Christofer Dutz a écrit :
> Hi Herve,
>
> he'll only get the same content if everything is done right ...
> unfortunately for example the IoTDB project had to re-do quite a number of
> RCs as they were still not 100% doing things the maven way.
 In PLC4X we
> just had to cancel a release of a sub-project due to the maven-wrapper.jar
> being packaged in.
> There were tests writing things outside the target directory, which they
> were manually excluding. The maven-wrapper.jar keeps on being packaged by
> the default apache source assembly.
 Sometimes GIT excluded temp files are
> packaged. Only by running the release in the target/checkout directory you
> can be 100% sure it's a clean structure you're packaging.
> I know that most problems are due to people not doing things the official
> "maven way" (TM) ... but I have to admit being involved in a lot of
> projects,
 if we could ensure some of the common pitfalls don't have such a

> severe impact, we could help a lot of projects.
>
> Chris
>
>
>
> Am 05.01.20, 18:33 schrieb "Hervé BOUTEMY" <[hidden email]>:
>
>     notice that with Reproducible Builds being active by default, the source
>
 archive is reproducible and RM will get the same content in both
> locations
>     There is only the .asc signature file that contains a different
> timestamp
 
>     I'm just launching the long overdue Apache Parent 22 vote that will
> enable
 Reproducible Builds by default, we'll see if a new release will be

> ready soon
>     Regards,
>    
>     Hervé
>    
>     Le dimanche 5 janvier 2020, 14:41:37 CET Christofer Dutz a écrit :
>
>     > Hi Robert,
>     >
>     > I would be more than happy to assist with making releases with maven
>     > easier
 for Apache projects ...
>
>      So if you give me a little update on what needs to
>
>     > be done, I might be able to spare some time and work on this.
>     > But using the files from the prepare is the issue I'm trying to
>     > address ...
 the perform build is a clean checkout of the tagged

>     > version.
>
>      The clean
>
>     > checkout ensures the build is 100% clean ... things like packing in
>     > test-results, packing in maven-wrapper jars etc. only happens because
>     > the
>     > source distribution is not built in a clean environment.
>     > I would love if it was possible to extend questions the
>     > release-prepare
>     > plugin asks for ... so if it would ask for a RC number,
>
>      we could add a
>
>     > call to the scm:checkin goal in the deploy phase of a release-perform
>     > build
 as part of the "apache-release" profile. The only piece of

>     > information that’s missing, would be the release-candidate number.
>     >
>     > Chris
>     >
>     >
>     >
>     > Am 05.01.20, 14:07 schrieb "Robert Scholte" <[hidden email]>:
>     >
>     >
>     >     ASF deserves a customized release strategy, which is now possible
>     >     with
>     >
>     > MRELEASE-956
>
>      My idea is that during "prepare" the plugin should upload
>
>     > several files to the ASF dist folder besides the tagging. During
>     > "perform"
>     > it should use these files instead of the tag in SCM (because these
>     > files
>     > are the official releases, not the tag).
>     >
>     >     Just waiting for someone to pick it up.
>     >    
>     >     thanks,
>     >     Robert
>     >    
>     >    
>     >     [1] https://jira.apache.org/jira/browse/MRELEASE-956
>     >     On 5-1-2020 11:28:22, Christofer Dutz <[hidden email]>
>     >
>     > wrote:
>
>      Hi all,
>
>     >    
>     >     I just wanted to suggest something I have noticed a lot of Apache
>     >
>     > projects were doing wrong. Especially when unexperienced RMs are doing
>     > the
>     > releases.
>
>      
>
>     >     Several times now after doing a release, the RMs have uploaded
>     >     the
>     >
>     > source bundles from “target” to the SVN. However they should have
>     > uploaded
>     > the “target/checkout/target”.
>
>      It is just too tempting to upload the
>
>     > versions left over from the release:prepare step as they too have the
>     > release version.
>     >
>     >     Usually this wouldn’t be a problem and I guess this has happened
>     >     quite
>     >
>     > often in the past. The thing is with the adoption of the maven-wrapper
>     > we
>     > can unfortunately see when something’s going wrong.
>
>      In this case there is
>
>     > a difference. The source bundles from the prepare step then usually
>     > contain
 the ./mvn/maven-wrapper.jar … which is usually my indicator

>     > for instantly knowing what went wrong. The version in the
>     > target/checkout simply couldn’t contain this file.
>     >
>     >     From the discussions with the reproducible builds I learned that
>     >     you can
>     >
>     > define pre and post actions to the prepare and perform steps. So how
>     > about
>     > adding a pre-perform action that simply cleans the target directory?
>
>      
>
>     >     I guess updating the default assembly to exclude the jar and class
>     >     files
>     >
>     > in the “./mvn” directory could cure some symptoms, but people would
>     > still
>     > upload the wrong file.
>
>      
>
>     >     I think this could prevent a lot of RCs being -1ed by Justin ;-)
>     >    
>     >    
>     >     Chris
>     >    
>     >
>     >
>
>    
>    
>    
>    
>    
>     ---------------------------------------------------------------------
>     To unsubscribe, e-mail: [hidden email]
>     For additional commands, e-mail: [hidden email]
>    
>    
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]