Re: Questions regarding improving the Apache Commons release process.

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Rob Tompkins-2


> On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email]> wrote:
>
> Hi Rob,
>
> one additional point: currently, for Maven itself, instead of adding new
> Maven-specific ReleasePhase(s) to the default configuration, or configure them in
> our parent pom (I'm not sure documentation on how to do that is available: I
> could not find it), we chose first to create a separate "dist-tool" to check
> consistency of what is currently published in misc places and provide some
> commands to fix when an inconsistency is found.
>
> This happens through daily reports done by a Jenkins job [1]:
> - distribution area vs Maven Central [2]
> - version from Maven site vs Maven Central [3]
>
> We did not produce any release nor made it really configurable for conventions
> different from Maven ones (like Common's -src & -bin), but at least there is a
> configuration file to define artifacts to check [4]

Interesting. Thanks,.

-Rob

>
> HTH
>
> Hervé
>
>
> [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/
>
> [2] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool-check-source-release.html
>
> [3] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool-check-index-page.html
>
> [4] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
> dist-tool.conf.html
>
> Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
>> Stephen,
>>
>> I can’t thank you enough for your reply. I’ll take your suggestions and
>> continue to sandbox around using the maven-release-plugin as a guideline.
>>
>> All the best and happy holidays,
>> -Rob
>>
>>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
>>> <[hidden email]> wrote:>
>>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
> <mailto:[hidden email]>> wrote:
>>>> Hello all,
>>>>
>>>> Pardon, maybe this should have gone to your @user list, but why not ping
>>>> the dev crew. I’ve been playing around the ideas surrounding our fairly
>>>> manual release process for the components in Commons, and I was hoping
>>>> for
>>>> some insights.
>>>>
>>>> Scripting the version changes isn’t really that big of a deal for us, and
>>>> that I can manage. But, when it comes to publishing our artifacts to the
>>>> apache nexus repository, and then separately publishing our -src and -bin
>>>> assemblies to the dev dist subversion repository (and consequently
>>>> deleting
>>>> those artifacts from nexus as they’re “attached” for the purpose of gpg
>>>> signing), I feel it a tad cumbersome.
>>>>
>>>> I’ve fiddled around a little with the idea of detaching the -src and -bin
>>>> assemblies after gpg signing with some success, but then I have to delve
>>>> into the mechanics of publishing those up to the subversion repository,
>>>> and
>>>> clearly that problem has already been solved.
>>>
>>> Is your problem you don’t want those going to Nexus staging but only to
>>> dist? Or is it that you want them *also* going to dist.
>>>
>>> Personally... I see no reason to remove them from going to Nexus staging
>>> (in fact I have a background plan to add secondary signing support to
>>> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
>>> though. That would mean that the PMC would be able to *add* their GPG
>>> signature to the staged artifacts as part of the voting... in which case
>>> you’d want to hold off uploading to dist until *after* the vote so you get
>>> the full set of signatures)
>>>
>>> If you want to upload a subset of attached artifacts to dist as part of
>>> the
>>> release, that seems much more tenable... just a derivative of the
>>> scm-publish plugin from what I can see.
>>>
>>> So I find myself in the space of trying to shoehorn our process into its
>>>
>>>> the main maven-release-plugin, which I’ve found a tad difficult, versus
>>>> writing our own release plugin, which feels like I would be duplicating
>>>> tons of code (which I don’t want to do).
>>>
>>> So the release plugin is really two parts:
>>>
>>> 1. A toolkit for writing release plugins
>>>
>>> 2. An example that does the job for the requirements of the Maven TLP and
>>> has seemed “sufficient” for a lot of other people.
>>>
>>> As such, if you have different needs, do not feel bad about having to
>>> encode differently... hopefully the toolkit half of the codebase is
>>> sufficient for you.
>>>
>>>> I’m curious if you guys have any thoughts on the matter as I’ve been
>>>> playing around in the space for a little while now.
>>>>
>>>> Cheers and happy holidays from UTC-5,
>>>> -Rob
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> <mailto:[hidden email]> For additional commands,
>>>> e-mail: [hidden email] <mailto:[hidden email]>
>>>>
>>>> --
>>>
>>> Sent from my phone
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

Rob Tompkins-2
Hello again Maven Team,

I’ve made substantive progress towards getting the commons team a release plugin (https://github.com/chtompki/commons-release-plugin <https://github.com/chtompki/commons-release-plugin>). However, in my attempt at writing plugin unit tests for the first time, I’ve found myself running into difficulties in dealing with which dependencies need to be in the classpath to effectively run the maven-plugin-testing-harness.

I was hoping to get another set of eyes on my work, namely the pom and the unit test that is in flight (see the above repository), such that I can get around these classpath issues and start writing proper tests for the plugin. It is quite easy to see the issues that I’m having by cloning the project and running “mvn test:"

Running org.apache.commons.release.plugin.mojos.CommonsSiteCompressionMojoTest
Jan 06, 2018 2:00:13 PM org.eclipse.sisu.inject.Logs$JULSink warn
WARNING: Error injecting: org.apache.maven.artifact.resolver.DefaultArtifactResolver
java.lang.NoClassDefFoundError: Lorg/apache/maven/artifact/transform/ArtifactTransformationManager;
        at java.lang.Class.getDeclaredFields0(Native Method)
        at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
        at java.lang.Class.getDeclaredFields(Class.java:1916)

Any guidance here would be much welcomed, and moreover I can’t thank you guys enough for the previous insights because they gave me the ability to make solid progress on our release automation.

Many thanks and all the best,
-Rob


> On Dec 28, 2017, at 4:53 PM, Rob Tompkins <[hidden email]> wrote:
>
>
>
>> On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email]> wrote:
>>
>> Hi Rob,
>>
>> one additional point: currently, for Maven itself, instead of adding new
>> Maven-specific ReleasePhase(s) to the default configuration, or configure them in
>> our parent pom (I'm not sure documentation on how to do that is available: I
>> could not find it), we chose first to create a separate "dist-tool" to check
>> consistency of what is currently published in misc places and provide some
>> commands to fix when an inconsistency is found.
>>
>> This happens through daily reports done by a Jenkins job [1]:
>> - distribution area vs Maven Central [2]
>> - version from Maven site vs Maven Central [3]
>>
>> We did not produce any release nor made it really configurable for conventions
>> different from Maven ones (like Common's -src & -bin), but at least there is a
>> configuration file to define artifacts to check [4]
>
> Interesting. Thanks,.
>
> -Rob
>
>>
>> HTH
>>
>> Hervé
>>
>>
>> [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/
>>
>> [2] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>> dist-tool-check-source-release.html
>>
>> [3] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>> dist-tool-check-index-page.html
>>
>> [4] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>> dist-tool.conf.html
>>
>> Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
>>> Stephen,
>>>
>>> I can’t thank you enough for your reply. I’ll take your suggestions and
>>> continue to sandbox around using the maven-release-plugin as a guideline.
>>>
>>> All the best and happy holidays,
>>> -Rob
>>>
>>>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
>>>> <[hidden email]> wrote:>
>>>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
>> <mailto:[hidden email]>> wrote:
>>>>> Hello all,
>>>>>
>>>>> Pardon, maybe this should have gone to your @user list, but why not ping
>>>>> the dev crew. I’ve been playing around the ideas surrounding our fairly
>>>>> manual release process for the components in Commons, and I was hoping
>>>>> for
>>>>> some insights.
>>>>>
>>>>> Scripting the version changes isn’t really that big of a deal for us, and
>>>>> that I can manage. But, when it comes to publishing our artifacts to the
>>>>> apache nexus repository, and then separately publishing our -src and -bin
>>>>> assemblies to the dev dist subversion repository (and consequently
>>>>> deleting
>>>>> those artifacts from nexus as they’re “attached” for the purpose of gpg
>>>>> signing), I feel it a tad cumbersome.
>>>>>
>>>>> I’ve fiddled around a little with the idea of detaching the -src and -bin
>>>>> assemblies after gpg signing with some success, but then I have to delve
>>>>> into the mechanics of publishing those up to the subversion repository,
>>>>> and
>>>>> clearly that problem has already been solved.
>>>>
>>>> Is your problem you don’t want those going to Nexus staging but only to
>>>> dist? Or is it that you want them *also* going to dist.
>>>>
>>>> Personally... I see no reason to remove them from going to Nexus staging
>>>> (in fact I have a background plan to add secondary signing support to
>>>> staging... i’m Waiting to see the Nexus 3 staging APIs before attempting
>>>> though. That would mean that the PMC would be able to *add* their GPG
>>>> signature to the staged artifacts as part of the voting... in which case
>>>> you’d want to hold off uploading to dist until *after* the vote so you get
>>>> the full set of signatures)
>>>>
>>>> If you want to upload a subset of attached artifacts to dist as part of
>>>> the
>>>> release, that seems much more tenable... just a derivative of the
>>>> scm-publish plugin from what I can see.
>>>>
>>>> So I find myself in the space of trying to shoehorn our process into its
>>>>
>>>>> the main maven-release-plugin, which I’ve found a tad difficult, versus
>>>>> writing our own release plugin, which feels like I would be duplicating
>>>>> tons of code (which I don’t want to do).
>>>>
>>>> So the release plugin is really two parts:
>>>>
>>>> 1. A toolkit for writing release plugins
>>>>
>>>> 2. An example that does the job for the requirements of the Maven TLP and
>>>> has seemed “sufficient” for a lot of other people.
>>>>
>>>> As such, if you have different needs, do not feel bad about having to
>>>> encode differently... hopefully the toolkit half of the codebase is
>>>> sufficient for you.
>>>>
>>>>> I’m curious if you guys have any thoughts on the matter as I’ve been
>>>>> playing around in the space for a little while now.
>>>>>
>>>>> Cheers and happy holidays from UTC-5,
>>>>> -Rob
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: [hidden email]
>>>>> <mailto:[hidden email]> For additional commands,
>>>>> e-mail: [hidden email] <mailto:[hidden email]>
>>>>>
>>>>> --
>>>>
>>>> Sent from my phone

Reply | Threaded
Open this post in threaded view
|

Re: Questions regarding improving the Apache Commons release process.

rfscholte
Hi Rob,

at the moment I'm busy upgrading the maven-release-plugin to Maven3 (i.e.  
dropping Maven2 support)
That meant rewriting quite some unittests and upgrading the  
maven-plugin-testing-harness (current one is really old).

In some cases the maven-plugin-testing-harness is hard to use, so most of  
the time we're using the maven-invoker-plugin to make the integration  
tests. These tests are a little bit harder to debug from IDE when  
required, but they should fit better with the way the plugin is executed  
by Maven.

thanks,
Robert

On Sat, 06 Jan 2018 20:03:20 +0100, Rob Tompkins <[hidden email]>  
wrote:

> Hello again Maven Team,
>
> I’ve made substantive progress towards getting the commons team a  
> release plugin (https://github.com/chtompki/commons-release-plugin 
> <https://github.com/chtompki/commons-release-plugin>). However, in my  
> attempt at writing plugin unit tests for the first time, I’ve found  
> myself running into difficulties in dealing with which dependencies need  
> to be in the classpath to effectively run the  
> maven-plugin-testing-harness.
>
> I was hoping to get another set of eyes on my work, namely the pom and  
> the unit test that is in flight (see the above repository), such that I  
> can get around these classpath issues and start writing proper tests for  
> the plugin. It is quite easy to see the issues that I’m having by  
> cloning the project and running “mvn test:"
>
> Running  
> org.apache.commons.release.plugin.mojos.CommonsSiteCompressionMojoTest
> Jan 06, 2018 2:00:13 PM org.eclipse.sisu.inject.Logs$JULSink warn
> WARNING: Error injecting:  
> org.apache.maven.artifact.resolver.DefaultArtifactResolver
> java.lang.NoClassDefFoundError:  
> Lorg/apache/maven/artifact/transform/ArtifactTransformationManager;
>         at java.lang.Class.getDeclaredFields0(Native Method)
>         at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
>         at java.lang.Class.getDeclaredFields(Class.java:1916)
>
> Any guidance here would be much welcomed, and moreover I can’t thank you  
> guys enough for the previous insights because they gave me the ability  
> to make solid progress on our release automation.
>
> Many thanks and all the best,
> -Rob
>
>
>> On Dec 28, 2017, at 4:53 PM, Rob Tompkins <[hidden email]> wrote:
>>
>>
>>
>>> On Dec 28, 2017, at 4:05 AM, Hervé BOUTEMY <[hidden email]>  
>>> wrote:
>>>
>>> Hi Rob,
>>>
>>> one additional point: currently, for Maven itself, instead of adding  
>>> new
>>> Maven-specific ReleasePhase(s) to the default configuration, or  
>>> configure them in
>>> our parent pom (I'm not sure documentation on how to do that is  
>>> available: I
>>> could not find it), we chose first to create a separate "dist-tool" to  
>>> check
>>> consistency of what is currently published in misc places and provide  
>>> some
>>> commands to fix when an inconsistency is found.
>>>
>>> This happens through daily reports done by a Jenkins job [1]:
>>> - distribution area vs Maven Central [2]
>>> - version from Maven site vs Maven Central [3]
>>>
>>> We did not produce any release nor made it really configurable for  
>>> conventions
>>> different from Maven ones (like Common's -src & -bin), but at least  
>>> there is a
>>> configuration file to define artifacts to check [4]
>>
>> Interesting. Thanks,.
>>
>> -Rob
>>
>>>
>>> HTH
>>>
>>> Hervé
>>>
>>>
>>> [1] https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/
>>>
>>> [2]  
>>> https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>>> dist-tool-check-source-release.html
>>>
>>> [3]  
>>> https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>>> dist-tool-check-index-page.html
>>>
>>> [4]  
>>> https://builds.apache.org/view/M-R/view/Maven/job/dist-tool-plugin/site/
>>> dist-tool.conf.html
>>>
>>> Le mercredi 27 décembre 2017, 23:32:49 CET Rob Tompkins a écrit :
>>>> Stephen,
>>>>
>>>> I can’t thank you enough for your reply. I’ll take your suggestions  
>>>> and
>>>> continue to sandbox around using the maven-release-plugin as a  
>>>> guideline.
>>>>
>>>> All the best and happy holidays,
>>>> -Rob
>>>>
>>>>> On Dec 26, 2017, at 5:27 AM, Stephen Connolly
>>>>> <[hidden email]> wrote:>
>>>>> On Tue 26 Dec 2017 at 03:10, Rob Tompkins <[hidden email]
>>> <mailto:[hidden email]>> wrote:
>>>>>> Hello all,
>>>>>>
>>>>>> Pardon, maybe this should have gone to your @user list, but why not  
>>>>>> ping
>>>>>> the dev crew. I’ve been playing around the ideas surrounding our  
>>>>>> fairly
>>>>>> manual release process for the components in Commons, and I was  
>>>>>> hoping
>>>>>> for
>>>>>> some insights.
>>>>>>
>>>>>> Scripting the version changes isn’t really that big of a deal for  
>>>>>> us, and
>>>>>> that I can manage. But, when it comes to publishing our artifacts  
>>>>>> to the
>>>>>> apache nexus repository, and then separately publishing our -src  
>>>>>> and -bin
>>>>>> assemblies to the dev dist subversion repository (and consequently
>>>>>> deleting
>>>>>> those artifacts from nexus as they’re “attached” for the purpose of  
>>>>>> gpg
>>>>>> signing), I feel it a tad cumbersome.
>>>>>>
>>>>>> I’ve fiddled around a little with the idea of detaching the -src  
>>>>>> and -bin
>>>>>> assemblies after gpg signing with some success, but then I have to  
>>>>>> delve
>>>>>> into the mechanics of publishing those up to the subversion  
>>>>>> repository,
>>>>>> and
>>>>>> clearly that problem has already been solved.
>>>>>
>>>>> Is your problem you don’t want those going to Nexus staging but only  
>>>>> to
>>>>> dist? Or is it that you want them *also* going to dist.
>>>>>
>>>>> Personally... I see no reason to remove them from going to Nexus  
>>>>> staging
>>>>> (in fact I have a background plan to add secondary signing support to
>>>>> staging... i’m Waiting to see the Nexus 3 staging APIs before  
>>>>> attempting
>>>>> though. That would mean that the PMC would be able to *add* their GPG
>>>>> signature to the staged artifacts as part of the voting... in which  
>>>>> case
>>>>> you’d want to hold off uploading to dist until *after* the vote so  
>>>>> you get
>>>>> the full set of signatures)
>>>>>
>>>>> If you want to upload a subset of attached artifacts to dist as part  
>>>>> of
>>>>> the
>>>>> release, that seems much more tenable... just a derivative of the
>>>>> scm-publish plugin from what I can see.
>>>>>
>>>>> So I find myself in the space of trying to shoehorn our process into  
>>>>> its
>>>>>
>>>>>> the main maven-release-plugin, which I’ve found a tad difficult,  
>>>>>> versus
>>>>>> writing our own release plugin, which feels like I would be  
>>>>>> duplicating
>>>>>> tons of code (which I don’t want to do).
>>>>>
>>>>> So the release plugin is really two parts:
>>>>>
>>>>> 1. A toolkit for writing release plugins
>>>>>
>>>>> 2. An example that does the job for the requirements of the Maven  
>>>>> TLP and
>>>>> has seemed “sufficient” for a lot of other people.
>>>>>
>>>>> As such, if you have different needs, do not feel bad about having to
>>>>> encode differently... hopefully the toolkit half of the codebase is
>>>>> sufficient for you.
>>>>>
>>>>>> I’m curious if you guys have any thoughts on the matter as I’ve been
>>>>>> playing around in the space for a little while now.
>>>>>>
>>>>>> Cheers and happy holidays from UTC-5,
>>>>>> -Rob
>>>>>> ---------------------------------------------------------------------
>>>>>> To unsubscribe, e-mail: [hidden email]
>>>>>> <mailto:[hidden email]> For additional commands,
>>>>>> e-mail: [hidden email] <mailto:[hidden email]>
>>>>>>
>>>>>> --
>>>>>
>>>>> Sent from my phone

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]