Re: The maven-archetype-plugin paradox

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: The maven-archetype-plugin paradox

Amelie
Hi Anders,

Thanks for your clarification.
In understand well the idea behind the 3.x version and the concern for more security and avoid to fetch anything directly from the Internet. Every "external" access should go throud the repositories configured in the Maven settings.xml, I completely agree with this, and this was indeed a flaw in the 2.x version of the plugin.

However, with the 3.x version, there are only 3 possible sources for the archetype catalog [1]:
- "remote": this is atually Maven Central i.e. the http://repo.maven.apache.org/maven2/archetype-catalog.xml catalog
- "local": the local Maven repository on the user's machine
- "internal": the plugin's internal catalog

None of these is suitable for our case.
What we need is to use a catalog hosted on our internal repository manager i.e. http://myrepo.mycompany/archetype-catalog.xml and unless I am completely mistaken this is not possible anymore.
Although the repository manager is well configured in the Maven settings, it is used for fetching dependencies, is is used to fetch the archetype artifacts themselves, but no way to use it for the catalog.
So using our own internal catalog hosted on our own internal repository is just not possible anymore.
Several users report the same issue in the ARCHETYPE-519 issue.

To my mind, the possibility to use an "alternative" catalog is an important feature of the plugin that should not be removed in the 3.x version.
I totally agree that the alternative catalog should not be configured as a direct URL in the CI, the 2.x way to do it was not the right way.
But there should still be a way to have it configured via Maven settings.xml.

Do you think my concern is valid?

Amélie

[1] http://maven.apache.org/archetype/maven-archetype-plugin/examples/generate-alternative-catalog.html

On 05/16/2017 08:56 AM, Anders Hammar wrote:

Amélie,

Thanks for describing your use case here on the list! As I am one of the
reporters for the tickets behind the change in question I'd like to
describe my reasoning:

First off, your use case is actually (if I understand it correctly)
standard for Maven usage. For a company environment I expect that central
repo (and any other external repo) to be proxied by a repo manager and
Maven should be configured to use the internal repo manager as mirror for
e.g. central. The idea is that Maven should NEVER go out on the Internet to
fetch artifacts (and archetypes are in fact artifacts).

However, the archetype maven plugin didn't cope well with that setup.
Regardless of a correctly Maven configuration it would reach out to the
Internet in many cases. And if you specified an archetype catalog via the
CLI param it would then often go out to the Internet to download the
archetype itself. It just didn't follow the Maven idea of convention over
configuration but the users had to do workarounds to get things to work
(like specifying the archetype catalog on the CLI although it actually
existed in there internal repo manager). If there was security (auth)
involved with the repos it got even more messy.

This we wanted to fix to simplify for our users. Possibly, not everything
worked out perfectly in v3.0.0 and there might be some bugs and I know
Robert has done some changes in this area in the upcoming v3.0.1.
Unfortunately I haven't had the time to verify and test this but I invite
you to help out by testing SNAPSHOT versions while changes are applied or
participate in the actual vote. By doing so you will ensure that your
actual use case is covered.

/Anders

On Mon, May 15, 2017 at 4:42 PM, Amélie Deltour <[hidden email]><mailto:[hidden email]>
wrote:



Sorry guys, but as a maven-archetype-plugin user I don't share your views
on
this subject.

Of course, I totally agree with the aim of this new 3.x release and the
idea to
be compliant with Maven3 behaviour and in particular the security features.
However, there have been quite a lot of complaints from users about
regressions
with this new plugin version.
The ARCHETYPE-439 issue [1] you mention does not give much information
about the
reasons of these complaints.
But you can see on ARCHETYPE-519 [2] comments that users *really* have a
problem
with this release, in case you doubt it.

I will try to explain the use case.
In my company we have several Maven archetypes allowing to create quickly
new
projects, with all our standard Maven setup and code skeleton.
We use these archetypes quite a lot.
The problem with the new release is the archetype *catalog*.
Indeed the plugin needs a catalog to be able to use an archetype to create
a new
project.
In our case, we use our own private catalog stored in our internal
artifacts
manager (Artifactory).
With maven-archetype-plugin 2.4 we used to refer to this catalog with the
-DarchetypeCatalog parameter.

Now with the 3.x plugin we do not have the ability to refer to our catalog
any more.
The "remote" catalog represents the
http://repo.maven.apache.org/maven2/archetype-catalog.xml catalog file, as
explained in the doc [3].
But we don't want our archetypes to be published there...
Or am I completely mistaken and did I miss anything about the new plugin
behaviour?

It is a good thing that the plugin now uses the Maven settings.xml, and
only
these settings, to resolve the archetypes from the standard artifact
repository
and associated settings, and not a custom archetype repository defined
elsewhere.
But the plugin should have the same behaviour regarding the catalog, i.e.
be
able to search the catalogs published in the repositories defined in
settings.xml, and not restrain the catalogs being searched to only the
Maven
Central and local catalogs.

So we don't ask to keep the archetypeCatalog parameter, what we need is
just a
way to keep a feature which is mandatory for us, the ability to use
archetypes
defined in a catalog that is not published in Maven Central (and not
available
locally on the machine either).
I am convinced from ARCHETYPE-519 that many users need this feature.
To my mind, the problem with the new plugin release is not a compatibility
issue
(i.e. a different way to use or configure Maven), but really a break in
functionality i.e. a feature that is not available any more.

BTW, the documentation about the new behaviour is not as clear as you say,
the
documentation still mentions "The Archetype Plugin can use catalogs from
local
filesystem and from HTTP connections." and "The Archetype Plugin can also
read
catalogs from filesystem/HTTP by providing the path/URL of a catalog file
or of
a directory containing an archetype-catalog.xml file." [4]

I really hope you will understand the concern and do something about it,
either
revert the plugin or implement something allowing to use private remote
catalogs.
For the time being we need to stuck to the 2.4 version of the plugin but
this is
not an acceptable solution for the long-term.

Regards,

Amélie

[1] https://issues.apache.org/jira/browse/ARCHETYPE-439
[2] https://issues.apache.org/jira/browse/ARCHETYPE-519
[3]
https://maven.apache.org/archetype/archetype-models/archetyp
e-catalog/archetype-catalog.html
[4]
https://maven.apache.org/archetype/maven-archetype-plugin/
specification/archetype-catalog.html

On 05/08/2017 07:38 PM, Robert Scholte wrote:



So we have this plugin, which has been released lately as requested by the
community.
It has been released as a 3.x, so it now requires Maven3 and with this
major release[1] we used this opportunity to break compatibility in case
there are parameters we don't want to use anymore.

One of the things changed is the usage of the reference to the archetype
repository. The original implementation was based on Maven2 and wasn't
using all security features as available in Maven3. This also made it hard
to maintain.
So for example, now it is picking up the artifact repository manager by
default, it'll use its credentials when required, etc.
By removing these parameters is should also be easier to use this plugin
(less parameters =ess chance of mistakes)

So I think we made quite some people happy now that things are working
much more according to Maven default behavior. However, other have issues
to use the archetype. Sometimes it is because they are using deprecated
parameters (or use parameters which should have been removed as well),
others have a local setup which now requires to add the repository to
their settings.xml.

I still think that ARCHETYPE-439[2] is valid, so I'd prefer not to revert.
Instead I hope we can find a solution which will fit better for the most.

I can think of the following solutions:
1. Continue with taken decision and further improve usage without extra
parameters
2. Find somebody willing to maintain the plugin at ASF
3. Donate the plugin
4. Revert

#3 is a serious option, because it seems that within the team there's
nobody willing to maintain the plugin, probably due to other Maven
sub-projects which have a higher priority.

Any thoughts on this topic?

Robert

[1] http://semver.org/
[2] https://issues.apache.org/jira/browse/ARCHETYPE-439








Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à
l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
destinataire de ce message, merci de le détruire et d'en avertir
l'expéditeur.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>





--
Amélie Deltour
Software Engineer
[http://www.kk-data.com/s/content/common/UX/KLGClogos.png]<http://www.kelkoo.com/>

E [hidden email]<mailto:[hidden email]>     XMPP [hidden email]<mailto:[hidden email]>
T +33 (0)4 56 09 07 41                    M -
A Parc Sud Galaxie - Le Calypso, 6 rue des Méridiens, 38130 Echirolles, FRANCE




________________________________
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.
Reply | Threaded
Open this post in threaded view
|

Re: The maven-archetype-plugin paradox

Amelie
ARCHETYPE-520 (and ARCHETYPE-358) are indeed close to the problem I try to describe, but not exactly the same.
These issues are about using the mirror configuration to access archetype data, i.e. using http://myrepo.mycompany/maven/archetype-catalog.xml instead of direct access to http://repo1.maven.org/maven2/archetype-catalog.xml, but this is still the Maven central catalog, just accessed through a mirror instead of accessing it directly.

But my issue, and the issue behind most ARCHETYPE-519 comments, is not a mirror issue, it is not either a CLI vs settings.xml issue, it is about the ability to use an *alternative catalog* which is not published on Maven central.

In my settings.xml I have several repositories defined:

        <repository>
          <id>internal</id>
          <name>repository for internal artifacts</name>
          <url>http://myrepo.mycompany/internal</url>
          ...
        </repository>
        <repository>
          <id>central</id>
          <name>repository for external artifacts</name>
          <url>http://myrepo.mycompany/external</url>
          ...
        </repository>



With 3.x it seems that the "remote" catalog uses the catalog located at http://myrepo.mycompany/external/archetype-catalog.xml (which is a proxy for Maven central catalog) but we need the catalog published at http://myrepo.mycompany/internal/archetype-catalog.xml to be used as well and this one is ignored.
That's to say, "remote" is interpreted as the "central" repository only, and other repositories configured in settings.xml are ignored.

This is what I call an "alternative catalog" and I think it is a valid use case for the plugin.
This was possible with the archetypeCatalog CLI parameter (giving the catalog URL), and it is probably what the parameter was meant for.
But is not possible anymore with 3.x.
I agree the CLI parameter can be removed, *but* the other repositories defined in settings.xml have to be taken into account when searching for remote archetype catalogs.

Please confirm if my use case is valid - in that case I think ARCHETYPE-519 should be reopend or I can open a new issue if you think it is more appropriate.
Or I misunderstood / misconfigured something: if my Maven configuration or usage is wrong I would be glad to understand what's wrong and why.

Thanks,

Amélie


On 05/16/2017 07:06 PM, Robert Scholte wrote:

Looks like ARCHETYPE-520[1] to me. I've created an integration-test for
it, so that should work with 3.0.1

The more I think about it, the more I'm convinced we should also remove
the archetypeCatalog parameter. Right now it is only causing confusion.
Does it still make sense to have "internal", which are in fact a small set
of embedded archetypes, and they are old (and remote available as well)?
Aether/Artifact Resolver should solve the rest: use both remote and local
as everybody is used to. Want to only use local, then I guess you should
add --offline.

WDYT?

Robert

[1] https://issues.apache.org/jira/browse/ARCHETYPE-520

On Tue, 16 May 2017 13:02:11 +0200, Anders Hammar <[hidden email]><mailto:[hidden email]>
wrote:



I would expect "remote" not be the central repo but what repo(s) (or
mirrors) are configured in settings.xml.

Robert, what's your view of how this works in the plugin now?

/Anders

On Tue, May 16, 2017 at 11:06 AM, Amélie Deltour
<[hidden email]><mailto:[hidden email]>
wrote:



Hi Anders,

Thanks for your clarification.
In understand well the idea behind the 3.x version and the concern for
more security and avoid to fetch anything directly from the Internet.
Every
"external" access should go throud the repositories configured in the
Maven
settings.xml, I completely agree with this, and this was indeed a flaw
in
the 2.x version of the plugin.

However, with the 3.x version, there are only 3 possible sources for the
archetype catalog [1]:
- "remote": this is atually Maven Central i.e. the
http://repo.maven.apache.org/maven2/archetype-catalog.xml catalog
- "local": the local Maven repository on the user's machine
- "internal": the plugin's internal catalog

None of these is suitable for our case.
What we need is to use a catalog hosted on our internal repository
manager
i.e. http://myrepo.mycompany/archetype-catalog.xml and unless I am
completely mistaken this is not possible anymore.
Although the repository manager is well configured in the Maven
settings,
it is used for fetching dependencies, is is used to fetch the archetype
artifacts themselves, but no way to use it for the catalog.
So using our own internal catalog hosted on our own internal repository
is
just not possible anymore.
Several users report the same issue in the ARCHETYPE-519 issue.

To my mind, the possibility to use an "alternative" catalog is an
important feature of the plugin that should not be removed in the 3.x
version.
I totally agree that the alternative catalog should not be configured
as a
direct URL in the CI, the 2.x way to do it was not the right way.
But there should still be a way to have it configured via Maven
settings.xml.

Do you think my concern is valid?

Amélie

[1] http://maven.apache.org/archetype/maven-archetype-plugin/
examples/generate-alternative-catalog.html

On 05/16/2017 08:56 AM, Anders Hammar wrote:

Amélie,

Thanks for describing your use case here on the list! As I am one of the
reporters for the tickets behind the change in question I'd like to
describe my reasoning:

First off, your use case is actually (if I understand it correctly)
standard for Maven usage. For a company environment I expect that
central
repo (and any other external repo) to be proxied by a repo manager and
Maven should be configured to use the internal repo manager as mirror
for
e.g. central. The idea is that Maven should NEVER go out on the
Internet to
fetch artifacts (and archetypes are in fact artifacts).

However, the archetype maven plugin didn't cope well with that setup.
Regardless of a correctly Maven configuration it would reach out to the
Internet in many cases. And if you specified an archetype catalog via
the
CLI param it would then often go out to the Internet to download the
archetype itself. It just didn't follow the Maven idea of convention
over
configuration but the users had to do workarounds to get things to work
(like specifying the archetype catalog on the CLI although it actually
existed in there internal repo manager). If there was security (auth)
involved with the repos it got even more messy.

This we wanted to fix to simplify for our users. Possibly, not
everything
worked out perfectly in v3.0.0 and there might be some bugs and I know
Robert has done some changes in this area in the upcoming v3.0.1.
Unfortunately I haven't had the time to verify and test this but I
invite
you to help out by testing SNAPSHOT versions while changes are applied
or
participate in the actual vote. By doing so you will ensure that your
actual use case is covered.

/Anders

On Mon, May 15, 2017 at 4:42 PM, Amélie Deltour
<[hidden email]<mailto:[hidden email]>


<mailto:[hidden email]><mailto:[hidden email]>


wrote:



Sorry guys, but as a maven-archetype-plugin user I don't share your
views
on
this subject.

Of course, I totally agree with the aim of this new 3.x release and the
idea to
be compliant with Maven3 behaviour and in particular the security
features.
However, there have been quite a lot of complaints from users about
regressions
with this new plugin version.
The ARCHETYPE-439 issue [1] you mention does not give much information
about the
reasons of these complaints.
But you can see on ARCHETYPE-519 [2] comments that users *really* have a
problem
with this release, in case you doubt it.

I will try to explain the use case.
In my company we have several Maven archetypes allowing to create
quickly
new
projects, with all our standard Maven setup and code skeleton.
We use these archetypes quite a lot.
The problem with the new release is the archetype *catalog*.
Indeed the plugin needs a catalog to be able to use an archetype to
create
a new
project.
In our case, we use our own private catalog stored in our internal
artifacts
manager (Artifactory).
With maven-archetype-plugin 2.4 we used to refer to this catalog with
the
-DarchetypeCatalog parameter.

Now with the 3.x plugin we do not have the ability to refer to our
catalog
any more.
The "remote" catalog represents the
http://repo.maven.apache.org/maven2/archetype-catalog.xml catalog file,
as
explained in the doc [3].
But we don't want our archetypes to be published there...
Or am I completely mistaken and did I miss anything about the new plugin
behaviour?

It is a good thing that the plugin now uses the Maven settings.xml, and
only
these settings, to resolve the archetypes from the standard artifact
repository
and associated settings, and not a custom archetype repository defined
elsewhere.
But the plugin should have the same behaviour regarding the catalog,
i.e.
be
able to search the catalogs published in the repositories defined in
settings.xml, and not restrain the catalogs being searched to only the
Maven
Central and local catalogs.

So we don't ask to keep the archetypeCatalog parameter, what we need is
just a
way to keep a feature which is mandatory for us, the ability to use
archetypes
defined in a catalog that is not published in Maven Central (and not
available
locally on the machine either).
I am convinced from ARCHETYPE-519 that many users need this feature.
To my mind, the problem with the new plugin release is not a
compatibility
issue
(i.e. a different way to use or configure Maven), but really a break in
functionality i.e. a feature that is not available any more.

BTW, the documentation about the new behaviour is not as clear as you
say,
the
documentation still mentions "The Archetype Plugin can use catalogs from
local
filesystem and from HTTP connections." and "The Archetype Plugin can
also
read
catalogs from filesystem/HTTP by providing the path/URL of a catalog
file
or of
a directory containing an archetype-catalog.xml file." [4]

I really hope you will understand the concern and do something about it,
either
revert the plugin or implement something allowing to use private remote
catalogs.
For the time being we need to stuck to the 2.4 version of the plugin but
this is
not an acceptable solution for the long-term.

Regards,

Amélie

[1] https://issues.apache.org/jira/browse/ARCHETYPE-439
[2] https://issues.apache.org/jira/browse/ARCHETYPE-519
[3]
https://maven.apache.org/archetype/archetype-models/archetyp
e-catalog/archetype-catalog.html
[4]
https://maven.apache.org/archetype/maven-archetype-plugin/
specification/archetype-catalog.html

On 05/08/2017 07:38 PM, Robert Scholte wrote:



So we have this plugin, which has been released lately as requested by
the
community.
It has been released as a 3.x, so it now requires Maven3 and with this
major release[1] we used this opportunity to break compatibility in case
there are parameters we don't want to use anymore.

One of the things changed is the usage of the reference to the archetype
repository. The original implementation was based on Maven2 and wasn't
using all security features as available in Maven3. This also made it
hard
to maintain.
So for example, now it is picking up the artifact repository manager by
default, it'll use its credentials when required, etc.
By removing these parameters is should also be easier to use this plugin
(less parameters =ess chance of mistakes)

So I think we made quite some people happy now that things are working
much more according to Maven default behavior. However, other have
issues
to use the archetype. Sometimes it is because they are using deprecated
parameters (or use parameters which should have been removed as well),
others have a local setup which now requires to add the repository to
their settings.xml.

I still think that ARCHETYPE-439[2] is valid, so I'd prefer not to
revert.
Instead I hope we can find a solution which will fit better for the
most.

I can think of the following solutions:
1. Continue with taken decision and further improve usage without extra
parameters
2. Find somebody willing to maintain the plugin at ASF
3. Donate the plugin
4. Revert

#3 is a serious option, because it seems that within the team there's
nobody willing to maintain the plugin, probably due to other Maven
sub-projects which have a higher priority.

Any thoughts on this topic?

Robert

[1] http://semver.org/
[2] https://issues.apache.org/jira/browse/ARCHETYPE-439








Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à
l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
destinataire de ce message, merci de le détruire et d'en avertir
l'expéditeur.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]><mailto:
[hidden email]><mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]><mailto:
[hidden email]><mailto:[hidden email]>





--
Amélie Deltour
Software Engineer
[http://www.kk-data.com/s/content/common/UX/KLGClogos.png]<h
ttp://www.kelkoo.com/><http://www.kelkoo.com/>

E [hidden email]<mailto:[hidden email]><mailto:[hidden email]><mailto:[hidden email]>     XMPP
[hidden email]<mailto:[hidden email]><mailto:[hidden email]><mailto:[hidden email]>
T +33 (0)4 56 09 07 41                    M -
A Parc Sud Galaxie - Le Calypso, 6 rue des Méridiens, 38130 Echirolles,
FRANCE




________________________________
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à
l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
destinataire de ce message, merci de le détruire et d'en avertir
l'expéditeur.



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>




--
Amélie Deltour
Software Engineer
[http://www.kk-data.com/s/content/common/UX/KLGClogos.png]<http://www.kelkoo.com/>

E [hidden email]<mailto:[hidden email]>     XMPP [hidden email]<mailto:[hidden email]>
T +33 (0)4 56 09 07 41                    M -
A Parc Sud Galaxie - Le Calypso, 6 rue des Méridiens, 38130 Echirolles, FRANCE




________________________________
Kelkoo SAS
Société par Actions Simplifiée
Au capital de € 4.168.964,30
Siège social : 158 Ter Rue du Temple 75003 Paris
425 093 069 RCS Paris

Ce message et les pièces jointes sont confidentiels et établis à l'attention exclusive de leurs destinataires. Si vous n'êtes pas le destinataire de ce message, merci de le détruire et d'en avertir l'expéditeur.