Quantcast

Avoiding duplicate POM code: profiles, inheritance, properties

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Avoiding duplicate POM code: profiles, inheritance, properties

Gillet Thomas (2)
Hello all,

I'm currently working in several projects using maven 3, and I need some help defining a new POM hierarchy to avoid POM code duplication.

My projects are composed of multiple bundles (it's OSGi based).
Each project has its own deployment and source repositories and defines a different set of bundles, but all bundles in all projects are sharing common basic configurations.
Note: I don't use the word "project" in the meaning of "maven project" (that is, a project is a bunch of bundles, each bundle being a single maven project).

For now, each project is totally independent and has its own POM hierarchy:

The "deployment" POM : top most POM, contains project specific properties (deployment repository, scm, developers...).
The "common" POM : next one in the hierarchy tree, contains common features (dependencies, plugin configurations...).
The "type" POMs : the third hierarchy level has several POMs, each one with features for a specific bundle type (web bundle, persistence bundle...).
The common POM and some of the type POMs have different profiles corresponding to different build targets (onboard, offboard...).

This way, each bundle defines its type by inheriting from the right parent, and building for a given target is done through profile activation on the command-line (ex. mvn install -P onboard).

It is working, but, bundle types being the same for all projects, parent POMs are almost identical.
I just copy the POM hierarchy into each new project, changing only the properties of the deployment POM.

The build infrastructure and deployment process being still a work in progress, the POMs are changing quite often and the maintenance of all those duplicated configs is becoming really annoying. So here am I, looking for answers.
What I would like to do is package all "technical" stuff (dependencies, plugin configurations) in global POM(s) being shared by all projects,
and leave management properties (the deployment POM) inside each project.

What I tried:

The current approach forbid this because the deployment POM is at the top of the hierarchy.
Because there is multiple leaf POMs (the type POMs), putting deployment at the bottom would need multiple project specific POMs, each one inheriting from a  type POM. Some kind of "proxy" POMs adding properties along the way.
But then deployment information would be duplicated. Not good enough.
Unless there is a way to put those information in a property file, and use it in all those proxys. But I don't know how to do that.

I also tried to group all type POMs into one single POM with profiles.
One profile by type, activated by a marker file in the inheriting bundle (didn't find any other way to activate a parent profile from a child artifact).
But then the target specific behavior is not working anymore, because profile activations are ORed together, so I cannot create a profile running only for the "persistence" type AND the "onboard" target.
So this is not working either.

Now, I start wondering if I'm not asking maven more than it can manage.
Not the first time I struggle with its poor inheritance capabilities and the lack of an "include" feature (heard about mixins, but seems it won't be there before a long time).

Thanks in advance to all people who will reach the end of this really long post (if any).

Regards,
Thomas GILLET

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avoiding duplicate POM code: profiles, inheritance, properties

Ron Wheeler
I read it to the end but I am not sure that I got it all.

One suggestion that we found helpful is to start to look at some of the
maven projects as utilities that are really no different from a
developers point of view than utilities that we use from Apache or
Springframework or others.
They get their own lifecycle and live in our Nexus where all projects
can use them as dependencies.

I am not sure why you have more than one deployment repo since you
should be able to separate things out by their GAV and possibly classifiers.

Like you, each of our applications has a parent pom maven project and a
bunch of maven projects that create
- jars that are dependencies for other projects
- jars that are executables
- wars that are webapps.

These are all held in the single repo since they all have unique GAVs.
There is no reason why each project could not have its own repo. I can
only think of internal security or secrecy as generating this requirement.

I am not sure that I have advanced your cause in any way except perhaps
in the area of separating applications from utilities.
Perhaps others who use OSGI might be able to advance the conversation a
bit more.

Ron

On 30/03/2012 9:36 AM, Gillet Thomas (2) wrote:

> Hello all,
>
> I'm currently working in several projects using maven 3, and I need some help defining a new POM hierarchy to avoid POM code duplication.
>
> My projects are composed of multiple bundles (it's OSGi based).
> Each project has its own deployment and source repositories and defines a different set of bundles, but all bundles in all projects are sharing common basic configurations.
> Note: I don't use the word "project" in the meaning of "maven project" (that is, a project is a bunch of bundles, each bundle being a single maven project).
>
> For now, each project is totally independent and has its own POM hierarchy:
>
> The "deployment" POM : top most POM, contains project specific properties (deployment repository, scm, developers...).
> The "common" POM : next one in the hierarchy tree, contains common features (dependencies, plugin configurations...).
> The "type" POMs : the third hierarchy level has several POMs, each one with features for a specific bundle type (web bundle, persistence bundle...).
> The common POM and some of the type POMs have different profiles corresponding to different build targets (onboard, offboard...).
>
> This way, each bundle defines its type by inheriting from the right parent, and building for a given target is done through profile activation on the command-line (ex. mvn install -P onboard).
>
> It is working, but, bundle types being the same for all projects, parent POMs are almost identical.
> I just copy the POM hierarchy into each new project, changing only the properties of the deployment POM.
>
> The build infrastructure and deployment process being still a work in progress, the POMs are changing quite often and the maintenance of all those duplicated configs is becoming really annoying. So here am I, looking for answers.
> What I would like to do is package all "technical" stuff (dependencies, plugin configurations) in global POM(s) being shared by all projects,
> and leave management properties (the deployment POM) inside each project.
>
> What I tried:
>
> The current approach forbid this because the deployment POM is at the top of the hierarchy.
> Because there is multiple leaf POMs (the type POMs), putting deployment at the bottom would need multiple project specific POMs, each one inheriting from a  type POM. Some kind of "proxy" POMs adding properties along the way.
> But then deployment information would be duplicated. Not good enough.
> Unless there is a way to put those information in a property file, and use it in all those proxys. But I don't know how to do that.
>
> I also tried to group all type POMs into one single POM with profiles.
> One profile by type, activated by a marker file in the inheriting bundle (didn't find any other way to activate a parent profile from a child artifact).
> But then the target specific behavior is not working anymore, because profile activations are ORed together, so I cannot create a profile running only for the "persistence" type AND the "onboard" target.
> So this is not working either.
>
> Now, I start wondering if I'm not asking maven more than it can manage.
> Not the first time I struggle with its poor inheritance capabilities and the lack of an "include" feature (heard about mixins, but seems it won't be there before a long time).
>
> Thanks in advance to all people who will reach the end of this really long post (if any).
>
> Regards,
> Thomas GILLET
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

--
Ron Wheeler
President
Artifact Software Inc
email: [hidden email]
skype: ronaldmwheeler
phone: 866-970-2435, ext 102




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Avoiding duplicate POM code: profiles, inheritance, properties

Gillet Thomas (2)
Hello Ron,

I'm not sure I understand what you meant...

My problem is all about the POMs, that is, how can I share pieces of configuration (i.e pieces of POM file) between bundles, the main problems being that I have not a single inheritance chain (because of project specific deployment properties and target specific configuration).

The "real" artifacts (the ones with actual code in it, that is, the bundles)  are not really part of the problem. So I fail to see the link with the utilities...
Unless maybe you were implying that a POM can use another POM the same way a bundle can use a library (which would indeed be the solution to all my problems), but I don't know how to do that.

The fact that we have multiple repositories is just a convenience. We split up our Artifactory into several sub-repositories so we can manage permissions of each developer differently for each repo (that is, for each project).
We would have been using one big repo if it had helped, but still remains the per-project SVN repos which we want to keep like that.
So using the release plugin for example would still require at least this property to be specific to each project.

That is maybe a lot of wondering just for one property (well, one for now, but who knows). Still, I really hate duplicate stuff.

Read my post again, and that's indeed not so clear what I am speaking about. I should rewrite the question.
Anyway, thanks for your answer.

Thomas GILLET


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avoiding duplicate POM code: profiles, inheritance, properties

Wayne Fay
> My problem is all about the POMs, that is, how can I share pieces of
> configuration (i.e pieces of POM file) between bundles, the main problems
> being that I have not a single inheritance chain (because of project specific
> deployment properties and target specific configuration).

I'm not sure that I understand your problem very well either. I can
offer up a few bits that may be helpful.

1. Inheritance does not necessarily follow aggregation. As an example:
Project A with modules SubAA, SubAB, SubAC
+ Project SubAA [parent is A]
+ Project SubAB [parent is B]
+ Project SubAC [parent is C]
Project B (no modules)
Project C (no modules)

2. The maven-remote-resources-plugin can help share resources
(configuration files etc) between modules:
http://maven.apache.org/plugins/maven-remote-resources-plugin/examples/sharing-resources.html

3. Look into the "import" scope (but this really only helps managing
dependencies, not necessarily useful to you):
http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Importing_Dependencies

I don't know if any of these will help you but they are good things to know.

Wayne

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Avoiding duplicate POM code: profiles, inheritance, properties

Gillet Thomas (2)
Hello Wayne,

Seems my first post was really not clear. Trying to make it simpler:

Basically, my main concern is to be able to package separately parts of the POM:
- plugin configuration and/or dependencies (previously the "type POM")
- distribution management information (previously the "deployment POM")

So I can later define one set of distribution management information per project, and one set of configuration per bundle type.
Then each bundle will, depending on the project it is member of and depending on its type, inherit from one set of distribution management information, and from one set of configuration.

Hope this is a better explanation.

Then, about your proposals:

> 1. Inheritance does not necessarily follow aggregation.

That's right, I didn't think about aggregator POMs. Actually if properties defined in an aggregator POM are accessible in the sub-modules, then it could be a place to put shared properties for distribution management.
But I'm not sure it is the case, and anyway it would make those properties available only when building the aggregator. I need to be able to build each module alone.

> 2. The maven-remote-resources-plugin can help share resources (configuration files etc) between modules

Didn't know that. But I need to share properties, dependencies and plugin configurations, not only resources.

> 3. Look into the "import" scope (but this really only helps managing dependencies, not necessarily useful to you)

Didn't know that either, thanks to point it out.
It actually could be used for the sets of configuration which only contain dependencies, but when it comes to plugin configuration, again, not working.

Not helping this particular problem I think, but that's indeed good thinks to know.
Thanks for the pointers.

Thomas GILLET

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avoiding duplicate POM code: profiles, inheritance, properties

Ron Wheeler
It sounds like you are describing the usage of a parent POM.
A parent is a project specific POM that ideally carries your plug-in
versions, deployment information and code versions for all of the POMs
in the maven projects that makes up your application. Some of this is
properties and some is full specifications.

This puts these decisions under the control of the project team and
reduces the concerns of the individual developers and shrinks the maven
project POMs by a lot.

You should read the docs on the usage of parent POMs.

Remember
1) A lot of people have built whatever it is that you are doing and
there are Best Practices for doing it.
2) Maven has a very particular way of doing things and until you comply,
you will be in a constant losing battle. "Resistance is futile".

Have a look at the list of Maven PMC members and the committers to see
the quality of advice that you are getting.

Ron

On 02/04/2012 7:55 AM, Gillet Thomas (2) wrote:

> Hello Wayne,
>
> Seems my first post was really not clear. Trying to make it simpler:
>
> Basically, my main concern is to be able to package separately parts of the POM:
> - plugin configuration and/or dependencies (previously the "type POM")
> - distribution management information (previously the "deployment POM")
>
> So I can later define one set of distribution management information per project, and one set of configuration per bundle type.
> Then each bundle will, depending on the project it is member of and depending on its type, inherit from one set of distribution management information, and from one set of configuration.
>
> Hope this is a better explanation.
>
> Then, about your proposals:
>
>> 1. Inheritance does not necessarily follow aggregation.
> That's right, I didn't think about aggregator POMs. Actually if properties defined in an aggregator POM are accessible in the sub-modules, then it could be a place to put shared properties for distribution management.
> But I'm not sure it is the case, and anyway it would make those properties available only when building the aggregator. I need to be able to build each module alone.
>
>> 2. The maven-remote-resources-plugin can help share resources (configuration files etc) between modules
> Didn't know that. But I need to share properties, dependencies and plugin configurations, not only resources.
>
>> 3. Look into the "import" scope (but this really only helps managing dependencies, not necessarily useful to you)
> Didn't know that either, thanks to point it out.
> It actually could be used for the sets of configuration which only contain dependencies, but when it comes to plugin configuration, again, not working.
>
> Not helping this particular problem I think, but that's indeed good thinks to know.
> Thanks for the pointers.
>
> Thomas GILLET
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

--
Ron Wheeler
President
Artifact Software Inc
email: [hidden email]
skype: ronaldmwheeler
phone: 866-970-2435, ext 102




---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avoiding duplicate POM code: profiles, inheritance, properties

Curtis Rueden
In reply to this post by Gillet Thomas (2)
Hi Gillet,


Basically, my main concern is to be able to package separately parts of the
> POM:
> - plugin configuration and/or dependencies (previously the "type POM")
> - distribution management information (previously the "deployment POM")
>

To me, it sounds like you are asking for multiple inheritance in POMs—or
else a non-inheritance-based include mechanism. AFAIK, Maven does not have
either of those capabilities.

Just like in Java, which is also single-inheritance only, it can be
difficult or impossible to avoid code duplication in certain complex cases.
But in my experience it is fairly rare, and often you can find an
alternative design that avoids the issue.

Does your project really need to "mix and match" the type and deployment
POMs in many different configurations? In other words, are you sure you
need a graph of POMs, rather than only a tree? As we all know, graphs can
be much trickier to deal with.

I suggest identifying the commonalities across most of your projects, and
including those in a common parent POM. In the case of dependencies and
plugins, ideally all projects will use the same versions of those anyway,
so they can largely be declared at the top level. In places where 9 out of
10 projects use version A, but the 10th uses version B, declare version A
in your toplevel, but override with B in the 10th project's POM. Then there
is no repetition, and you are consistent with Maven's "convention over
configuration" approach.

Maybe some day Maven will support some form of multiple inheritance for
adding in extra common configuration more efficiently. Until then, it
certainly is not perfectly DRY, but you can usually do fairly well.

Regards,
Curtis


On Mon, Apr 2, 2012 at 6:55 AM, Gillet Thomas (2) <
[hidden email]> wrote:

> Hello Wayne,
>
> Seems my first post was really not clear. Trying to make it simpler:
>
> Basically, my main concern is to be able to package separately parts of
> the POM:
> - plugin configuration and/or dependencies (previously the "type POM")
> - distribution management information (previously the "deployment POM")
>
> So I can later define one set of distribution management information per
> project, and one set of configuration per bundle type.
> Then each bundle will, depending on the project it is member of and
> depending on its type, inherit from one set of distribution management
> information, and from one set of configuration.
>
> Hope this is a better explanation.
>
> Then, about your proposals:
>
> > 1. Inheritance does not necessarily follow aggregation.
>
> That's right, I didn't think about aggregator POMs. Actually if properties
> defined in an aggregator POM are accessible in the sub-modules, then it
> could be a place to put shared properties for distribution management.
> But I'm not sure it is the case, and anyway it would make those properties
> available only when building the aggregator. I need to be able to build
> each module alone.
>
> > 2. The maven-remote-resources-plugin can help share resources
> (configuration files etc) between modules
>
> Didn't know that. But I need to share properties, dependencies and plugin
> configurations, not only resources.
>
> > 3. Look into the "import" scope (but this really only helps managing
> dependencies, not necessarily useful to you)
>
> Didn't know that either, thanks to point it out.
> It actually could be used for the sets of configuration which only contain
> dependencies, but when it comes to plugin configuration, again, not working.
>
> Not helping this particular problem I think, but that's indeed good thinks
> to know.
> Thanks for the pointers.
>
> Thomas GILLET
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Avoiding duplicate POM code: profiles, inheritance, properties

ljnelson
In reply to this post by Gillet Thomas (2)
On Mon, Apr 2, 2012 at 7:55 AM, Gillet Thomas (2) <
[hidden email]> wrote:

> Seems my first post was really not clear. Trying to make it simpler:
>
> Basically, my main concern is to be able to package separately parts of
> the POM:
> - plugin configuration and/or dependencies (previously the "type POM")
> - distribution management information (previously the "deployment POM")
>

Yes, this is a clear and obvious thing to want to do.  You are not crazy
for wanting to be able to do this.  But you can't do it in Maven (to the
best of my knowledge).  You can get close though.

You can make a grandparent pom.xml with non-project but company-specific
information in it (company name, maybe the location of your Nexus, various
domain- and project-independent properties).  Ideally this pom.xml changes
rarely, if ever.  Do not be tempted to load it up with plugin version
specifiers and dependency information; this is really just common
boilerplate, and that's it.

You can then make several child poms beneath it that inherit from it that
contain various plugin or pluginManagement stanzas--maybe one, for example,
combines together plugins that are useful for JPA development (or whatever)
and you don't want to keep specifying those plugins over and over and over
again for your various JPA projects.  Just an example.

You can repeat this as many levels deep as you want.

Finally at the end of the inheritance stack here you would have a project
pom.xml that would inherit from one of these "type" poms.  This would allow
you to pick and choose which effective combinations of plugins and
pluginManagement (and/or dependencies, etc.) you might want for that
project (maybe one project needs the

But there will come a time when you discover you want a mix of several of
these pom.xmls.  Maven has no way to import sections from another pom.xml
(sure wish it did, or that it could permit this sort of thing via magic XML
namespaces or xinclude or something; I'm no W3C XML wonk so I don't know
fully what's possible here).  Nor does it offer a way to have a project
extend from several poms at once.

For dependencies (and this one took me probably about a year before I fully
got it), recall that dependencies are transient, so in this one case you
can actually declare a dependency on an artifact of <type>pom</type>.
Think about that for a moment.  If *that* artifact (a pom file) in turn
declares a pile of dependencies, then THOSE dependencies will be pulled in
transitively.

This is a nice hack--and it is a hack--that allows you to at *least* group
dependencies into bundles that you can suck in fairly conveniently, and, to
your point, maintain separately (something that I think maybe got lost in
your original request).

There is no equivalent hack for plugins to my knowledge.

I've usually done a moderately deep inheritance stack and then pretty much
just deal with the duplicate code after that.

I hope this helps you out.  Just another user's data point; always
interested to hear better approaches.

Best,
Laird

--
http://about.me/lairdnelson
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

RE: Avoiding duplicate POM code: profiles, inheritance, properties

Gillet Thomas (2)
Hello all, thanks for all those long replies.

Ron, I would be happy to follow any good practice and to comply to the way maven does things.
But in this particular case, I wasn't able to find anything to comply to. I am indeed describing the usage of a parent POM, but with the exception that only *one* parent POM is likely to be not enough for my purpose.

As pointed out by Curtis and Laird, my problem seems to boil down to a lack of multi-inheritance and/or include mechanism.
By the way, bravo guys, you used the exact words I was trying not to hear. More seriously, I didn't start by the "simple" explanation because I thought it was obviously leading to this issue, and I was hoping there was something else behind that which I had not been able to see. Well, seems there is not.

About transforming my graph into a tree as said Curtis, the only way I can think of is to group all different configurations in one POM, from which I can then inherit to add deployment information for each project.
And the only way to group configurations I know are profiles, but, as said in my first post, I already use profiles for another purpose so it's not working (missing an AND behavior in activations).
Besides, using profiles to produce an alternate artifact seems to be considered bad practice (read this in various place, here is the one I remember: http://java.dzone.com/articles/maven-profile-best-practices).

Anyway, I I'll use one of the dependency packaging methods given by Wayne and Laird, keep inheritance for the plugin configuration, and try something with the properties plugin for the deployment information.
I just found a properties plugin modification making possible to get the property file from a dependency, this way the file can be deployed. Could be a good way to make those deployment information available from any bundle (but I'll have to start writing plugins, not happy with that).

properties-ext-maven-plugin:
    http://stackoverflow.com/questions/1231561/how-to-share-common-properties-among-several-maven-projects/1265428#1265428

A note: to follow Laird remark about POM inclusions, I always wondered what could be forbidding such includes, while it is already possible to "merge" several configurations through profiles.
Maybe it has just never been done (hence the future mixins).

Thanks again for all your valuable advices.

Regards,
Thomas GILLET

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Loading...