how to centralize configuration accross multiple modules

[emerson cargnin]

Hi

I have a integration test with several sub-modules. Currently each one has
its specific configuration. How can I centralise the configuration in 1
single place? I would like to have 1 single file per environment with:

scrum1_team.properties
frontend=st1_fe_server
backend=st1_be_server
...

scrum2_team.properties
frontend=st2_fe_server
backend=st2_be_server
...

And have all modules to use the same file when they need to use some of the
properties.
I would like then to pass a parameter to the mvn command (eg. mvn -Denv=st1)
and it would pick up the appropriate resource file depending on the
environment.

What would be the best way to implement something on these lines?

regards
Emerson

[Jesse Farinacci]

Hi Emerson,

On Tue, Oct 5, 2010 at 8:50 AM, emerson <[hidden email]> wrote:
>
> I would like then to pass a parameter to the mvn command (eg. mvn -Denv=st1)
> and it would pick up the appropriate resource file depending on the
> environment.
> What would be the best way to implement something on these lines?

You should look at
http://maven.apache.org/plugins/maven-remote-resources-plugin/ .. you
could bundle them up in one module, and extract them (via
property-enabled profile) wherever needed. Then your integration tests
just need to pick up your properties bundle.

-Jesse

--
There are 10 types of people in this world, those
that can read binary and those that can not.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Jon Paynter]

For property values -- I setup a .properties file for each of our
environments with the default being 'dev'.  So for a default build, the dev
properties are used.  but when its time to build for QA or Production, you
add a cmd line argument accordingly:   mvn install -DenvType=QA

So I something like this in my top level pom:
<properties>
    <envType>dev</envType> <!-- Default Environment value -->
    <PomRoot> path to root pom </PomRoot>
   ....

and then use:
<build>
    <filters>
      <filter>${PomRoot}/${envType}Config.properties</filter>
    </filters>

Then create:
devConfig.properties
qaConfig.properties
... or whatever environment names you need

This lets me use environment specific properties anywhere in the build.  It
works quite well.

On Tue, Oct 5, 2010 at 5:57 AM, Jesse Farinacci <[hidden email]> wrote:

> Hi Emerson,
>
> On Tue, Oct 5, 2010 at 8:50 AM, emerson <[hidden email]>
> wrote:
> >
> > I would like then to pass a parameter to the mvn command (eg. mvn
> -Denv=st1)
> > and it would pick up the appropriate resource file depending on the
> > environment.
> > What would be the best way to implement something on these lines?
>
> You should look at
> http://maven.apache.org/plugins/maven-remote-resources-plugin/ .. you
> could bundle them up in one module, and extract them (via
> property-enabled profile) wherever needed. Then your integration tests
> just need to pick up your properties bundle.
>
> -Jesse
>
> --
> There are 10 types of people in this world, those
> that can read binary and those that can not.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

[Wayne Fay]

> For property values -- I setup a .properties file for each of our
> environments with the default being 'dev'.  So for a default build, the dev
> properties are used.  but when its time to build for QA or Production, you
> add a cmd line argument accordingly:   mvn install -DenvType=QA

IMO this is an anti-pattern for Maven usage. You should be able to
build one single artifact in say Dev and then promote that same
untouched/changed artifact through your various environments.
Otherwise the artifact that gets QA'ed is not the same artifact that
lands in Production... which defeats the purpose of QA.

You should generally deal with these variances in the environments via
JNDI or other mechanisms available in the platform (JavaSE/JEE or
whatever container you're deploying into).

Wayne

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Jon Paynter]

well it was a suggestion - do what you will with it.

But how would you implement this using the maven way?

given the following restrictions/requirements:
each environment has its own hostname and number of hosts (dev 1 host, qa 4
hosts, prod 8 hosts)
each environment has a different db connection string
each environment has its own set of passwords for accounts (db, unix, and
tibco)
each environment has its own set of port numbers fo the various processes.
each environemnt has its own set of webservice urls.
and all of our j2ee processes have other application specific config values

Add to that, most of the config values change frequently as things evolve
and change.  we found its much easier to put all these values into a single
file,  edit one file, and run a build.  Granted this was with a morass of
ant scripts, but given the volume of changes, the .properties files worked
quite well.

On Tue, Oct 5, 2010 at 10:27 AM, Wayne Fay <[hidden email]> wrote:

> > For property values -- I setup a .properties file for each of our
> > environments with the default being 'dev'.  So for a default build, the
> dev
> > properties are used.  but when its time to build for QA or Production,
> you
> > add a cmd line argument accordingly:   mvn install -DenvType=QA
>
> IMO this is an anti-pattern for Maven usage. You should be able to
> build one single artifact in say Dev and then promote that same
> untouched/changed artifact through your various environments.
> Otherwise the artifact that gets QA'ed is not the same artifact that
> lands in Production... which defeats the purpose of QA.
>
> You should generally deal with these variances in the environments via
> JNDI or other mechanisms available in the platform (JavaSE/JEE or
> whatever container you're deploying into).
>
> Wayne
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

[Yanko, Curtis]

 Each environment identifies itself and your app uses that info to
connect to the right properties at runtime.


________________________________

Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
Making IT Happen, one build at a time

-----Original Message-----
From: Jon Paynter [mailto:[hidden email]]
Sent: Tuesday, October 05, 2010 2:08 PM
To: Maven Users List
Subject: Re: how to centralize configuration accross multiple modules

well it was a suggestion - do what you will with it.

But how would you implement this using the maven way?

given the following restrictions/requirements:
each environment has its own hostname and number of hosts (dev 1 host,
qa 4 hosts, prod 8 hosts) each environment has a different db connection
string each environment has its own set of passwords for accounts (db,
unix, and
tibco)
each environment has its own set of port numbers fo the various
processes.
each environemnt has its own set of webservice urls.
and all of our j2ee processes have other application specific config
values

Add to that, most of the config values change frequently as things
evolve and change.  we found its much easier to put all these values
into a single file,  edit one file, and run a build.  Granted this was
with a morass of ant scripts, but given the volume of changes, the
.properties files worked quite well.

On Tue, Oct 5, 2010 at 10:27 AM, Wayne Fay <[hidden email]> wrote:

> > For property values -- I setup a .properties file for each of our
> > environments with the default being 'dev'.  So for a default build,
> > the
> dev
> > properties are used.  but when its time to build for QA or
> > Production,
> you
> > add a cmd line argument accordingly:   mvn install -DenvType=QA
>
> IMO this is an anti-pattern for Maven usage. You should be able to
> build one single artifact in say Dev and then promote that same
> untouched/changed artifact through your various environments.
> Otherwise the artifact that gets QA'ed is not the same artifact that
> lands in Production... which defeats the purpose of QA.
>
> You should generally deal with these variances in the environments via

> JNDI or other mechanisms available in the platform (JavaSE/JEE or
> whatever container you're deploying into).
>
> Wayne
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Anders Hammar]

I want to second Wayne here and stress that doing different builds for each
environment is NOT the Maven way. I want to stress this as I very often run
into this solution which people have grown used to it and it make it very
hard to set up correct Maven environments with a repository. It is an
anti-pattern when using Maven - there can be just one flavor of an artifact
for a specific version.

Sure, you could solve it by using classifiers and create several artifacts
for each project. But it is so much better to separate the binary and the
configuration. However, I understand that it can sometimes be tricky. Most
of your reqs can be accomplished by just reading the config from a
properties file. You read the properties file from the classpath and then
just make sure the properties file is on the classpath in runtime. I'm sure
all containers have a way to do that keeping the file outside of the
deployable. I mostly use JBoss and there you put it in the conf folder.

Regarding the fact that these config values change, it makes even more sense
keeping them outside of the build. So when a db connection string changes,
you don't have to roll a new build (which MUST be a new version in the Maven
world).

/Anders
On Tue, Oct 5, 2010 at 20:08, Jon Paynter <[hidden email]> wrote:

> well it was a suggestion - do what you will with it.
>
> But how would you implement this using the maven way?
>
> given the following restrictions/requirements:
> each environment has its own hostname and number of hosts (dev 1 host, qa 4
> hosts, prod 8 hosts)
> each environment has a different db connection string
> each environment has its own set of passwords for accounts (db, unix, and
> tibco)
> each environment has its own set of port numbers fo the various processes.
> each environemnt has its own set of webservice urls.
> and all of our j2ee processes have other application specific config values
>
> Add to that, most of the config values change frequently as things evolve
> and change.  we found its much easier to put all these values into a single
> file,  edit one file, and run a build.  Granted this was with a morass of
> ant scripts, but given the volume of changes, the .properties files worked
> quite well.
>
> On Tue, Oct 5, 2010 at 10:27 AM, Wayne Fay <[hidden email]> wrote:
>
> > > For property values -- I setup a .properties file for each of our
> > > environments with the default being 'dev'.  So for a default build, the
> > dev
> > > properties are used.  but when its time to build for QA or Production,
> > you
> > > add a cmd line argument accordingly:   mvn install -DenvType=QA
> >
> > IMO this is an anti-pattern for Maven usage. You should be able to
> > build one single artifact in say Dev and then promote that same
> > untouched/changed artifact through your various environments.
> > Otherwise the artifact that gets QA'ed is not the same artifact that
> > lands in Production... which defeats the purpose of QA.
> >
> > You should generally deal with these variances in the environments via
> > JNDI or other mechanisms available in the platform (JavaSE/JEE or
> > whatever container you're deploying into).
> >
> > Wayne
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
>

[Wayne Fay]

> I want to second Wayne here and stress that doing different builds for each
> environment is NOT the Maven way. I want to stress this as I very often run

This isn't just a violation of "the one true Maven way." It really is
a build anti-pattern that must be stopped any time you run into it --
no matter what build tool you're using, or what kind of app you're
working on.

Separate the configuration from the binary.

Wayne

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Anders Hammar]

Yes, but the problem is the "we did this with Ant" mentality...

/Anders

On Tue, Oct 5, 2010 at 22:01, Wayne Fay <[hidden email]> wrote:

> > I want to second Wayne here and stress that doing different builds for
> each
> > environment is NOT the Maven way. I want to stress this as I very often
> run
>
> This isn't just a violation of "the one true Maven way." It really is
> a build anti-pattern that must be stopped any time you run into it --
> no matter what build tool you're using, or what kind of app you're
> working on.
>
> Separate the configuration from the binary.
>
> Wayne
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

[Jon Paynter]

On Tue, Oct 5, 2010 at 1:07 PM, Anders Hammar <[hidden email]> wrote:

> Yes, but the problem is the "we did this with Ant" mentality...
>
Yep I suspect thats causing me a lot of problems here -- which is why im
asking for suggestions on how else to accomplish the same thing.

Most of our binaries are already setup to use external config files.
But part of the build (filtered resources)  is to generate said external
config files.

But then some of the binary artifacts have environment specific values in
them:
  --ear file for j2ee deployment has libraryPath defined.
  --rar file for j2ee deployment has user/password values

To be able to run 1 build and generate stuff for all 7 of our environments
at the same time would be a serious boon to getting my company to switch to
maven.  but asking/requiring a major re-architecting of the existing setup
and coding will guarantee its not going to happen -- something i dont want
to see.

But re-arranging/re-formatting config files, or putting the values in
different places is no big deal.

[Leon Rosenberg]

On Tue, Oct 5, 2010 at 2:50 PM, emerson <[hidden email]> wrote:
> Hi
>...
> I would like then to pass a parameter to the mvn command (eg. mvn -Denv=st1)
> and it would pick up the appropriate resource file depending on the
> environment.
>
Hi,

I don't know whether it is what you want, but configureme supports
exactly this kind of configuration:
define different, cascading realms, provide the realm you are in as
startparameter and the framework decides which value is the proper one
in current environment.

http://infra.anotheria.net/confluence/display/CONFIGUREME/GettingStarted
http://www.configureme.org

regards
Leon

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[emerson cargnin]

Hi Wayne

For app deployment we use anthill in such a way that we use the same build
for all environments and the resources are processed during the deployment.

that said, this is a special case, as this project and its modules are all
jbehave and selenium projects, which won't generate artifacts and will just
run the tests.

But I agree with you in that this is somewhat of a anti-pattern when used to
deploy final artifacts.

Regards
Emerson

On 5 October 2010 18:27, Wayne Fay <[hidden email]> wrote:

> > For property values -- I setup a .properties file for each of our
> > environments with the default being 'dev'.  So for a default build, the
> dev
> > properties are used.  but when its time to build for QA or Production,
> you
> > add a cmd line argument accordingly:   mvn install -DenvType=QA
>
> IMO this is an anti-pattern for Maven usage. You should be able to
> build one single artifact in say Dev and then promote that same
> untouched/changed artifact through your various environments.
> Otherwise the artifact that gets QA'ed is not the same artifact that
> lands in Production... which defeats the purpose of QA.
>
> You should generally deal with these variances in the environments via
> JNDI or other mechanisms available in the platform (JavaSE/JEE or
> whatever container you're deploying into).
>
> Wayne
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

[emerson cargnin]

Hi

I implemented using the following:

In the parent pom:

     <properties>
      <envType>local</envType> <!-- Default Environment value -->
      <envProps>../autotestProperties/</envProps>
    </properties>
<build>
    <resources>
      <resource>
        <directory>${envProps}/resources</directory>
        <filtering>true</filtering>
      </resource>
    </resources>
    <filters>
        <filter>${envProps}/env/${envType}.properties</filter>
    </filters>

As far as maven goes, it works fine, but when I generate eclipse project, it
tries to add the resource entry, but all it gets is a incorrect build path
entry:

"Build path entry is missing:
home/emerson/workspace/trunk/autotests/autotestProperties/resources"

Is there anyway to stop eclipse from trying to create this entry?

The normal use of this will be for the developers and qa to create the
project structure using a maven eclipse:eclipse  install -DskipTests build
once via maven

On 6 October 2010 11:31, emerson <[hidden email]> wrote:

> Hi Wayne
>
> For app deployment we use anthill in such a way that we use the same build
> for all environments and the resources are processed during the deployment.
>
> that said, this is a special case, as this project and its modules are all
> jbehave and selenium projects, which won't generate artifacts and will just
> run the tests.
>
> But I agree with you in that this is somewhat of a anti-pattern when used
> to deploy final artifacts.
>
> Regards
> Emerson
>
>
> On 5 October 2010 18:27, Wayne Fay <[hidden email]> wrote:
>
>> > For property values -- I setup a .properties file for each of our
>> > environments with the default being 'dev'.  So for a default build, the
>> dev
>> > properties are used.  but when its time to build for QA or Production,
>> you
>> > add a cmd line argument accordingly:   mvn install -DenvType=QA
>>
>> IMO this is an anti-pattern for Maven usage. You should be able to
>> build one single artifact in say Dev and then promote that same
>> untouched/changed artifact through your various environments.
>> Otherwise the artifact that gets QA'ed is not the same artifact that
>> lands in Production... which defeats the purpose of QA.
>>
>> You should generally deal with these variances in the environments via
>> JNDI or other mechanisms available in the platform (JavaSE/JEE or
>> whatever container you're deploying into).
>>
>> Wayne
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>