Can Maven be used in an nmake environment with VPATH?

classic Classic list List threaded Threaded
17 messages Options
Reply | Threaded
Open this post in threaded view
|

Can Maven be used in an nmake environment with VPATH?

scohen
I work for an organization which uses an SCM/Build process based on the
following:

SCM: a ancient legacy horror of a system

Build: Alcatel-Lucent nmake

With this system the organization maintains a large suite of
applications.  The system is monstrously inflexible and a pain to work
with.  They do manage to have an automated build process with it, but no
continuous integration.

A large proportion of the actual code built by this system is java.
Deployment is onto various servers using versions of containers such as
weblogic, or sometimes standalone. This requires old JVMs, a few of
which are as old as JDK-1.3, and none use a version of java that is
still supported by Oracle (>=1.7).  Deployment is done through RPMs and
in some cases Solaris packaging.

As you might imagine, change, in such an organization is difficult.  The
main impediment to change is the accreted base of thousands of makefiles
that have been created over the years.

But a few intrepid (or maybe foolhardy) souls are thinking of trying.
We'd like to use maven to handle the java portion of this process.  Its
dependency management features would be worth the effort if we could get
them.  Since replacing the whole system is not in scope, the idea is to
use maven to handle the java compilation, archiving into jars, wars,
ears, etc., while leaving the packaging, deployment, source control
systems as they are.  Alcatel-Lucent nmake would invoke maven as it now
invokes javac, jar, etc.  If we can get this far, future upgrading of
other portions of the system may come into play, but not in step 1.
Such a transition will happen gradually or not at all.

The problem is this.  Alcatel-Lucent nmake (and other versions of make
such as GNU make) includes the concept of the VPATH, an environment
variable containing a path (similar to PATH, etc.) along which to search
for dependent source.  If a necessary file is not found along the first
node of the path, the second is searched for it, then the third, etc.
Only if the full VPATH is exhausted is the dependency not satisfied and
the build fails.  Importantly, if the dependency IS satisfied, then
nodes further down the path are not looked at for that dependency.

There is a little tutorial here, explaining how this works:
http://nmake.alcatel-lucent.com/tutorial/s10.html

Needless to say, this is not the way Maven works, especially the
compiler plugin, certainly not under default settings.  There is the
sourcepath setting which invokes the -sourcepath switch on javac, which
might be part of a solution.  There would then be a need for something
that could translate the $VPATH envvar to a sourcepath which would need
to dig down through several layers of a directory tree (at least they
would be identical in each node -e.g. $NODE/$PROJECT/src/main/java) to
produce a sourcepath.

I don't think this will work because if I turn on the verbose debug
output, I see that maven is putting a path to each source file on the
javac command line, and am guessing maven is not going to do this
looking over each node of the VPATH.

Another option would be to pull the source from the various vpath nodes
in reverse order and then use maven in a more normal way.  But I imagine
that this would have negative performance consequences.

Has anyone on this list ever tried anything like this?  Or is this too
big a hill to even contemplate climbing?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Can Maven be used in an nmake environment with VPATH?

cody.a.fyler
Have you looked into using Jenkins or Hudson to automate those builds?

You can set up custom environment variables, different jvms, etc. etc. It can run any number of utilities via shell scripts, Maven, Ant, Gradle, etc. etc.

That might be a more fruitful place to start.

Cody Fyler
Lending Grid Build Team
G=Lending Grid Builds
(515) – 441 - 0814

-----Original Message-----
From: Steve Cohen [mailto:[hidden email]]
Sent: Tuesday, March 31, 2015 1:26 PM
To: [hidden email]
Subject: Can Maven be used in an nmake environment with VPATH?

I work for an organization which uses an SCM/Build process based on the
following:

SCM: a ancient legacy horror of a system

Build: Alcatel-Lucent nmake

With this system the organization maintains a large suite of applications.  The system is monstrously inflexible and a pain to work with.  They do manage to have an automated build process with it, but no continuous integration.

A large proportion of the actual code built by this system is java.
Deployment is onto various servers using versions of containers such as weblogic, or sometimes standalone. This requires old JVMs, a few of which are as old as JDK-1.3, and none use a version of java that is still supported by Oracle (>=1.7).  Deployment is done through RPMs and in some cases Solaris packaging.

As you might imagine, change, in such an organization is difficult.  The main impediment to change is the accreted base of thousands of makefiles that have been created over the years.

But a few intrepid (or maybe foolhardy) souls are thinking of trying.
We'd like to use maven to handle the java portion of this process.  Its dependency management features would be worth the effort if we could get them.  Since replacing the whole system is not in scope, the idea is to use maven to handle the java compilation, archiving into jars, wars, ears, etc., while leaving the packaging, deployment, source control systems as they are.  Alcatel-Lucent nmake would invoke maven as it now invokes javac, jar, etc.  If we can get this far, future upgrading of other portions of the system may come into play, but not in step 1.
Such a transition will happen gradually or not at all.

The problem is this.  Alcatel-Lucent nmake (and other versions of make such as GNU make) includes the concept of the VPATH, an environment variable containing a path (similar to PATH, etc.) along which to search for dependent source.  If a necessary file is not found along the first node of the path, the second is searched for it, then the third, etc.
Only if the full VPATH is exhausted is the dependency not satisfied and the build fails.  Importantly, if the dependency IS satisfied, then nodes further down the path are not looked at for that dependency.

There is a little tutorial here, explaining how this works:
http://nmake.alcatel-lucent.com/tutorial/s10.html

Needless to say, this is not the way Maven works, especially the compiler plugin, certainly not under default settings.  There is the sourcepath setting which invokes the -sourcepath switch on javac, which might be part of a solution.  There would then be a need for something that could translate the $VPATH envvar to a sourcepath which would need to dig down through several layers of a directory tree (at least they would be identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a sourcepath.

I don't think this will work because if I turn on the verbose debug output, I see that maven is putting a path to each source file on the javac command line, and am guessing maven is not going to do this looking over each node of the VPATH.

Another option would be to pull the source from the various vpath nodes in reverse order and then use maven in a more normal way.  But I imagine that this would have negative performance consequences.

Has anyone on this list ever tried anything like this?  Or is this too big a hill to even contemplate climbing?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
As I indicated in my post, we need a gradual transition, and changing
the automated build part of it would be the last step rather than the
first step.  We want to integrate maven into an existing nmake-based
process rather than change the entire automated build process first.

At some point down the road the goal would be to use something like
Hudson/Jenkins and CI, but we'll never get there if we have to redo
everything first.

On 03/31/2015 02:12 PM, [hidden email] wrote:

> Have you looked into using Jenkins or Hudson to automate those builds?
>
> You can set up custom environment variables, different jvms, etc. etc. It can run any number of utilities via shell scripts, Maven, Ant, Gradle, etc. etc.
>
> That might be a more fruitful place to start.
>
> Cody Fyler
> Lending Grid Build Team
> G=Lending Grid Builds
> (515) – 441 - 0814
>
> -----Original Message-----
> From: Steve Cohen [mailto:[hidden email]]
> Sent: Tuesday, March 31, 2015 1:26 PM
> To: [hidden email]
> Subject: Can Maven be used in an nmake environment with VPATH?
>
> I work for an organization which uses an SCM/Build process based on the
> following:
>
> SCM: a ancient legacy horror of a system
>
> Build: Alcatel-Lucent nmake
>
> With this system the organization maintains a large suite of applications.  The system is monstrously inflexible and a pain to work with.  They do manage to have an automated build process with it, but no continuous integration.
>
> A large proportion of the actual code built by this system is java.
> Deployment is onto various servers using versions of containers such as weblogic, or sometimes standalone. This requires old JVMs, a few of which are as old as JDK-1.3, and none use a version of java that is still supported by Oracle (>=1.7).  Deployment is done through RPMs and in some cases Solaris packaging.
>
> As you might imagine, change, in such an organization is difficult.  The main impediment to change is the accreted base of thousands of makefiles that have been created over the years.
>
> But a few intrepid (or maybe foolhardy) souls are thinking of trying.
> We'd like to use maven to handle the java portion of this process.  Its dependency management features would be worth the effort if we could get them.  Since replacing the whole system is not in scope, the idea is to use maven to handle the java compilation, archiving into jars, wars, ears, etc., while leaving the packaging, deployment, source control systems as they are.  Alcatel-Lucent nmake would invoke maven as it now invokes javac, jar, etc.  If we can get this far, future upgrading of other portions of the system may come into play, but not in step 1.
> Such a transition will happen gradually or not at all.
>
> The problem is this.  Alcatel-Lucent nmake (and other versions of make such as GNU make) includes the concept of the VPATH, an environment variable containing a path (similar to PATH, etc.) along which to search for dependent source.  If a necessary file is not found along the first node of the path, the second is searched for it, then the third, etc.
> Only if the full VPATH is exhausted is the dependency not satisfied and the build fails.  Importantly, if the dependency IS satisfied, then nodes further down the path are not looked at for that dependency.
>
> There is a little tutorial here, explaining how this works:
> http://nmake.alcatel-lucent.com/tutorial/s10.html
>
> Needless to say, this is not the way Maven works, especially the compiler plugin, certainly not under default settings.  There is the sourcepath setting which invokes the -sourcepath switch on javac, which might be part of a solution.  There would then be a need for something that could translate the $VPATH envvar to a sourcepath which would need to dig down through several layers of a directory tree (at least they would be identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a sourcepath.
>
> I don't think this will work because if I turn on the verbose debug output, I see that maven is putting a path to each source file on the javac command line, and am guessing maven is not going to do this looking over each node of the VPATH.
>
> Another option would be to pull the source from the various vpath nodes in reverse order and then use maven in a more normal way.  But I imagine that this would have negative performance consequences.
>
> Has anyone on this list ever tried anything like this?  Or is this too big a hill to even contemplate climbing?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

kaosko
In reply to this post by scohen
build-helper-maven-plugin allows you to add multiple sourcepaths to the
build, which could be part of the answer. I've never attempted scripting
the sourcepaths in but might be possible. In any case, I've done something
similar in the past in an ancient c+java monolithic make build environment
paired with ClearCase. Good times. Anyway, our solution was to introduce a
Maven repo manager (Nexus in our case) and start modularizing, packaging up
and versioning the Java parts properly. If you can use a repo manager, it's
much easier to write custom Maven builds to deal with pre-built
dependencies than try to add a Maven build on top of a non-standard,
convoluted directory hierarchy. Of the 4+h build times where only 1-2% of
the codebase would ever change, we got down to ~1h builds, with most of it
spent on the C compilation and only a few minutes on the Java side. If
there's any good side dealing with legacy envrionments, it's that a large
part of the codebase is usually stable and really doesn't have to be
compiled over and over again for every build.

Kalle

On Tue, Mar 31, 2015 at 11:25 AM, Steve Cohen <[hidden email]>
wrote:

> I work for an organization which uses an SCM/Build process based on the
> following:
>
> SCM: a ancient legacy horror of a system
>
> Build: Alcatel-Lucent nmake
>
> With this system the organization maintains a large suite of
> applications.  The system is monstrously inflexible and a pain to work
> with.  They do manage to have an automated build process with it, but no
> continuous integration.
>
> A large proportion of the actual code built by this system is java.
> Deployment is onto various servers using versions of containers such as
> weblogic, or sometimes standalone. This requires old JVMs, a few of which
> are as old as JDK-1.3, and none use a version of java that is still
> supported by Oracle (>=1.7).  Deployment is done through RPMs and in some
> cases Solaris packaging.
>
> As you might imagine, change, in such an organization is difficult.  The
> main impediment to change is the accreted base of thousands of makefiles
> that have been created over the years.
>
> But a few intrepid (or maybe foolhardy) souls are thinking of trying. We'd
> like to use maven to handle the java portion of this process.  Its
> dependency management features would be worth the effort if we could get
> them.  Since replacing the whole system is not in scope, the idea is to use
> maven to handle the java compilation, archiving into jars, wars, ears,
> etc., while leaving the packaging, deployment, source control systems as
> they are.  Alcatel-Lucent nmake would invoke maven as it now invokes javac,
> jar, etc.  If we can get this far, future upgrading of other portions of
> the system may come into play, but not in step 1. Such a transition will
> happen gradually or not at all.
>
> The problem is this.  Alcatel-Lucent nmake (and other versions of make
> such as GNU make) includes the concept of the VPATH, an environment
> variable containing a path (similar to PATH, etc.) along which to search
> for dependent source.  If a necessary file is not found along the first
> node of the path, the second is searched for it, then the third, etc. Only
> if the full VPATH is exhausted is the dependency not satisfied and the
> build fails.  Importantly, if the dependency IS satisfied, then nodes
> further down the path are not looked at for that dependency.
>
> There is a little tutorial here, explaining how this works:
> http://nmake.alcatel-lucent.com/tutorial/s10.html
>
> Needless to say, this is not the way Maven works, especially the compiler
> plugin, certainly not under default settings.  There is the sourcepath
> setting which invokes the -sourcepath switch on javac, which might be part
> of a solution.  There would then be a need for something that could
> translate the $VPATH envvar to a sourcepath which would need to dig down
> through several layers of a directory tree (at least they would be
> identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a
> sourcepath.
>
> I don't think this will work because if I turn on the verbose debug
> output, I see that maven is putting a path to each source file on the javac
> command line, and am guessing maven is not going to do this looking over
> each node of the VPATH.
>
> Another option would be to pull the source from the various vpath nodes in
> reverse order and then use maven in a more normal way.  But I imagine that
> this would have negative performance consequences.
>
> Has anyone on this list ever tried anything like this?  Or is this too big
> a hill to even contemplate climbing?
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
Thanks Kalle, but I don't think this meets the need either.

Note
> Importantly, if the dependency IS satisfied, then nodes
> further down the path are not looked at for that dependency.

This plugin simply provides a way to add additional source paths, but
not with any logic governing their interrelation and use, which is required.

Steve

On 03/31/2015 03:03 PM, Kalle Korhonen wrote:

> build-helper-maven-plugin allows you to add multiple sourcepaths to the
> build, which could be part of the answer. I've never attempted scripting
> the sourcepaths in but might be possible. In any case, I've done something
> similar in the past in an ancient c+java monolithic make build environment
> paired with ClearCase. Good times. Anyway, our solution was to introduce a
> Maven repo manager (Nexus in our case) and start modularizing, packaging up
> and versioning the Java parts properly. If you can use a repo manager, it's
> much easier to write custom Maven builds to deal with pre-built
> dependencies than try to add a Maven build on top of a non-standard,
> convoluted directory hierarchy. Of the 4+h build times where only 1-2% of
> the codebase would ever change, we got down to ~1h builds, with most of it
> spent on the C compilation and only a few minutes on the Java side. If
> there's any good side dealing with legacy envrionments, it's that a large
> part of the codebase is usually stable and really doesn't have to be
> compiled over and over again for every build.
>
> Kalle
>
> On Tue, Mar 31, 2015 at 11:25 AM, Steve Cohen <[hidden email]>
> wrote:
>
>> I work for an organization which uses an SCM/Build process based on the
>> following:
>>
>> SCM: a ancient legacy horror of a system
>>
>> Build: Alcatel-Lucent nmake
>>
>> With this system the organization maintains a large suite of
>> applications.  The system is monstrously inflexible and a pain to work
>> with.  They do manage to have an automated build process with it, but no
>> continuous integration.
>>
>> A large proportion of the actual code built by this system is java.
>> Deployment is onto various servers using versions of containers such as
>> weblogic, or sometimes standalone. This requires old JVMs, a few of which
>> are as old as JDK-1.3, and none use a version of java that is still
>> supported by Oracle (>=1.7).  Deployment is done through RPMs and in some
>> cases Solaris packaging.
>>
>> As you might imagine, change, in such an organization is difficult.  The
>> main impediment to change is the accreted base of thousands of makefiles
>> that have been created over the years.
>>
>> But a few intrepid (or maybe foolhardy) souls are thinking of trying. We'd
>> like to use maven to handle the java portion of this process.  Its
>> dependency management features would be worth the effort if we could get
>> them.  Since replacing the whole system is not in scope, the idea is to use
>> maven to handle the java compilation, archiving into jars, wars, ears,
>> etc., while leaving the packaging, deployment, source control systems as
>> they are.  Alcatel-Lucent nmake would invoke maven as it now invokes javac,
>> jar, etc.  If we can get this far, future upgrading of other portions of
>> the system may come into play, but not in step 1. Such a transition will
>> happen gradually or not at all.
>>
>> The problem is this.  Alcatel-Lucent nmake (and other versions of make
>> such as GNU make) includes the concept of the VPATH, an environment
>> variable containing a path (similar to PATH, etc.) along which to search
>> for dependent source.  If a necessary file is not found along the first
>> node of the path, the second is searched for it, then the third, etc. Only
>> if the full VPATH is exhausted is the dependency not satisfied and the
>> build fails.  Importantly, if the dependency IS satisfied, then nodes
>> further down the path are not looked at for that dependency.
>>
>> There is a little tutorial here, explaining how this works:
>> http://nmake.alcatel-lucent.com/tutorial/s10.html
>>
>> Needless to say, this is not the way Maven works, especially the compiler
>> plugin, certainly not under default settings.  There is the sourcepath
>> setting which invokes the -sourcepath switch on javac, which might be part
>> of a solution.  There would then be a need for something that could
>> translate the $VPATH envvar to a sourcepath which would need to dig down
>> through several layers of a directory tree (at least they would be
>> identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a
>> sourcepath.
>>
>> I don't think this will work because if I turn on the verbose debug
>> output, I see that maven is putting a path to each source file on the javac
>> command line, and am guessing maven is not going to do this looking over
>> each node of the VPATH.
>>
>> Another option would be to pull the source from the various vpath nodes in
>> reverse order and then use maven in a more normal way.  But I imagine that
>> this would have negative performance consequences.
>>
>> Has anyone on this list ever tried anything like this?  Or is this too big
>> a hill to even contemplate climbing?
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
On 03/31/2015 05:07 PM, Steve Cohen wrote:
Okay, I am thinking that this could be best done by writing a custom
plugin.  This plugin would walk down the VPATH, capturing the names of
all the java files to be compiled, under each element in the VPATH, and
pruning the list according to the VPATH logic:

In other words, given a vpath of "
        vpath/patha:external/vpathb:external/vpathc
it would first find all the java files under
        vpath/patha/src/main/java
It would then scan
        vpath/external/vpathb
adding to the list only those java files not already in the list from
the first iteration
and finally scan
        vpath/external/vpathc and add only those not already in the list
from the first two iterations, etc.

This list would then be passed to javac

My question is whether there is a hook

> Thanks Kalle, but I don't think this meets the need either.
>
> Note
>> Importantly, if the dependency IS satisfied, then nodes
>> further down the path are not looked at for that dependency.
>
> This plugin simply provides a way to add additional source paths, but
> not with any logic governing their interrelation and use, which is
> required.
>
> Steve
>
> On 03/31/2015 03:03 PM, Kalle Korhonen wrote:
>> build-helper-maven-plugin allows you to add multiple sourcepaths to the
>> build, which could be part of the answer. I've never attempted scripting
>> the sourcepaths in but might be possible. In any case, I've done
>> something
>> similar in the past in an ancient c+java monolithic make build
>> environment
>> paired with ClearCase. Good times. Anyway, our solution was to
>> introduce a
>> Maven repo manager (Nexus in our case) and start modularizing,
>> packaging up
>> and versioning the Java parts properly. If you can use a repo manager,
>> it's
>> much easier to write custom Maven builds to deal with pre-built
>> dependencies than try to add a Maven build on top of a non-standard,
>> convoluted directory hierarchy. Of the 4+h build times where only 1-2% of
>> the codebase would ever change, we got down to ~1h builds, with most
>> of it
>> spent on the C compilation and only a few minutes on the Java side. If
>> there's any good side dealing with legacy envrionments, it's that a large
>> part of the codebase is usually stable and really doesn't have to be
>> compiled over and over again for every build.
>>
>> Kalle
>>
>> On Tue, Mar 31, 2015 at 11:25 AM, Steve Cohen <[hidden email]>
>> wrote:
>>
>>> I work for an organization which uses an SCM/Build process based on the
>>> following:
>>>
>>> SCM: a ancient legacy horror of a system
>>>
>>> Build: Alcatel-Lucent nmake
>>>
>>> With this system the organization maintains a large suite of
>>> applications.  The system is monstrously inflexible and a pain to work
>>> with.  They do manage to have an automated build process with it, but no
>>> continuous integration.
>>>
>>> A large proportion of the actual code built by this system is java.
>>> Deployment is onto various servers using versions of containers such as
>>> weblogic, or sometimes standalone. This requires old JVMs, a few of
>>> which
>>> are as old as JDK-1.3, and none use a version of java that is still
>>> supported by Oracle (>=1.7).  Deployment is done through RPMs and in
>>> some
>>> cases Solaris packaging.
>>>
>>> As you might imagine, change, in such an organization is difficult.  The
>>> main impediment to change is the accreted base of thousands of makefiles
>>> that have been created over the years.
>>>
>>> But a few intrepid (or maybe foolhardy) souls are thinking of trying.
>>> We'd
>>> like to use maven to handle the java portion of this process.  Its
>>> dependency management features would be worth the effort if we could get
>>> them.  Since replacing the whole system is not in scope, the idea is
>>> to use
>>> maven to handle the java compilation, archiving into jars, wars, ears,
>>> etc., while leaving the packaging, deployment, source control systems as
>>> they are.  Alcatel-Lucent nmake would invoke maven as it now invokes
>>> javac,
>>> jar, etc.  If we can get this far, future upgrading of other portions of
>>> the system may come into play, but not in step 1. Such a transition will
>>> happen gradually or not at all.
>>>
>>> The problem is this.  Alcatel-Lucent nmake (and other versions of make
>>> such as GNU make) includes the concept of the VPATH, an environment
>>> variable containing a path (similar to PATH, etc.) along which to search
>>> for dependent source.  If a necessary file is not found along the first
>>> node of the path, the second is searched for it, then the third, etc.
>>> Only
>>> if the full VPATH is exhausted is the dependency not satisfied and the
>>> build fails.  Importantly, if the dependency IS satisfied, then nodes
>>> further down the path are not looked at for that dependency.
>>>
>>> There is a little tutorial here, explaining how this works:
>>> http://nmake.alcatel-lucent.com/tutorial/s10.html
>>>
>>> Needless to say, this is not the way Maven works, especially the
>>> compiler
>>> plugin, certainly not under default settings.  There is the sourcepath
>>> setting which invokes the -sourcepath switch on javac, which might be
>>> part
>>> of a solution.  There would then be a need for something that could
>>> translate the $VPATH envvar to a sourcepath which would need to dig down
>>> through several layers of a directory tree (at least they would be
>>> identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a
>>> sourcepath.
>>>
>>> I don't think this will work because if I turn on the verbose debug
>>> output, I see that maven is putting a path to each source file on the
>>> javac
>>> command line, and am guessing maven is not going to do this looking over
>>> each node of the VPATH.
>>>
>>> Another option would be to pull the source from the various vpath
>>> nodes in
>>> reverse order and then use maven in a more normal way.  But I imagine
>>> that
>>> this would have negative performance consequences.
>>>
>>> Has anyone on this list ever tried anything like this?  Or is this
>>> too big
>>> a hill to even contemplate climbing?
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
Please disregard my previous message.  I inadvertently pressed "send"
before it was finished.  Sorry.

On 04/01/2015 08:43 AM, Steve Cohen wrote:

> On 03/31/2015 05:07 PM, Steve Cohen wrote:
> Okay, I am thinking that this could be best done by writing a custom
> plugin.  This plugin would walk down the VPATH, capturing the names of
> all the java files to be compiled, under each element in the VPATH, and
> pruning the list according to the VPATH logic:
>
> In other words, given a vpath of "
>      vpath/patha:external/vpathb:external/vpathc
> it would first find all the java files under
>      vpath/patha/src/main/java
> It would then scan
>      vpath/external/vpathb
> adding to the list only those java files not already in the list from
> the first iteration
> and finally scan
>      vpath/external/vpathc and add only those not already in the list
> from the first two iterations, etc.
>
> This list would then be passed to javac
>
> My question is whether there is a hook
>> Thanks Kalle, but I don't think this meets the need either.
>>
>> Note
>>> Importantly, if the dependency IS satisfied, then nodes
>>> further down the path are not looked at for that dependency.
>>
>> This plugin simply provides a way to add additional source paths, but
>> not with any logic governing their interrelation and use, which is
>> required.
>>
>> Steve
>>
>> On 03/31/2015 03:03 PM, Kalle Korhonen wrote:
>>> build-helper-maven-plugin allows you to add multiple sourcepaths to the
>>> build, which could be part of the answer. I've never attempted scripting
>>> the sourcepaths in but might be possible. In any case, I've done
>>> something
>>> similar in the past in an ancient c+java monolithic make build
>>> environment
>>> paired with ClearCase. Good times. Anyway, our solution was to
>>> introduce a
>>> Maven repo manager (Nexus in our case) and start modularizing,
>>> packaging up
>>> and versioning the Java parts properly. If you can use a repo manager,
>>> it's
>>> much easier to write custom Maven builds to deal with pre-built
>>> dependencies than try to add a Maven build on top of a non-standard,
>>> convoluted directory hierarchy. Of the 4+h build times where only
>>> 1-2% of
>>> the codebase would ever change, we got down to ~1h builds, with most
>>> of it
>>> spent on the C compilation and only a few minutes on the Java side. If
>>> there's any good side dealing with legacy envrionments, it's that a
>>> large
>>> part of the codebase is usually stable and really doesn't have to be
>>> compiled over and over again for every build.
>>>
>>> Kalle
>>>
>>> On Tue, Mar 31, 2015 at 11:25 AM, Steve Cohen <[hidden email]>
>>> wrote:
>>>
>>>> I work for an organization which uses an SCM/Build process based on the
>>>> following:
>>>>
>>>> SCM: a ancient legacy horror of a system
>>>>
>>>> Build: Alcatel-Lucent nmake
>>>>
>>>> With this system the organization maintains a large suite of
>>>> applications.  The system is monstrously inflexible and a pain to work
>>>> with.  They do manage to have an automated build process with it,
>>>> but no
>>>> continuous integration.
>>>>
>>>> A large proportion of the actual code built by this system is java.
>>>> Deployment is onto various servers using versions of containers such as
>>>> weblogic, or sometimes standalone. This requires old JVMs, a few of
>>>> which
>>>> are as old as JDK-1.3, and none use a version of java that is still
>>>> supported by Oracle (>=1.7).  Deployment is done through RPMs and in
>>>> some
>>>> cases Solaris packaging.
>>>>
>>>> As you might imagine, change, in such an organization is difficult.
>>>> The
>>>> main impediment to change is the accreted base of thousands of
>>>> makefiles
>>>> that have been created over the years.
>>>>
>>>> But a few intrepid (or maybe foolhardy) souls are thinking of trying.
>>>> We'd
>>>> like to use maven to handle the java portion of this process.  Its
>>>> dependency management features would be worth the effort if we could
>>>> get
>>>> them.  Since replacing the whole system is not in scope, the idea is
>>>> to use
>>>> maven to handle the java compilation, archiving into jars, wars, ears,
>>>> etc., while leaving the packaging, deployment, source control
>>>> systems as
>>>> they are.  Alcatel-Lucent nmake would invoke maven as it now invokes
>>>> javac,
>>>> jar, etc.  If we can get this far, future upgrading of other
>>>> portions of
>>>> the system may come into play, but not in step 1. Such a transition
>>>> will
>>>> happen gradually or not at all.
>>>>
>>>> The problem is this.  Alcatel-Lucent nmake (and other versions of make
>>>> such as GNU make) includes the concept of the VPATH, an environment
>>>> variable containing a path (similar to PATH, etc.) along which to
>>>> search
>>>> for dependent source.  If a necessary file is not found along the first
>>>> node of the path, the second is searched for it, then the third, etc.
>>>> Only
>>>> if the full VPATH is exhausted is the dependency not satisfied and the
>>>> build fails.  Importantly, if the dependency IS satisfied, then nodes
>>>> further down the path are not looked at for that dependency.
>>>>
>>>> There is a little tutorial here, explaining how this works:
>>>> http://nmake.alcatel-lucent.com/tutorial/s10.html
>>>>
>>>> Needless to say, this is not the way Maven works, especially the
>>>> compiler
>>>> plugin, certainly not under default settings.  There is the sourcepath
>>>> setting which invokes the -sourcepath switch on javac, which might be
>>>> part
>>>> of a solution.  There would then be a need for something that could
>>>> translate the $VPATH envvar to a sourcepath which would need to dig
>>>> down
>>>> through several layers of a directory tree (at least they would be
>>>> identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a
>>>> sourcepath.
>>>>
>>>> I don't think this will work because if I turn on the verbose debug
>>>> output, I see that maven is putting a path to each source file on the
>>>> javac
>>>> command line, and am guessing maven is not going to do this looking
>>>> over
>>>> each node of the VPATH.
>>>>
>>>> Another option would be to pull the source from the various vpath
>>>> nodes in
>>>> reverse order and then use maven in a more normal way.  But I imagine
>>>> that
>>>> this would have negative performance consequences.
>>>>
>>>> Has anyone on this list ever tried anything like this?  Or is this
>>>> too big
>>>> a hill to even contemplate climbing?
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [hidden email]
>>>> For additional commands, e-mail: [hidden email]
>>>>
>>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
In reply to this post by Steve Cohen-2
Previously I prematurely posted something to the list and then asked
people to ignore it.  This is the completed version of that post.
Please read and respond to it.

========================================================================

Okay, I am thinking that this could best be done by writing a custom
plugin.  This plugin would walk down the VPATH, capturing the names of
all the java files to be compiled, under each element in the VPATH, and
pruning the list according to the VPATH logic:

In other words, given a VPATH of "
     vpath/patha:external/vpathb:external/vpathc
it would first find all the java files under
     vpath/patha/src/main/java
It would then scan
     vpath/external/vpathb
adding to the list only those java files not already in the list from
the first iteration
and finally scan
     vpath/external/vpathc and add only those not already in the list
from the first two iterations, etc.

It would seem that I would need to somehow "extend" the Maven Compiler
plugin by creating a custom Mojo (extending CompilerMojo) that would use
a custom SourceInclusionScanner
to perform the selection that the VPATH logic demands.  Once this is
done, I would want all the rest of the maven compiler plugin to be executed.

I have never created a plugin before.  How difficult, or doable, is it
to "extend" a plugin in this way? Can someone point to an example where
such an extension has been done?

Thanks.


On 03/31/2015 05:07 PM, Steve Cohen wrote:

> Thanks Kalle, but I don't think this meets the need either.
>
> Note
>> Importantly, if the dependency IS satisfied, then nodes
>> further down the path are not looked at for that dependency.
>
> This plugin simply provides a way to add additional source paths, but
> not with any logic governing their interrelation and use, which is
> required.
>
> Steve
>
> On 03/31/2015 03:03 PM, Kalle Korhonen wrote:
>> build-helper-maven-plugin allows you to add multiple sourcepaths to the
>> build, which could be part of the answer. I've never attempted scripting
>> the sourcepaths in but might be possible. In any case, I've done
>> something
>> similar in the past in an ancient c+java monolithic make build
>> environment
>> paired with ClearCase. Good times. Anyway, our solution was to
>> introduce a
>> Maven repo manager (Nexus in our case) and start modularizing,
>> packaging up
>> and versioning the Java parts properly. If you can use a repo manager,
>> it's
>> much easier to write custom Maven builds to deal with pre-built
>> dependencies than try to add a Maven build on top of a non-standard,
>> convoluted directory hierarchy. Of the 4+h build times where only 1-2% of
>> the codebase would ever change, we got down to ~1h builds, with most
>> of it
>> spent on the C compilation and only a few minutes on the Java side. If
>> there's any good side dealing with legacy envrionments, it's that a large
>> part of the codebase is usually stable and really doesn't have to be
>> compiled over and over again for every build.
>>
>> Kalle
>>
>> On Tue, Mar 31, 2015 at 11:25 AM, Steve Cohen <[hidden email]>
>> wrote:
>>
>>> I work for an organization which uses an SCM/Build process based on the
>>> following:
>>>
>>> SCM: a ancient legacy horror of a system
>>>
>>> Build: Alcatel-Lucent nmake
>>>
>>> With this system the organization maintains a large suite of
>>> applications.  The system is monstrously inflexible and a pain to work
>>> with.  They do manage to have an automated build process with it, but no
>>> continuous integration.
>>>
>>> A large proportion of the actual code built by this system is java.
>>> Deployment is onto various servers using versions of containers such as
>>> weblogic, or sometimes standalone. This requires old JVMs, a few of
>>> which
>>> are as old as JDK-1.3, and none use a version of java that is still
>>> supported by Oracle (>=1.7).  Deployment is done through RPMs and in
>>> some
>>> cases Solaris packaging.
>>>
>>> As you might imagine, change, in such an organization is difficult.  The
>>> main impediment to change is the accreted base of thousands of makefiles
>>> that have been created over the years.
>>>
>>> But a few intrepid (or maybe foolhardy) souls are thinking of trying.
>>> We'd
>>> like to use maven to handle the java portion of this process.  Its
>>> dependency management features would be worth the effort if we could get
>>> them.  Since replacing the whole system is not in scope, the idea is
>>> to use
>>> maven to handle the java compilation, archiving into jars, wars, ears,
>>> etc., while leaving the packaging, deployment, source control systems as
>>> they are.  Alcatel-Lucent nmake would invoke maven as it now invokes
>>> javac,
>>> jar, etc.  If we can get this far, future upgrading of other portions of
>>> the system may come into play, but not in step 1. Such a transition will
>>> happen gradually or not at all.
>>>
>>> The problem is this.  Alcatel-Lucent nmake (and other versions of make
>>> such as GNU make) includes the concept of the VPATH, an environment
>>> variable containing a path (similar to PATH, etc.) along which to search
>>> for dependent source.  If a necessary file is not found along the first
>>> node of the path, the second is searched for it, then the third, etc.
>>> Only
>>> if the full VPATH is exhausted is the dependency not satisfied and the
>>> build fails.  Importantly, if the dependency IS satisfied, then nodes
>>> further down the path are not looked at for that dependency.
>>>
>>> There is a little tutorial here, explaining how this works:
>>> http://nmake.alcatel-lucent.com/tutorial/s10.html
>>>
>>> Needless to say, this is not the way Maven works, especially the
>>> compiler
>>> plugin, certainly not under default settings.  There is the sourcepath
>>> setting which invokes the -sourcepath switch on javac, which might be
>>> part
>>> of a solution.  There would then be a need for something that could
>>> translate the $VPATH envvar to a sourcepath which would need to dig down
>>> through several layers of a directory tree (at least they would be
>>> identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a
>>> sourcepath.
>>>
>>> I don't think this will work because if I turn on the verbose debug
>>> output, I see that maven is putting a path to each source file on the
>>> javac
>>> command line, and am guessing maven is not going to do this looking over
>>> each node of the VPATH.
>>>
>>> Another option would be to pull the source from the various vpath
>>> nodes in
>>> reverse order and then use maven in a more normal way.  But I imagine
>>> that
>>> this would have negative performance consequences.
>>>
>>> Has anyone on this list ever tried anything like this?  Or is this
>>> too big
>>> a hill to even contemplate climbing?
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email]
>>> For additional commands, e-mail: [hidden email]
>>>
>>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

James Klo
In reply to this post by Steve Cohen-2
Is there a reason you cannot just use the exec plugin?  We use that to manage all sorts of esoteric make-like systems the have similar problems as you list.

Jim Klo
Senior Software Engineer
SRI International
t: @nsomnac

On Mar 31, 2015, at 12:24 PM, Steve Cohen <[hidden email]<mailto:[hidden email]>> wrote:

As I indicated in my post, we need a gradual transition, and changing the automated build part of it would be the last step rather than the first step.  We want to integrate maven into an existing nmake-based process rather than change the entire automated build process first.

At some point down the road the goal would be to use something like Hudson/Jenkins and CI, but we'll never get there if we have to redo everything first.

On 03/31/2015 02:12 PM, [hidden email]<mailto:[hidden email]> wrote:
Have you looked into using Jenkins or Hudson to automate those builds?

You can set up custom environment variables, different jvms, etc. etc. It can run any number of utilities via shell scripts, Maven, Ant, Gradle, etc. etc.

That might be a more fruitful place to start.

Cody Fyler
Lending Grid Build Team
G=Lending Grid Builds
(515) – 441 - 0814

-----Original Message-----
From: Steve Cohen [mailto:[hidden email]]
Sent: Tuesday, March 31, 2015 1:26 PM
To: [hidden email]<mailto:[hidden email]>
Subject: Can Maven be used in an nmake environment with VPATH?

I work for an organization which uses an SCM/Build process based on the
following:

SCM: a ancient legacy horror of a system

Build: Alcatel-Lucent nmake

With this system the organization maintains a large suite of applications.  The system is monstrously inflexible and a pain to work with.  They do manage to have an automated build process with it, but no continuous integration.

A large proportion of the actual code built by this system is java.
Deployment is onto various servers using versions of containers such as weblogic, or sometimes standalone. This requires old JVMs, a few of which are as old as JDK-1.3, and none use a version of java that is still supported by Oracle (>=1.7).  Deployment is done through RPMs and in some cases Solaris packaging.

As you might imagine, change, in such an organization is difficult.  The main impediment to change is the accreted base of thousands of makefiles that have been created over the years.

But a few intrepid (or maybe foolhardy) souls are thinking of trying.
We'd like to use maven to handle the java portion of this process.  Its dependency management features would be worth the effort if we could get them.  Since replacing the whole system is not in scope, the idea is to use maven to handle the java compilation, archiving into jars, wars, ears, etc., while leaving the packaging, deployment, source control systems as they are.  Alcatel-Lucent nmake would invoke maven as it now invokes javac, jar, etc.  If we can get this far, future upgrading of other portions of the system may come into play, but not in step 1.
Such a transition will happen gradually or not at all.

The problem is this.  Alcatel-Lucent nmake (and other versions of make such as GNU make) includes the concept of the VPATH, an environment variable containing a path (similar to PATH, etc.) along which to search for dependent source.  If a necessary file is not found along the first node of the path, the second is searched for it, then the third, etc.
Only if the full VPATH is exhausted is the dependency not satisfied and the build fails.  Importantly, if the dependency IS satisfied, then nodes further down the path are not looked at for that dependency.

There is a little tutorial here, explaining how this works:
http://nmake.alcatel-lucent.com/tutorial/s10.html

Needless to say, this is not the way Maven works, especially the compiler plugin, certainly not under default settings.  There is the sourcepath setting which invokes the -sourcepath switch on javac, which might be part of a solution.  There would then be a need for something that could translate the $VPATH envvar to a sourcepath which would need to dig down through several layers of a directory tree (at least they would be identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a sourcepath.

I don't think this will work because if I turn on the verbose debug output, I see that maven is putting a path to each source file on the javac command line, and am guessing maven is not going to do this looking over each node of the VPATH.

Another option would be to pull the source from the various vpath nodes in reverse order and then use maven in a more normal way.  But I imagine that this would have negative performance consequences.

Has anyone on this list ever tried anything like this?  Or is this too big a hill to even contemplate climbing?

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]<mailto:[hidden email]>
For additional commands, e-mail: [hidden email]<mailto:[hidden email]>

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
That sounds like it might be a possibility.  But after looking at it, my
initial take (which is quite possibly wrong!!) is that it might be
easier to extend the compiler plugin, and the jar plugin, to use the
vpath.  Basically, you're just writing a new SourceInclusionScanner.
This seems like it would have the benefit of staying within known Maven
channels.

On 04/01/2015 10:23 AM, Jim Klo wrote:
> Is there a reason you cannot just use the exec plugin?  We use that to manage all sorts of esoteric make-like systems the have similar problems as you list.
>
> Jim Klo
> Senior Software Engineer
> SRI International
> t: @nsomnac
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

James Klo
Don’t let me discourage you on that choice if you choose it - however it sounds like this might be a stopgap in a transition to a more modern solution?

From my POV, which is a defensive approach towards configuration management - if this is just a transitionary step to removing nmake, I would not bother investing that time, unless:
1) you are going to make a conscious decision on embracing and maintaining your modified fork for a long time because this is going to be a critical part of your solution.
        or
2) you were going to contribute that enhancement back to the plugin projects and get it adopted  - which I don’t know what the process is for that with the maven project (I’ve done it with other apache projects - and it’s not exactly an easy process)

Otherwise it would just add one more piece of forked code to your bucket list maintain.  We all know none of us has time, or desire usually, to manage yet another forked project. :)

I’m sure others have their own opinion on this, but as a "maven user” and not a "maven developer” - I personally don’t like enhancing other tools unless there’s an easy way for me to contribute that fix back and get it into the main build.

Just my 2 cents advice.

- JK


> On Apr 1, 2015, at 9:48 AM, Steve Cohen <[hidden email]> wrote:
>
> That sounds like it might be a possibility.  But after looking at it, my initial take (which is quite possibly wrong!!) is that it might be easier to extend the compiler plugin, and the jar plugin, to use the vpath.  Basically, you're just writing a new SourceInclusionScanner. This seems like it would have the benefit of staying within known Maven channels.
>
> On 04/01/2015 10:23 AM, Jim Klo wrote:
>> Is there a reason you cannot just use the exec plugin?  We use that to manage all sorts of esoteric make-like systems the have similar problems as you list.
>>
>> Jim Klo
>> Senior Software Engineer
>> SRI International
>> t: @nsomnac



smime.p7s (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
JK-

After half a day messing around, I am now convinced that you are right.
  Writing a new plugin is harder than it looked at first, even though I
was able to grok what the plugin was doing fairly well. But there's a
lot of stuff going on under the covers that isn't well documented.

for just one example ${project.compileSourceRoots}

Plus - even if that was all okay, how would you turn
maven-compiler-plugin off?  I notice it appears in my effective POM in
Eclipse without even specifying to use it in the POM.

So a hack on a lesser scale it must be!  Preprocessing it into a form
maven is comfortable with would work better.  In fact, the current nmake
system uses its own form of preprocessing.

Steve

On 04/01/2015 12:14 PM, Jim Klo wrote:

> Don’t let me discourage you on that choice if you choose it - however it sounds like this might be a stopgap in a transition to a more modern solution?
>
>  From my POV, which is a defensive approach towards configuration management - if this is just a transitionary step to removing nmake, I would not bother investing that time, unless:
> 1) you are going to make a conscious decision on embracing and maintaining your modified fork for a long time because this is going to be a critical part of your solution.
> or
> 2) you were going to contribute that enhancement back to the plugin projects and get it adopted  - which I don’t know what the process is for that with the maven project (I’ve done it with other apache projects - and it’s not exactly an easy process)
>
> Otherwise it would just add one more piece of forked code to your bucket list maintain.  We all know none of us has time, or desire usually, to manage yet another forked project. :)
>
> I’m sure others have their own opinion on this, but as a "maven user” and not a "maven developer” - I personally don’t like enhancing other tools unless there’s an easy way for me to contribute that fix back and get it into the main build.
>
> Just my 2 cents advice.
>
> - JK
>
>
>> On Apr 1, 2015, at 9:48 AM, Steve Cohen <[hidden email]> wrote:
>>
>> That sounds like it might be a possibility.  But after looking at it, my initial take (which is quite possibly wrong!!) is that it might be easier to extend the compiler plugin, and the jar plugin, to use the vpath.  Basically, you're just writing a new SourceInclusionScanner. This seems like it would have the benefit of staying within known Maven channels.
>>
>> On 04/01/2015 10:23 AM, Jim Klo wrote:
>>> Is there a reason you cannot just use the exec plugin?  We use that to manage all sorts of esoteric make-like systems the have similar problems as you list.
>>>
>>> Jim Klo
>>> Senior Software Engineer
>>> SRI International
>>> t: @nsomnac
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Wayne Fay
This blog post may be helpful to you:
http://blog.sonatype.com/2009/08/create-a-customized-build-process-in-maven/

If I was in your shoes, I would at least consider this approach.

Wayne

On Wed, Apr 1, 2015 at 4:50 PM, Steve Cohen <[hidden email]> wrote:

> JK-
>
> After half a day messing around, I am now convinced that you are right.
> Writing a new plugin is harder than it looked at first, even though I was
> able to grok what the plugin was doing fairly well. But there's a lot of
> stuff going on under the covers that isn't well documented.
>
> for just one example ${project.compileSourceRoots}
>
> Plus - even if that was all okay, how would you turn maven-compiler-plugin
> off?  I notice it appears in my effective POM in Eclipse without even
> specifying to use it in the POM.
>
> So a hack on a lesser scale it must be!  Preprocessing it into a form
> maven is comfortable with would work better.  In fact, the current nmake
> system uses its own form of preprocessing.
>
> Steve
>
>
> On 04/01/2015 12:14 PM, Jim Klo wrote:
>
>> Don’t let me discourage you on that choice if you choose it - however it
>> sounds like this might be a stopgap in a transition to a more modern
>> solution?
>>
>>  From my POV, which is a defensive approach towards configuration
>> management - if this is just a transitionary step to removing nmake, I
>> would not bother investing that time, unless:
>> 1) you are going to make a conscious decision on embracing and
>> maintaining your modified fork for a long time because this is going to be
>> a critical part of your solution.
>>         or
>> 2) you were going to contribute that enhancement back to the plugin
>> projects and get it adopted  - which I don’t know what the process is for
>> that with the maven project (I’ve done it with other apache projects - and
>> it’s not exactly an easy process)
>>
>> Otherwise it would just add one more piece of forked code to your bucket
>> list maintain.  We all know none of us has time, or desire usually, to
>> manage yet another forked project. :)
>>
>> I’m sure others have their own opinion on this, but as a "maven user” and
>> not a "maven developer” - I personally don’t like enhancing other tools
>> unless there’s an easy way for me to contribute that fix back and get it
>> into the main build.
>>
>> Just my 2 cents advice.
>>
>> - JK
>>
>>
>>  On Apr 1, 2015, at 9:48 AM, Steve Cohen <[hidden email]> wrote:
>>>
>>> That sounds like it might be a possibility.  But after looking at it, my
>>> initial take (which is quite possibly wrong!!) is that it might be easier
>>> to extend the compiler plugin, and the jar plugin, to use the vpath.
>>> Basically, you're just writing a new SourceInclusionScanner. This seems
>>> like it would have the benefit of staying within known Maven channels.
>>>
>>> On 04/01/2015 10:23 AM, Jim Klo wrote:
>>>
>>>> Is there a reason you cannot just use the exec plugin?  We use that to
>>>> manage all sorts of esoteric make-like systems the have similar problems as
>>>> you list.
>>>>
>>>> Jim Klo
>>>> Senior Software Engineer
>>>> SRI International
>>>> t: @nsomnac
>>>>
>>>
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

stephenconnolly
In reply to this post by scohen
On 31 March 2015 at 19:25, Steve Cohen <[hidden email]> wrote:

> The problem is this.  Alcatel-Lucent nmake (and other versions of make
> such as GNU make) includes the concept of the VPATH, an environment
> variable containing a path (similar to PATH, etc.) along which to search
> for dependent source.  If a necessary file is not found along the first
> node of the path, the second is searched for it, then the third, etc. Only
> if the full VPATH is exhausted is the dependency not satisfied and the
> build fails.  Importantly, if the dependency IS satisfied, then nodes
> further down the path are not looked at for that dependency.
>
> There is a little tutorial here, explaining how this works:
> http://nmake.alcatel-lucent.com/tutorial/s10.html
>
> Needless to say, this is not the way Maven works, especially the compiler
> plugin, certainly not under default settings.  There is the sourcepath
> setting which invokes the -sourcepath switch on javac, which might be part
> of a solution.  There would then be a need for something that could
> translate the $VPATH envvar to a sourcepath which would need to dig down
> through several layers of a directory tree (at least they would be
> identical in each node -e.g. $NODE/$PROJECT/src/main/java) to produce a
> sourcepath.
>
>
So I think you are on a hiding to nothing with regards to VPATH support.

Let's take the use case where your VPATH=/tmp/n1:/tmp/n2

If I have /tmp/n2/pom.xml and /tmp/n2/src/main/java/com/lucent/Foo.java

proper VPATH support would mean that I could do

$ cd /tmp/n1
$ mkdir -p src/main/java/com/lucent
$ cp /tmp/n2/src/main/java/com/lucent/Foo.java src/main/java/com/lucent/

and the the VPATH support would mean that in /tmp/n1 Maven would pick up
the pom.xml as being from /tmp/n2 but to be used in /tmp/n1

It seems to me that VPATH was really a feature added to make working
without a SCM possible. With a good SCM I would never need the VPATH
feature at all.

The best you could do is have VPATH support for non-pom.xml files but users
will still need to propagate the pom.xml files to the "current" VPATH node
Reply | Threaded
Open this post in threaded view
|

RE: Can Maven be used in an nmake environment with VPATH?

scohen
In reply to this post by scohen
Thanks for your reply, Stephen.

"On a hiding to nothing" - that's not an expression I'm familiar with,
but I can guess it isn't a harbinger of good news.  :-(

Actually, though, you're wrong about the VPATH being a substitute for
Version Control (although it may have started that way).  This use of
VPATH is almost entirely directed at SUPPORTING the ancient Version
Control system that the organization uses:

For the developer we might have a 3-element vpath like:

path-to-developer-workspace/path-to-version-controlled-code-revised-from-baseline/path-to-baselined-code

whereas

the automated build would just have the last two elements.

If code has not been changed from the baseline, it will not be in
path-to-version-controlled-code-revised-from-baseline, so vpath is
necessary in this system to pull in baselined code.

Typically the developer will run a command to pull down the latest code
from version control whether baselined or not, so vpath is actually
irrelevant to his use of the build process: nothing will not be in his
node.  But if should delete a file from his node, it will be picked up
from the other nodes.  (It follows from this that deleting files is a
somewhat difficult process! but I digress).

In any case, maintaining the existence of the current version control
and build system is the main reason we need to maintain the VPATH.


Anyway, in regard to your point about pom.xml, you are right, it would
need to exist in the top node.  But that would not be that hard to do.
It seems that I would have to run some sort of preprocess that would
resolve the vpath tree:

1) just pull down the code from all the nodes first, as the developer
does.
or

2) create some god-awful process to create a parallel directory of
symbolic links under one tree that point to each source code file
wherever it may be found.



====================================================================================

Stephen Connolly <[hidden email]> wrote


So I think you are on a hiding to nothing with regards to VPATH support.

Let's take the use case where your VPATH=/tmp/n1:/tmp/n2

If I have /tmp/n2/pom.xml and /tmp/n2/src/main/java/com/lucent/Foo.java

proper VPATH support would mean that I could do

$ cd /tmp/n1
$ mkdir -p src/main/java/com/lucent
$ cp /tmp/n2/src/main/java/com/lucent/Foo.java src/main/java/com/lucent/

and the the VPATH support would mean that in /tmp/n1 Maven would pick up
the pom.xml as being from /tmp/n2 but to be used in /tmp/n1

It seems to me that VPATH was really a feature added to make working
without a SCM possible. With a good SCM I would never need the VPATH
feature at all.

The best you could do is have VPATH support for non-pom.xml files but
users
will still need to propagate the pom.xml files to the "current" VPATH
node

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

Steve Cohen-2
In any case Stephen Connolly has provided the final nail in the coffin
for this idea.  At the end of the day, there is no GOOD way to do this
that would be worth the effort.

Switching over to a modern version control system (without VPATH) and
Maven will need to be done together, and be for new work.  Old
applications will need to be ported.  There's no way around it.

Thanks to everyone for trying.

Steve Cohen

On 04/02/2015 10:35 AM, [hidden email] wrote:

> Thanks for your reply, Stephen.
>
> "On a hiding to nothing" - that's not an expression I'm familiar with,
> but I can guess it isn't a harbinger of good news.  :-(
>
> Actually, though, you're wrong about the VPATH being a substitute for
> Version Control (although it may have started that way).  This use of
> VPATH is almost entirely directed at SUPPORTING the ancient Version
> Control system that the organization uses:
>
> For the developer we might have a 3-element vpath like:
>
> path-to-developer-workspace/path-to-version-controlled-code-revised-from-baseline/path-to-baselined-code
>
> whereas
>
> the automated build would just have the last two elements.
>
> If code has not been changed from the baseline, it will not be in
> path-to-version-controlled-code-revised-from-baseline, so vpath is
> necessary in this system to pull in baselined code.
>
> Typically the developer will run a command to pull down the latest code
> from version control whether baselined or not, so vpath is actually
> irrelevant to his use of the build process: nothing will not be in his
> node.  But if should delete a file from his node, it will be picked up
> from the other nodes.  (It follows from this that deleting files is a
> somewhat difficult process! but I digress).
>
> In any case, maintaining the existence of the current version control
> and build system is the main reason we need to maintain the VPATH.
>
>
> Anyway, in regard to your point about pom.xml, you are right, it would
> need to exist in the top node.  But that would not be that hard to do.
> It seems that I would have to run some sort of preprocess that would
> resolve the vpath tree:
>
> 1) just pull down the code from all the nodes first, as the developer
> does.
> or
>
> 2) create some god-awful process to create a parallel directory of
> symbolic links under one tree that point to each source code file
> wherever it may be found.
>
>
>
> ====================================================================================
>
> Stephen Connolly <[hidden email]> wrote
>
>
> So I think you are on a hiding to nothing with regards to VPATH support.
>
> Let's take the use case where your VPATH=/tmp/n1:/tmp/n2
>
> If I have /tmp/n2/pom.xml and /tmp/n2/src/main/java/com/lucent/Foo.java
>
> proper VPATH support would mean that I could do
>
> $ cd /tmp/n1
> $ mkdir -p src/main/java/com/lucent
> $ cp /tmp/n2/src/main/java/com/lucent/Foo.java src/main/java/com/lucent/
>
> and the the VPATH support would mean that in /tmp/n1 Maven would pick up
> the pom.xml as being from /tmp/n2 but to be used in /tmp/n1
>
> It seems to me that VPATH was really a feature added to make working
> without a SCM possible. With a good SCM I would never need the VPATH
> feature at all.
>
> The best you could do is have VPATH support for non-pom.xml files but
> users
> will still need to propagate the pom.xml files to the "current" VPATH
> node
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Can Maven be used in an nmake environment with VPATH?

james northrup
teraform-mvn at https://code.google.com/p/teraform-mvn/ produces poms from
netbeans projects and/or anything with src/*.java and lib/*.jar as maven
reactor builds + does some hueristics to locate existing libraries as
dependencies, and just subsumes the rest of the jars in lib/ as a private
local maven repo.

the conversion only works against maven 2.1, for the brave, however you can
use later edictions of maven on the produced poms.

reorganizing the java source and relaxing the testing requirements can get
you clear of many large java projects in a  relatively short time.

On Thu, Apr 2, 2015 at 4:15 PM Steve Cohen <[hidden email]> wrote:

> In any case Stephen Connolly has provided the final nail in the coffin
> for this idea.  At the end of the day, there is no GOOD way to do this
> that would be worth the effort.
>
> Switching over to a modern version control system (without VPATH) and
> Maven will need to be done together, and be for new work.  Old
> applications will need to be ported.  There's no way around it.
>
> Thanks to everyone for trying.
>
> Steve Cohen
>
> On 04/02/2015 10:35 AM, [hidden email] wrote:
> > Thanks for your reply, Stephen.
> >
> > "On a hiding to nothing" - that's not an expression I'm familiar with,
> > but I can guess it isn't a harbinger of good news.  :-(
> >
> > Actually, though, you're wrong about the VPATH being a substitute for
> > Version Control (although it may have started that way).  This use of
> > VPATH is almost entirely directed at SUPPORTING the ancient Version
> > Control system that the organization uses:
> >
> > For the developer we might have a 3-element vpath like:
> >
> > path-to-developer-workspace/path-to-version-controlled-
> code-revised-from-baseline/path-to-baselined-code
> >
> > whereas
> >
> > the automated build would just have the last two elements.
> >
> > If code has not been changed from the baseline, it will not be in
> > path-to-version-controlled-code-revised-from-baseline, so vpath is
> > necessary in this system to pull in baselined code.
> >
> > Typically the developer will run a command to pull down the latest code
> > from version control whether baselined or not, so vpath is actually
> > irrelevant to his use of the build process: nothing will not be in his
> > node.  But if should delete a file from his node, it will be picked up
> > from the other nodes.  (It follows from this that deleting files is a
> > somewhat difficult process! but I digress).
> >
> > In any case, maintaining the existence of the current version control
> > and build system is the main reason we need to maintain the VPATH.
> >
> >
> > Anyway, in regard to your point about pom.xml, you are right, it would
> > need to exist in the top node.  But that would not be that hard to do.
> > It seems that I would have to run some sort of preprocess that would
> > resolve the vpath tree:
> >
> > 1) just pull down the code from all the nodes first, as the developer
> > does.
> > or
> >
> > 2) create some god-awful process to create a parallel directory of
> > symbolic links under one tree that point to each source code file
> > wherever it may be found.
> >
> >
> >
> > ============================================================
> ========================
> >
> > Stephen Connolly <[hidden email]> wrote
> >
> >
> > So I think you are on a hiding to nothing with regards to VPATH support.
> >
> > Let's take the use case where your VPATH=/tmp/n1:/tmp/n2
> >
> > If I have /tmp/n2/pom.xml and /tmp/n2/src/main/java/com/lucent/Foo.java
> >
> > proper VPATH support would mean that I could do
> >
> > $ cd /tmp/n1
> > $ mkdir -p src/main/java/com/lucent
> > $ cp /tmp/n2/src/main/java/com/lucent/Foo.java src/main/java/com/lucent/
> >
> > and the the VPATH support would mean that in /tmp/n1 Maven would pick up
> > the pom.xml as being from /tmp/n2 but to be used in /tmp/n1
> >
> > It seems to me that VPATH was really a feature added to make working
> > without a SCM possible. With a good SCM I would never need the VPATH
> > feature at all.
> >
> > The best you could do is have VPATH support for non-pom.xml files but
> > users
> > will still need to propagate the pom.xml files to the "current" VPATH
> > node
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>