Quantcast

NMaven CI

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

NMaven CI

brettporter
Administrator
Hi,

For some short term sanity checking, I added NMaven trunk to vmbuild.  
There is one IT failure I need to fix up due to case sensitivity on  
Linux. I moved the repository to http://vmbuild.apache.org/archiva/repository/dotnet/ 
.

Thoughts?

- Brett

--
Brett Porter
[hidden email]
http://blogs.exist.com/bporter/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NMaven CI

Wendy Smoak
On Fri, Oct 3, 2008 at 2:39 AM, Brett Porter <[hidden email]> wrote:

> For some short term sanity checking, I added NMaven trunk to vmbuild. There
> is one IT failure I need to fix up due to case sensitivity on Linux. I moved
> the repository to http://vmbuild.apache.org/archiva/repository/dotnet/.

Thanks, Brett!  Is it publishing snapshots anywhere?  I could do more
testing if I didn't have to build it myself, which never seems to go
smoothly.  I tried
http://vmbuild.apache.org/archiva/repository/snapshots/ , or doesn't
Continuum have a "Deployment Repository" we could expose?

Thanks,
--
Wendy
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NMaven CI

brettporter
Administrator
On 11/10/2008, at 4:31 AM, Wendy Smoak wrote:

> On Fri, Oct 3, 2008 at 2:39 AM, Brett Porter <[hidden email]> wrote:
>
>> For some short term sanity checking, I added NMaven trunk to  
>> vmbuild. There
>> is one IT failure I need to fix up due to case sensitivity on  
>> Linux. I moved
>> the repository to http://vmbuild.apache.org/archiva/repository/dotnet/ 
>> .
>
> Thanks, Brett!  Is it publishing snapshots anywhere?  I could do more
> testing if I didn't have to build it myself, which never seems to go
> smoothly.  I tried
> http://vmbuild.apache.org/archiva/repository/snapshots/ , or doesn't
> Continuum have a "Deployment Repository" we could expose?

Could do, it's not right now. I've taken a note.

If you do have problems on trunk, let us know, since it is pretty  
cooperative for me (as long as you have everything including nunit in  
the path - still need to fix up the toolchains properly).

BTW, I checked with infra yesterday and they're going to set up a  
windows VM that we can use for testing purposes as well.

Cheers,
Brett

--
Brett Porter
[hidden email]
http://blogs.exist.com/bporter/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NMaven CI

Wendy Smoak
On Fri, Oct 10, 2008 at 7:39 PM, Brett Porter <[hidden email]> wrote:

> Could do, it's not right now. I've taken a note.
>
> If you do have problems on trunk, let us know, since it is pretty
> cooperative for me (as long as you have everything including nunit in the
> path - still need to fix up the toolchains properly).

Yes, just listening to you and Carlos talk it seems a bit more complex
than svn co and mvn install.  But it's not always NMaven's fault,
there are also virtual machines and network access including corporate
firewalls and proxies to deal with.  If I don't do it regularly, by
the time I get it building I've forgotten why I wanted it.

I'm also thinking of the way people get involved.  For me it was using
nightly builds for a long time before I learned to build things from
source... so that's another point in favor of publishing snapshots. :)

--
Wendy
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: NMaven CI

brettporter
Administrator

On 13/10/2008, at 2:15 PM, Wendy Smoak wrote:

> On Fri, Oct 10, 2008 at 7:39 PM, Brett Porter <[hidden email]>  
> wrote:
>
>> Could do, it's not right now. I've taken a note.
>>
>> If you do have problems on trunk, let us know, since it is pretty
>> cooperative for me (as long as you have everything including nunit  
>> in the
>> path - still need to fix up the toolchains properly).
>
> Yes, just listening to you and Carlos talk it seems a bit more complex
> than svn co and mvn install.  But it's not always NMaven's fault,
> there are also virtual machines and network access including corporate
> firewalls and proxies to deal with.  If I don't do it regularly, by
> the time I get it building I've forgotten why I wanted it.

Applying Carlos' patch and the subsequent changes I made should have  
stabilised this. The toolchains are still a bit useless, but it works  
just fine with everything in the path. I have my head around the  
toolchain problem now (I think a fix in Maven itself will help), and  
hopefully will get some cycles in the next couple of weeks to put that  
to bed too.

> I'm also thinking of the way people get involved.  For me it was using
> nightly builds for a long time before I learned to build things from
> source... so that's another point in favor of publishing snapshots. :)

No objections there. I was also thinking of queueing up another  
release now that NUnit has been added back would be a good idea -  
perhaps just after straightening out the toolchain issue.

Cheers,
Brett

--
Brett Porter
[hidden email]
http://blogs.exist.com/bporter/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

0.16+/trunk build and Visual Studio references

James Carpenter-2
Until better Visual Studio support is available along the trunk, what is the best way to create/maintain Visual Studio solution and project files?  In particular, how should I deal with project references for dependencies outside of the solution?  Should I configure the dependency:copy-dependencies goal to drop all of the dependencies for each child module in a given location within each child module?  What interim solutions are others here using?

I am assuming I will often reference dependencies which have been assigned a final name without a version number.  I'm not sure this will be relevant to the approach used, but it may.

Background Assumptions/Understanding:
As far as I know the nmaven trunk does not yet contain a maven plugin to  spin up Visual Studio project and solution files.  Furthermore there is no Visual Studio plugin available on the trunk.  I realize this is simply because they are yet to be ported from the 0.14 branch.

In a typical java project the IDE files reference jars in the local maven cache.  Since the dependency references in Java IDE files are typically spun by maven (eclipse:eclipse, idea:idea, etc.) or managed by maven support within the IDE this is quite reasonable.  If one had to maintain all of these references manually doing so would quickly become unreasonable.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

pdb files

James Carpenter-2
How do I configure a module being built by the nmaven trunk version of nmaven to produce pdb files?

On a related note, will they be considered as attached artifacts or will I need to use build-helper:attach-artifact to do that?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pdb files

James Carpenter-2
While running a simple nmaven build in the debugger I came to realize  org.apache.maven.dotnet.extensions.compiler.CShaprClassCompiler#getCommands() doesn't do anything with the /pdb or /debug options of csc.  Furthermore I didn't see any ability to pass in additional compiler arguments.

I have therefore concluded that the 0.16-SNAPSHOT version of nmaven simply doesn't currently support creation of pdb files.

Unfortunately, I don't currently have the time to dedicate to solving this and a what appears to be a multitude of other nmaven issues.  From a review of the JIRA issues it seems even the 0.14-SNAPSHOT code is extremely buggy.  Sadly, I think I'm going to have to punt and look into an ant/nant/msbuild/ivy solution until nmaven matures a bit more.

--- On Tue, 10/14/08, James Carpenter <[hidden email]> wrote:

> From: James Carpenter <[hidden email]>
> Subject: pdb files
> To: [hidden email]
> Date: Tuesday, October 14, 2008, 11:57 AM
> How do I configure a module being built by the nmaven trunk
> version of nmaven to produce pdb files?
>
> On a related note, will they be considered as attached
> artifacts or will I need to use build-helper:attach-artifact
> to do that?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pdb files

Wendy Smoak
I'm sorry to hear that -- I hope you'll stick around for discussions
even if you decide you can't use it right now.

Can add this feature request to JIRA?  Adding options to an existing
plugin shouldn't be too hard, and that would give you the additional
command line arguments.

Thanks,
--
Wendy

On Tue, Oct 14, 2008 at 4:21 PM, James Carpenter
<[hidden email]> wrote:

> While running a simple nmaven build in the debugger I came to realize  org.apache.maven.dotnet.extensions.compiler.CShaprClassCompiler#getCommands() doesn't do anything with the /pdb or /debug options of csc.  Furthermore I didn't see any ability to pass in additional compiler arguments.
>
> I have therefore concluded that the 0.16-SNAPSHOT version of nmaven simply doesn't currently support creation of pdb files.
>
> Unfortunately, I don't currently have the time to dedicate to solving this and a what appears to be a multitude of other nmaven issues.  From a review of the JIRA issues it seems even the 0.14-SNAPSHOT code is extremely buggy.  Sadly, I think I'm going to have to punt and look into an ant/nant/msbuild/ivy solution until nmaven matures a bit more.
>
> --- On Tue, 10/14/08, James Carpenter <[hidden email]> wrote:
>
>> From: James Carpenter <[hidden email]>
>> Subject: pdb files
>> To: [hidden email]
>> Date: Tuesday, October 14, 2008, 11:57 AM
>> How do I configure a module being built by the nmaven trunk
>> version of nmaven to produce pdb files?
>>
>> On a related note, will they be considered as attached
>> artifacts or will I need to use build-helper:attach-artifact
>> to do that?
>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pdb files

James Carpenter-2
I'm working to get the time, and or Tom Sawyer the formal architecture team to do the work for me.

--- On Wed, 10/15/08, Wendy Smoak <[hidden email]> wrote:

> From: Wendy Smoak <[hidden email]>
> Subject: Re: pdb files
> To: [hidden email]
> Date: Wednesday, October 15, 2008, 4:21 PM
> I'm sorry to hear that -- I hope you'll stick around
> for discussions
> even if you decide you can't use it right now.
>
> Can add this feature request to JIRA?  Adding options to an
> existing
> plugin shouldn't be too hard, and that would give you
> the additional
> command line arguments.
>
> Thanks,
> --
> Wendy
>
> On Tue, Oct 14, 2008 at 4:21 PM, James Carpenter
> <[hidden email]> wrote:
> > While running a simple nmaven build in the debugger I
> came to realize
> org.apache.maven.dotnet.extensions.compiler.CShaprClassCompiler#getCommands()
> doesn't do anything with the /pdb or /debug options of
> csc.  Furthermore I didn't see any ability to pass in
> additional compiler arguments.
> >
> > I have therefore concluded that the 0.16-SNAPSHOT
> version of nmaven simply doesn't currently support
> creation of pdb files.
> >
> > Unfortunately, I don't currently have the time to
> dedicate to solving this and a what appears to be a
> multitude of other nmaven issues.  From a review of the JIRA
> issues it seems even the 0.14-SNAPSHOT code is extremely
> buggy.  Sadly, I think I'm going to have to punt and
> look into an ant/nant/msbuild/ivy solution until nmaven
> matures a bit more.
> >
> > --- On Tue, 10/14/08, James Carpenter
> <[hidden email]> wrote:
> >
> >> From: James Carpenter
> <[hidden email]>
> >> Subject: pdb files
> >> To: [hidden email]
> >> Date: Tuesday, October 14, 2008, 11:57 AM
> >> How do I configure a module being built by the
> nmaven trunk
> >> version of nmaven to produce pdb files?
> >>
> >> On a related note, will they be considered as
> attached
> >> artifacts or will I need to use
> build-helper:attach-artifact
> >> to do that?
> >
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pdb files

James Carpenter-2
In reply to this post by Wendy Smoak
I think what really needs to happen is proper support for multiple  
compilation settings (Think release and debug builds.).  I believe  
there is already a JIRA issue for this.  I haven't really thought this  
out, but I think the compiler plugin should be configurable to create  
multiple compilation artifacts with different option values and an  
associated classifier for each.   Secondary artifacts like PDB files  
would of course just be attached artifacts with an appropriate  
classifier.

It is worth nothing, that PDB files can apparently be configured to  
have more or less information.  So called "private" PDB files have a  
lot more details than "public" PDB files.  Public PDB files contain a  
subset of the information in a private PDB.  The information in a  
private PDB is apparently sufficiently detailed to make reverse  
engineering/decompiling code much easier.  When a company releases a  
closed source library they will often release public PDB files along  
with the library, but will keep private PDB files for internal use.  
To further complicate things, I believe I read one can configure  
private PDB files to have less information than a typical PDB file but  
more than a public PDB file.  Each of these would likely need to  
correlate with their own classifier.

None of the above is really all that tricky.  The really tricky part  
will be how to deal with dependencies when compiling, running unit  
tests, running the assembly plugin, etc.  It seems dependencies should  
have an implicit classifier based on context.  When a release assembly/
dll is being built, maven should probably use release versions of the  
dependencies (implicit release classifier).  When a debug assembly is  
being built, maven should probably use debug versions of the project's  
dependencies.  This is to say, the implicit classifier of the  
dependencies should correspond to the classifier associated with the  
compiler plugin configuration being built.  By using the appropriate  
command line options it should be possible to concurrently build as  
many of these different assembly types as desired.  (Would this just  
be different compiler plugin execution configurations?)

I think nmaven is already using its own dependency resolver, so  
everything should be quite tractable, but it may be a bit tricky to  
sort out.  If I was to attack it I would probably just have to evolve  
the code until it smelled right.  I don't think I would be able to see  
all the way to the end solution when I started.

When building Java code maven doesn't really have to deal with any of  
this sticky mess.  It might be instructive to see how other non-Java (C
++?) related plugins are handling this.

On Oct 15, 2008, at 4:21 PM, Wendy Smoak wrote:

> I'm sorry to hear that -- I hope you'll stick around for discussions
> even if you decide you can't use it right now.
>
> Can add this feature request to JIRA?  Adding options to an existing
> plugin shouldn't be too hard, and that would give you the additional
> command line arguments.
>
> Thanks,
> --
> Wendy
>
> On Tue, Oct 14, 2008 at 4:21 PM, James Carpenter
> <[hidden email]> wrote:
>> While running a simple nmaven build in the debugger I came to  
>> realize  
>> org
>> .apache
>> .maven.dotnet.extensions.compiler.CShaprClassCompiler#getCommands()  
>> doesn't do anything with the /pdb or /debug options of csc.  
>> Furthermore I didn't see any ability to pass in additional compiler  
>> arguments.
>>
>> I have therefore concluded that the 0.16-SNAPSHOT version of nmaven  
>> simply doesn't currently support creation of pdb files.
>>
>> Unfortunately, I don't currently have the time to dedicate to  
>> solving this and a what appears to be a multitude of other nmaven  
>> issues.  From a review of the JIRA issues it seems even the 0.14-
>> SNAPSHOT code is extremely buggy.  Sadly, I think I'm going to have  
>> to punt and look into an ant/nant/msbuild/ivy solution until nmaven  
>> matures a bit more.
>>
>> --- On Tue, 10/14/08, James Carpenter <[hidden email]>  
>> wrote:
>>
>>> From: James Carpenter <[hidden email]>
>>> Subject: pdb files
>>> To: [hidden email]
>>> Date: Tuesday, October 14, 2008, 11:57 AM
>>> How do I configure a module being built by the nmaven trunk
>>> version of nmaven to produce pdb files?
>>>
>>> On a related note, will they be considered as attached
>>> artifacts or will I need to use build-helper:attach-artifact
>>> to do that?
>>

Sincerely,
James Carpenter
cell: 832-677-7247
email: [hidden email]



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pdb files

brettporter
Administrator

On 19/10/2008, at 2:38 PM, James Carpenter wrote:

> I think what really needs to happen is proper support for multiple  
> compilation settings (Think release and debug builds.).  I believe  
> there is already a JIRA issue for this.  I haven't really thought  
> this out, but I think the compiler plugin should be configurable to  
> create multiple compilation artifacts with different option values  
> and an associated classifier for each.   Secondary artifacts like  
> PDB files would of course just be attached artifacts with an  
> appropriate classifier.
>
> It is worth nothing, that PDB files can apparently be configured to  
> have more or less information.  So called "private" PDB files have a  
> lot more details than "public" PDB files.  Public PDB files contain  
> a subset of the information in a private PDB.  The information in a  
> private PDB is apparently sufficiently detailed to make reverse  
> engineering/decompiling code much easier.  When a company releases a  
> closed source library they will often release public PDB files along  
> with the library, but will keep private PDB files for internal use.  
> To further complicate things, I believe I read one can configure  
> private PDB files to have less information than a typical PDB file  
> but more than a public PDB file.  Each of these would likely need to  
> correlate with their own classifier.
>
> None of the above is really all that tricky.

Yep, it sounds about right to me.

> The really tricky part will be how to deal with dependencies when  
> compiling, running unit tests, running the assembly plugin, etc.  It  
> seems dependencies should have an implicit classifier based on  
> context.  When a release assembly/dll is being built, maven should  
> probably use release versions of the dependencies (implicit release  
> classifier).  When a debug assembly is being built, maven should  
> probably use debug versions of the project's dependencies.  This is  
> to say, the implicit classifier of the dependencies should  
> correspond to the classifier associated with the compiler plugin  
> configuration being built.  By using the appropriate command line  
> options it should be possible to concurrently build as many of these  
> different assembly types as desired.  (Would this just be different  
> compiler plugin execution configurations?)

Yes, I think it would be dangerous for nmaven to start to inventing  
solutions to things that belong in maven proper - having been down  
that road before and seen the problems it leads to.

Generally, Maven just builds one thing at a time (so there is a normal  
build, then a release profile, and maybe some other profile for other  
scenarios). In this case, it seems you might default to everything  
off, but then some may enable a full debug PDB in the normal builds,  
and in release turn that off and publish a public PDB instead.

If for any reason this requires multiple compilations you could add  
extra executions, but that sounds odd - you should be able to get the  
right set of options for your scenario and then package as  
appropriate, and configure differently in different profiles for  
different scenarios.

I believe this works out better for unit tests as well. Assembly  
plugin generally takes what you tell it so should work in most  
situations.

What you might want is to start flipping classifiers - debug info  
builds add debug info for dependencies, and so on. This is a missing  
capability in Maven right now (the corresponding Java use case is the  
jdk14/15 builds that have jdk14/15 dependenceis). It's a limitation in  
the classifier in that it does actually affect the metadata, though at  
present it is considered not to.

> I think nmaven is already using its own dependency resolver, so  
> everything should be quite tractable, but it may be a bit tricky to  
> sort out.  If I was to attack it I would probably just have to  
> evolve the code until it smelled right.  I don't think I would be  
> able to see all the way to the end solution when I started.

In trunk, nmaven uses the standard dependency resolver.

>
>
> When building Java code maven doesn't really have to deal with any  
> of this sticky mess.  It might be instructive to see how other non-
> Java (C++?) related plugins are handling this.

There's a few different ways, but you're right - some of the problems  
are similar.

Thanks!

Cheers,
Brett

--
Brett Porter
[hidden email]
http://blogs.exist.com/bporter/

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: pdb files

James Carpenter-2
The pdbcopy command (http://msdn.microsoft.com/en-us/library/cc501198.aspx 
) can apparently be used to strip a private PDB down to a public one.  
If both were needed you could always have csc generate a full version,  
and then have a simple pdbcopy plugin produce the public version.  Of  
course, you wouldn't even really need a pdbcopy plugin as the exec  
plugin could do the trick.  It simply might be a tad cleaner to have a  
pdbcopy plugin.

I expect most readers already knew this, I simply failed to mention it  
in the earlier post.

On Oct 19, 2008, at 9:00 AM, Brett Porter wrote:

>
> On 19/10/2008, at 2:38 PM, James Carpenter wrote:
>
>> I think what really needs to happen is proper support for multiple  
>> compilation settings (Think release and debug builds.).  I believe  
>> there is already a JIRA issue for this.  I haven't really thought  
>> this out, but I think the compiler plugin should be configurable to  
>> create multiple compilation artifacts with different option values  
>> and an associated classifier for each.   Secondary artifacts like  
>> PDB files would of course just be attached artifacts with an  
>> appropriate classifier.
>>
>> It is worth nothing, that PDB files can apparently be configured to  
>> have more or less information.  So called "private" PDB files have  
>> a lot more details than "public" PDB files.  Public PDB files  
>> contain a subset of the information in a private PDB.  The  
>> information in a private PDB is apparently sufficiently detailed to  
>> make reverse engineering/decompiling code much easier.  When a  
>> company releases a closed source library they will often release  
>> public PDB files along with the library, but will keep private PDB  
>> files for internal use.  To further complicate things, I believe I  
>> read one can configure private PDB files to have less information  
>> than a typical PDB file but more than a public PDB file.  Each of  
>> these would likely need to correlate with their own classifier.
>>
>> None of the above is really all that tricky.
>
> Yep, it sounds about right to me.
>
>> The really tricky part will be how to deal with dependencies when  
>> compiling, running unit tests, running the assembly plugin, etc.  
>> It seems dependencies should have an implicit classifier based on  
>> context.  When a release assembly/dll is being built, maven should  
>> probably use release versions of the dependencies (implicit release  
>> classifier).  When a debug assembly is being built, maven should  
>> probably use debug versions of the project's dependencies.  This is  
>> to say, the implicit classifier of the dependencies should  
>> correspond to the classifier associated with the compiler plugin  
>> configuration being built.  By using the appropriate command line  
>> options it should be possible to concurrently build as many of  
>> these different assembly types as desired.  (Would this just be  
>> different compiler plugin execution configurations?)
>
> Yes, I think it would be dangerous for nmaven to start to inventing  
> solutions to things that belong in maven proper - having been down  
> that road before and seen the problems it leads to.
>
> Generally, Maven just builds one thing at a time (so there is a  
> normal build, then a release profile, and maybe some other profile  
> for other scenarios). In this case, it seems you might default to  
> everything off, but then some may enable a full debug PDB in the  
> normal builds, and in release turn that off and publish a public PDB  
> instead.
>
> If for any reason this requires multiple compilations you could add  
> extra executions, but that sounds odd - you should be able to get  
> the right set of options for your scenario and then package as  
> appropriate, and configure differently in different profiles for  
> different scenarios.
>
> I believe this works out better for unit tests as well. Assembly  
> plugin generally takes what you tell it so should work in most  
> situations.
>
> What you might want is to start flipping classifiers - debug info  
> builds add debug info for dependencies, and so on. This is a missing  
> capability in Maven right now (the corresponding Java use case is  
> the jdk14/15 builds that have jdk14/15 dependenceis). It's a  
> limitation in the classifier in that it does actually affect the  
> metadata, though at present it is considered not to.
>
>> I think nmaven is already using its own dependency resolver, so  
>> everything should be quite tractable, but it may be a bit tricky to  
>> sort out.  If I was to attack it I would probably just have to  
>> evolve the code until it smelled right.  I don't think I would be  
>> able to see all the way to the end solution when I started.
>
> In trunk, nmaven uses the standard dependency resolver.
>
>>
>>
>> When building Java code maven doesn't really have to deal with any  
>> of this sticky mess.  It might be instructive to see how other non-
>> Java (C++?) related plugins are handling this.
>
> There's a few different ways, but you're right - some of the  
> problems are similar.
>
> Thanks!
>
> Cheers,
> Brett
>
> --
> Brett Porter
> [hidden email]
> http://blogs.exist.com/bporter/
>

Sincerely,
James Carpenter
cell: 832-677-7247
email: [hidden email]



Loading...