Continuous Delivery and Maven

[stug23]

I've been reading about Continuous Delivery <http://continuousdelivery.com/> and trying to understand how to best combine Maven and Continuous Delivery. There is a thread in the Continuous Delivery google group where this discussion has started:

<http://groups.google.com/group/continuousdelivery/browse_thread/thread/c8440681058f2db8>

I would be curious as to whether there are other Maven developers following the Maven Users forum that have been looking Continuous Delivery and trying to grapple with the best way to use Maven in this approach.

[Thiessen, Todd (Todd)]

We use maven and continuous integration and it works very very well. SNAPSHOTS are a godsend when it comes to being flexible and delivering something quickly and tested quickly.

We commit many times a day and it is unphathomable to create a concrete release for each of those commits. It would waste a lot of time and resources. When working with SNAPSHOTS you don't worry about "what released version was that problem found on?"  You just simply say the error occurs on the trunk and fix it there.

The trick here is to have fast feedback. So if you introduce an error today, you should know about that error today or at the latest tomorrow.  So you don't have days or weeks go by before you get that feedback.

My general advice for devs who get into the rut like the one on the google thread is to switch to concrete releases once you start approaching your release date.  So lets say iterations 1 to 10 are on snapshots. Your product owner then feels that you have enough content to ship a release. Create a branch, cut a formal release and have your testing team test that release and iteration 11 with a release.  You may have a couple of fixes to do on that branch and you may need to cut a new release to test what you fixed.  The idea here though is that 99% of all the feature and bugs were tested and ironed out on SNAPSHOT builds. You just do your last iteration on formal releases. You wait as long as you can before before you move to concrete releases so you don't have to manage multiple branches and tags and all the complication that comes with formal releases. The version numbering would go something like this:

1.0-SNAPSHOT
1.0-SNAPSHOT
1.0-SNAPSHOT
1.0-SNAPSHOT
1.0-SNAPSHOT
...
1.0-01
1.0-02
1.0-03

You release to the field with 1.0-03.

> -----Original Message-----
> From: stug23 [mailto:[hidden email]]
> Sent: Monday, November 01, 2010 1:14 PM
> To: [hidden email]
> Subject: Continuous Delivery and Maven
>
>
> I've been reading about Continuous Delivery
> <http://continuousdelivery.com/>
> and trying to understand how to best combine Maven and Continuous
> Delivery.
> There is a thread in the Continuous Delivery google group where this
> discussion has started:
>
> <http://groups.google.com/group/continuousdelivery/browse_thread/thread/c
> 8440681058f2db8>
>
> I would be curious as to whether there are other Maven developers
> following
> the Maven Users forum that have been looking Continuous Delivery and
> trying
> to grapple with the best way to use Maven in this approach.
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-
> tp3245370p3245370.html
> Sent from the Maven - Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[jhumble]

Hey Todd

The whole point of continuous delivery is that every check-in creates a potential release candidate.

When you're doing continuous deployment, you could be releasing multiple times a day, so you don't bother cutting branches or tagging or any of that stuff because of the overhead. I'd rather not get into the justification for this process on this thread - but I wrote a book on it if you're interested: http://www.amazon.com/gp/product/0321601912 and many other people have blogged about it.

You're right that creating a concrete release for each commit could potentially use up a lot of space - but that's fine, you can just delete the older ones. What this *does* mean in turn though is that it is essential to be able to recreate any given build given the version in source control it came from, and this is where Maven falls down. Snapshots just aren't suitable because they aren't reproducible: what the snapshot looks like depends not only on what versions of the dependencies are available at the time the snapshot is created, but also what Maven's configuration and plug-ins happen to be at the time you run it (assuming Maven is configured to auto-update - the default). I can't revert back to a particular revision in version control, run maven, and be sure that the artifact it generates is identical to the one it created when the commit was initially triggered.

Ideally what I'd like is for Maven to explicitly support the continuous delivery model and provide snapshots that are reproducible. Failing that, a guide to configuring Maven so that its binaries are reproducible (for example by switching off auto-update, and having sufficient metadata stored in pom files and Maven's artifacts repository to know what the state of each of the dependencies was at any given time.

[stug23]

I suspect that there is a reasonable way to adjust the builds in Maven such that they robustly support Continuous Delivery. I know that Jason van Zyl has started looking at CD as well.

On the Google forum <http://groups.google.com/group/continuousdelivery/browse_thread/thread/c8440681058f2db8> Chris Hilton commented:

"3. In my ideal world, I would probably either extend the release
plugin or write a new one with functionality to take a snapshot
artifact and make it a release artifact, which would still introduce
slight changes but pretty mild ones. I thought I had read a feature
request or wish list about this, but can't find it now."

The current Maven release plugin, as it is, was not designed to handle the notion of CD. So it does make sense to tailor this aspect of releasing Maven artifacts to better support CD.

Also a quick examination of the Maven versions plugin shows that it has facilities to handle updating versions of projects and their dependencies. So it looks as if it is more a matter of adjustment of Maven build to produce a result that aligns with CD than a matter of Maven not being suitable for CD. I think that the main stumbling block is SNAPSHOTS because they inherently are not releasable -- part of the process of releasing with the current Maven release plugin is to fail when there are SNAPSHOT dependencies.

Another clear requirement for any changes to Maven or its build behavior is that the current behavior must be maintained for the many projects that haven't embraced (or won't embrace) Continuous Delivery. So I imagine a Maven build's behavior being altered through configuration (or a switch of some sort) to comply with the desired behavior inherent in Continuous Delivery with respect to the release of artifacts. The major difference would appear to be with the SNAPSHOT dependencies and the Maven release process. Other aspects of Maven are for the most part already good as they are for CD.

This is an interesting challenge that I am sure is best served by a thorough examination by the Maven and CD experts out there.

[Michael Hüttermann]

Hello fellows,

perhaps it is interesting for you that the book "Agile ALM" [1],
illustrates strategies and tools (especially Maven). I'm also covering
aspects that are discussed here in this thread.

[1] http://huettermann.net/alm


Cheers
Michael

--
-Java Champion-
[hidden email]
http://huettermann.net
http://twitter.com/huettermann

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[stug23]

I wanted to comment on something we have done for our builds with respect to SNAPSHOTS that helps with being able repeat a build for the SNAPSHOT. We record the Subversion revision number for each SNAPSHOT in the MANIFEST.

Also we have a so-called base-pom, which all projects inherit from, that locks down all of the Maven plugin versions so that the build is repeatable at a later time.

[Brian Topping]

On Nov 7, 2010, at 11:14 AM, jhumble wrote:

> Ideally what I'd like is for Maven to explicitly support the continuous
> delivery model and provide snapshots that are reproducible.

Snapshots can be reproducible if developers set dependencies on the timestamped name of the deployed version.  I remember there being reasons that these strategies are discouraged (should be in the list archives), but those reasons may not apply here.

If I understand the paradigm, it's not that developers would want to reject the latest version of any dependencies, just that the snapshot builds should be reproducible.  Since the POM is the source of dependency resolution for any Maven build, it seems like the release plugin in use would have to update the project POMs to the currently resolved name of the actual dependency and check them in before the tag.  

But if the release checks in a POM and this is happening over and over for every checkin (a one line change in a heavily connected project could easily cause several POMs to be updated), suddenly dependent versions need to have their POMs changed, checked in, and new version propagated iteratively.  On developer desktops, all these new POMs would need to be kept up to date constantly to avoid text conflicts.  Maybe a VFS SCM could get around this, but.... ick!  The tooling is suddenly very heavyweight.

Does your book discuss ways to get around these issues?
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[jhumble]

On 7 November 2010 10:02, Brian Topping wrote:

 

Does your book discuss ways to get around these issues? 

No, it's a patterns / practices book so we don't go into a lot of detail on the tools because that information tends to go out of date fast. We discuss Maven a bit at the end of Chapter 13 ("Managing components and dependencies"), pp375-378

The advice we give is this: "have your CI server produce canonical versions of each dependency, using the build label as part of the artifact’s version number, and store these in your organization’s central artifact repoistory. You can then use Maven’s version quantifiers in your pom file to specify a range of acceptable versions to use. If you really need to do some exploratory work on your local machine, you can always edit your pom definition to temporarily enable snapshots"

This isn't really ideal, partly for the reason you specify: this leads to a lot of thrashing with the POM file. So I'm interested to see what could be done to make Maven work better with the CD paradigm, for instance, stug23's idea:

we have a so-called base-pom, which all projects inherit from, that locks down all of the Maven plugin versions so that the build is repeatable at a later time. 

I think you identify the problem exactly right:

If I understand the paradigm, it's not that developers would want to reject the latest version of any dependencies, just that the snapshot builds should be reproducible

In fact, I'd go further and say we want to encourage people taking the latest version of dependencies - otherwise you're not doing continuous integration. But of course we want all of those dependencies in the artifact repo, not built from scratch on the developer machines, because then it means firstly wasted time and secondly that everybody is working with potentially different binaries. 

One possibility to get repeatable builds without filling up an artifacts repository too fast could be to make Maven store the fully qualified pom files in the artifacts repo and an md5 of the binary but not necessarily the actual binary. I know artifacts repos already store some of this information.

That way you could make sure sufficient metadata is publicly available such that they can be reproduced, without using up loads of disk space. You could also happily delete older binaries, safe in the knowledge that people could reproduce them from the metadata in the artifacts repo.

As you can probably tell I'm no Maven maven, but I do want to help in whatever way I can to make it work well with a continuous delivery process.

Jez. 

--
Jez Humble

Co-author, Continuous Delivery

http://continuousdelivery.com/

http://jezhumble.net/

[Brian Topping]

On Nov 7, 2010, at 1:29 PM, jhumble wrote:

> One possibility to get repeatable builds without filling up an artifacts
> repository too fast could be to make Maven store the fully qualified pom
> files in the artifacts repo and an md5 of the binary but not necessarily the
> actual binary. I know artifacts repos already store some of this
> information.
>
> That way you could make sure sufficient metadata is publicly available such
> that they can be reproduced, without using up loads of disk space. You could
> also happily delete older binaries, safe in the knowledge that people could
> reproduce them from the metadata in the artifacts repo.

One of the things I like about snapshots is it just simply means "latest".  Though the thing about timestamped snapshots is that they aren't guaranteed to exist (the repository is not typically assumed to be reliable), and they aren't 100% reproducible (the timestamp offset includes the time it took to build the artifact and all the artifacts before it, meaning there's no way to know exactly what point in time the build came from).  Even if one could find the correct timestamp to check out from to get the same binary, whatever subsystem creates the timestamp on upload (wagon?) probably doesn't like being told what to call the snapshot.  

It follows the only way to get a reproducible build is either to tag the original sources or to know the SCM revision id.  The revision id is a natural tag that is automatically generated, and does not clutter the named tag space with thousands of tags that have no organizational meaning.  On my CI builds, the first thing that happens is grabbing the revision ID from SVN, and that's put in a properties file that can be used when the UI is generated.  Where the version number helps users identify the general features to expect of the current software, the revision ID is great for filing issues so devs don't have to guess at what sources have the issue.  

When the sources all come from the same SCM repository tree, the rev ID makes it a cinch to reproduce the build.  Of course, a better solution can span multiple trees and is reproducible.

It just seems like the rev ID is really useful here for identifying reproducible builds without creating releases every time, does it fit with your ideas?  If so, a hypothetical repository manager plugin could be maintaining information about snapshot dependencies based on SCM rev ID, thus allowing for reproducibility without modifying Maven or existing snapshot mechanics.  Such a plugin might be able to generate a POM that has the extra rev ID metadata that the repo manager would recognize, allowing for existing SNAPSHOT-style identifiers to keep working for developer desktops (avoiding SCM thrash), but also providing reproducibility through synthetic POMs.



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Yanko, Curtis]

Very interesting discussion. With all due respect to Mr. Humble, and I
am a big fan of CD, I am going to venture to say that you don't
understand Maven very well. As a thought experiment, you are correct in
saying that a build based on snapshots is not reproducible. As a more
practical matter however, I feel it is.

Dependencies come in two flavors, our and theirs (internal and 3rd
party). If, all of *our* dependencies are SNAPSHOT (we're doing the
developing) and all of *theirs* are 'versioned' then the build is in
fact reproducible assuming you build everything from a particular repo
version even with the default auto-update setting (in fact, it's
required).
________________________________

Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
Making IT Happen, one build at a time, 600 times a day

-----Original Message-----
From: jhumble [mailto:[hidden email]]
Sent: Sunday, November 07, 2010 11:15 AM
To: [hidden email]
Subject: RE: Continuous Delivery and Maven


Hey Todd

The whole point of continuous delivery is that every check-in creates a
potential release candidate.

When you're doing continuous deployment, you could be releasing multiple
times a day, so you don't bother cutting branches or tagging or any of
that stuff because of the overhead. I'd rather not get into the
justification for this process on this thread - but I wrote a book on it
if you're interested:
http://www.amazon.com/gp/product/0321601912 and many other people have
blogged about it.

You're right that creating a concrete release for each commit could
potentially use up a lot of space - but that's fine, you can just delete
the older ones. What this *does* mean in turn though is that it is
essential to be able to recreate any given build given the version in
source control it came from, and this is where Maven falls down.
Snapshots just aren't suitable because they aren't reproducible: what
the snapshot looks like depends not only on what versions of the
dependencies are available at the time the snapshot is created, but also
what Maven's configuration and plug-ins happen to be at the time you run
it (assuming Maven is configured to auto-update - the default). I can't
revert back to a particular revision in version control, run maven, and
be sure that the artifact it generates is identical to the one it
created when the commit was initially triggered.

Ideally what I'd like is for Maven to explicitly support the continuous
delivery model and provide snapshots that are reproducible. Failing
that, a guide to configuring Maven so that its binaries are reproducible
(for example by switching off auto-update, and having sufficient
metadata stored in pom files and Maven's artifacts repository to know
what the state of each of the dependencies was at any given time.
--
View this message in context:
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370
p3254090.html
Sent from the Maven - Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[jhumble]

Hi Curtis

I'm the first to admit I'm no Maven expert.

So please let me just confirm. Let's assume I am working on project A which depends on projects B and C. For the sake of argument, let's say that the source code for A, B and C have separate roots in SVN, and are usually checked out independently. The CI system builds A at version 50 using snapshot dependencies on B and C, which are fetched from a central snapshot repository. 

Later, there are multiple updates to projects B and C which result in new versions of them becoming available.

Say I now check out the source code to project A to version 50, because I want to try and debug some issue that cropped up in a performance testing environment, and I run a maven build. Will that use the latest versions of the snapshots from the repo, or the versions that were originally fetched when it was run on the CI system?

Can I even find out exactly which versions from svn the snapshots of B and C came from that were used by the CI system to generate the original build of A?

Thanks,

Jez.

On 7 November 2010 20:10, Yanko, Curtis [via Maven] <[hidden email]> wrote:

Very interesting discussion. With all due respect to Mr. Humble, and I
am a big fan of CD, I am going to venture to say that you don't
understand Maven very well. As a thought experiment, you are correct in
saying that a build based on snapshots is not reproducible. As a more
practical matter however, I feel it is.

Dependencies come in two flavors, our and theirs (internal and 3rd
party). If, all of *our* dependencies are SNAPSHOT (we're doing the
developing) and all of *theirs* are 'versioned' then the build is in
fact reproducible assuming you build everything from a particular repo
version even with the default auto-update setting (in fact, it's
required).
________________________________

Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
Making IT Happen, one build at a time, 600 times a day

-----Original Message-----
From: jhumble [mailto:[hidden email]]
Sent: Sunday, November 07, 2010 11:15 AM
To: [hidden email]
Subject: RE: Continuous Delivery and Maven


Hey Todd

The whole point of continuous delivery is that every check-in creates a
potential release candidate.

When you're doing continuous deployment, you could be releasing multiple
times a day, so you don't bother cutting branches or tagging or any of
that stuff because of the overhead. I'd rather not get into the
justification for this process on this thread - but I wrote a book on it
if you're interested:
http://www.amazon.com/gp/product/0321601912 and many other people have
blogged about it.

You're right that creating a concrete release for each commit could
potentially use up a lot of space - but that's fine, you can just delete
the older ones. What this *does* mean in turn though is that it is
essential to be able to recreate any given build given the version in
source control it came from, and this is where Maven falls down.
Snapshots just aren't suitable because they aren't reproducible: what
the snapshot looks like depends not only on what versions of the
dependencies are available at the time the snapshot is created, but also
what Maven's configuration and plug-ins happen to be at the time you run
it (assuming Maven is configured to auto-update - the default). I can't
revert back to a particular revision in version control, run maven, and
be sure that the artifact it generates is identical to the one it
created when the commit was initially triggered.

Ideally what I'd like is for Maven to explicitly support the continuous

delivery model and provide snapshots that are reproducible. Failing
that, a guide to configuring Maven so that its binaries are reproducible
(for example by switching off auto-update, and having sufficient
metadata stored in pom files and Maven's artifacts repository to know
what the state of each of the dependencies was at any given time.
--
View this message in context:

http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370

p3254090.html
Sent from the Maven - Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

---

View message @ http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370p3254520.html

To unsubscribe from Continuous Delivery and Maven, click here.

--
Jez Humble

Co-author, Continuous Delivery

http://continuousdelivery.com/

http://jezhumble.net/

[jhumble]

Hi Brian

It just seems like the rev ID is really useful here for identifying reproducible builds without creating releases every time, does it fit with your ideas?  If so, a hypothetical repository manager plugin could be maintaining information about snapshot dependencies based on SCM rev ID, thus allowing for reproducibility without modifying Maven or existing snapshot mechanics.  Such a plugin might be able to generate a POM that has the extra rev ID metadata that the repo manager would recognize, allowing for existing SNAPSHOT-style identifiers to keep working for developer desktops (avoiding SCM thrash), but also providing reproducibility through synthetic POMs. 

I think this is a great idea. If the pom for each snapshot contains enough metadata about each of its upstream snapshots to be able to reproduce an identical binary, we're good. Since I believe Maven already stores the md5 for each snapshot, this is verifiable.

Perhaps including in the POM for a snapshot the SCM URIs that were used to create it and the version id (for SCMs that support atomic commits - I can't see a happy way to do this for CVS, for example, without using tags) and the same information nested for any of their upstream snapshots?

And then for extra points a command that could look at this pom, regenerate the whole thing from scratch, and verify it against the md5 for the original binary.

Jez.  

--
Jez Humble

Co-author, Continuous Delivery

http://continuousdelivery.com/

http://jezhumble.net/

[Thiessen, Todd (Todd)]

Interestingly enough, I kind of feel that we have a different definition of continuous integration.

You generally don't want to, and hopefully very rarely need to, go back to a specific revision.  The whole idea is that you are always reproducing problems on the latest version of your project's trunk and all latest trunks of all its dependencies.  Reproducing it on a specific revision has little value. If you are not reproducing, fixes and adding new features on the latest trunk, then you are not really doing continuous integration.

Why? Older revisions are simply just that; old. If you do go back and reproduce it on a specific revision, you still have to fix and reproduce on the latest trunk. 99% of the time, you can go right to the latter. For this, Maven is excellent and is what I would hope you are your book would recommend.

For those few cases where you do need to reproduce a specific revision and all of its dependencies, you are right, it can get very messy with snapshots everywhere.  I think you are making a mountain out of mole hill here.

> -----Original Message-----
> From: jhumble [mailto:[hidden email]]
> Sent: Sunday, November 07, 2010 11:41 PM
> To: [hidden email]
> Subject: Re: Continuous Delivery and Maven
>
>
> Hi Curtis
>
> I'm the first to admit I'm no Maven expert.
>
> So please let me just confirm. Let's assume I am working on project A
> which
> depends on projects B and C. For the sake of argument, let's say that the
> source code for A, B and C have separate roots in SVN, and are usually
> checked out independently. The CI system builds A at version 50 using
> snapshot dependencies on B and C, which are fetched from a central
> snapshot
> repository.
>
> Later, there are multiple updates to projects B and C which result in new
> versions of them becoming available.
>
> Say I now check out the source code to project A to version 50, because I
> want to try and debug some issue that cropped up in a performance testing
> environment, and I run a maven build. Will that use the latest versions
> of
> the snapshots from the repo, or the versions that were originally fetched
> when it was run on the CI system?
>
> Can I even find out exactly which versions from svn the snapshots of B
> and C
> came from that were used by the CI system to generate the original build
> of
> A?
>
> Thanks,
>
> Jez.
>
> On 7 November 2010 20:10, Yanko, Curtis [via Maven] <
> [hidden email]<ml-node%2B3254520-
> [hidden email]>
> > wrote:
>
> > Very interesting discussion. With all due respect to Mr. Humble, and I
> > am a big fan of CD, I am going to venture to say that you don't
> > understand Maven very well. As a thought experiment, you are correct in
> > saying that a build based on snapshots is not reproducible. As a more
> > practical matter however, I feel it is.
> >
> > Dependencies come in two flavors, our and theirs (internal and 3rd
> > party). If, all of *our* dependencies are SNAPSHOT (we're doing the
> > developing) and all of *theirs* are 'versioned' then the build is in
> > fact reproducible assuming you build everything from a particular repo
> > version even with the default auto-update setting (in fact, it's
> > required).
> > ________________________________
> >
> > Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
> > Making IT Happen, one build at a time, 600 times a day
> >
> > -----Original Message-----
> > From: jhumble [mailto:[hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=0>]
> >
> > Sent: Sunday, November 07, 2010 11:15 AM
> > To: [hidden email]
> <http://user/SendEmail.jtp?type=node&node=3254520&i=1>
> > Subject: RE: Continuous Delivery and Maven
> >
> >
> > Hey Todd
> >
> > The whole point of continuous delivery is that every check-in creates a
> > potential release candidate.
> >
> > When you're doing continuous deployment, you could be releasing
> multiple
> > times a day, so you don't bother cutting branches or tagging or any of
> > that stuff because of the overhead. I'd rather not get into the
> > justification for this process on this thread - but I wrote a book on
> it
> > if you're interested:
> > http://www.amazon.com/gp/product/0321601912 and many other people have
> > blogged about it.
> >
> > You're right that creating a concrete release for each commit could
> > potentially use up a lot of space - but that's fine, you can just
> delete
> > the older ones. What this *does* mean in turn though is that it is
> > essential to be able to recreate any given build given the version in
> > source control it came from, and this is where Maven falls down.
> > Snapshots just aren't suitable because they aren't reproducible: what
> > the snapshot looks like depends not only on what versions of the
> > dependencies are available at the time the snapshot is created, but
> also
> > what Maven's configuration and plug-ins happen to be at the time you
> run
> > it (assuming Maven is configured to auto-update - the default). I can't
> > revert back to a particular revision in version control, run maven, and
> > be sure that the artifact it generates is identical to the one it
> > created when the commit was initially triggered.
> >
> > Ideally what I'd like is for Maven to explicitly support the continuous
> > delivery model and provide snapshots that are reproducible. Failing
> > that, a guide to configuring Maven so that its binaries are
> reproducible
> > (for example by switching off auto-update, and having sufficient
> > metadata stored in pom files and Maven's artifacts repository to know
> > what the state of each of the dependencies was at any given time.
> > --
> > View this message in context:
> > http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-
> tp3245370<http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-
> tp3245370?by-user=t>
> > p3254090.html
> > Sent from the Maven - Users mailing list archive at Nabble.com.
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=2>
> > For additional commands, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=3>
> >
> >
> > This e-mail, including attachments, may include confidential and/or
> > proprietary information, and may be used only by the person or entity
> > to which it is addressed. If the reader of this e-mail is not the
> intended
> > recipient or his or her authorized agent, the reader is hereby notified
> > that any dissemination, distribution or copying of this e-mail is
> > prohibited. If you have received this e-mail in error, please notify
> the
> > sender by replying to this message and delete this e-mail immediately.
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=4>
> > For additional commands, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=5>
> >
> >
> >
> > ------------------------------
> >  View message @
> > http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-
> tp3245370p3254520.html
> > To unsubscribe from Continuous Delivery and Maven, click
> here<http://maven.40175.n5.nabble.com/template/TplServlet.jtp?tpl=unsubsc
> ribe_by_code&node=3245370&code=amV6QGplemh1bWJsZS5uZXR8MzI0NTM3MHwtMTg4Mj
> M1NzMyNA==>.
> >
> >
> >
>
>
> --
> Jez Humble
> Co-author, *Continuous Delivery <http://continuousdelivery.com/>*
> http://continuousdelivery.com/
> http://jezhumble.net/
>
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-
> tp3245370p3254534.html
> Sent from the Maven - Users mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Thiessen, Todd (Todd)]

Yes. That is a good way to do. All your transitive dependencies would have to do this also to completely reproduce the entire environment. Doable, but a bit of a pain.

My whole point though is that you should very rarely have to do it.

> -----Original Message-----
> From: stug23 [mailto:[hidden email]]
> Sent: Sunday, November 07, 2010 12:21 PM
> To: [hidden email]
> Subject: RE: Continuous Delivery and Maven
>
>
> I wanted to comment on something we have done for our builds with respect
> to
> SNAPSHOTS that helps with being able repeat a build for the SNAPSHOT. We
> record the Subversion revision number for each SNAPSHOT in the MANIFEST.
>
> Also we have a so-called base-pom, which all projects inherit from, that
> locks down all of the Maven plugin versions so that the build is
> repeatable
> at a later time.
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-
> tp3245370p3254127.html
> Sent from the Maven - Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Thiessen, Todd (Todd)]

I think I see two points here.

1. Reproducing a problem on a particular revision.
2. Delivering a CI build to your final customer.

While somewhat related, I think it is important to distinguish these two things in this dicussion.

The concept of snapshots I think works extremely well.  True, you don't want to be delivering snapshots to a final customer. But this is ok because it really allows you the flexibility to always be working on the trunk. And it allows you to do so without changing your pom everytime you build.  Your developer build is the same as your CI with respect to your pom.

Imagine trying to have a CI build while always having enough information in the pom to point to an exact version of your dependencies? Lets say I get a feature working in my checkout, and do a build locally before committing.  The build would have to point to a fixed version of all my dependencies.  If the dependencies change while I am working, they would have to change "on the fly".

But of course the formal CI build would also have to ensure it points to the latest fixed version of all dependencies. It would have to modify the pom "on the fly" and it could change to point to a more recent version of a dependency since there is time between when I run the build in my checkout and the time I commit.

It just gets very complicated very quickly. I would rather have all revision numbers in all jars.

I am against the idea of having an automated process massaging a snapshot and turning it into a release. You simply run the risk of building something that you didn't anticipate. If you want to build a release, build a formal release. This will include ensuring that all of your transtive dependencies are also a formal release. That way you control exactly what revision of what transitive dependencies will be built as a release.

Now perhaps we could automate the process of building the formal release of the project and all of its transtive dependencies. But massaging existing snapshots into a release I don't think would be wise.

Or maybe there is a completely different kind of architecture here that would allow you to do this. I just don't think that it fits into the current Maven architecture; or any other build management system for that matter.

> -----Original Message-----
> From: stug23 [mailto:[hidden email]]
> Sent: Sunday, November 07, 2010 12:17 PM
> To: [hidden email]
> Subject: RE: Continuous Delivery and Maven
>
>
> I suspect that there is a reasonable way to adjust the builds in Maven
> such
> that they robustly support Continuous Delivery. I know that Jason van Zyl
> has started looking at CD as well.
>
> On the Google forum
> <http://groups.google.com/group/continuousdelivery/browse_thread/thread/c
> 8440681058f2db8>
> Chris Hilton commented:
>
> 3. In my ideal world, I would probably either extend the release
> plugin or write a new one with functionality to take a snapshot
> artifact and make it a release artifact, which would still introduce
> slight changes but pretty mild ones. I thought I had read a feature
> request or wish list about this, but can't find it now.
>
> The current Maven release plugin, as it is, was not designed to handle
> the
> notion of CD. So it does make sense to tailor this aspect of releasing
> Maven
> artifacts to better support CD.
>
> Also a quick examination of the Maven versions plugin shows that it has
> facilities to handle updating versions of projects and their
> dependencies.
> So it looks as if it is more a matter of adjustment of Maven build to
> produce a result that aligns with CD than a matter of Maven not being
> suitable for CD. I think that the main stumbling block is SNAPSHOTS
> because
> they inherently are not releasable -- part of the process of releasing
> with
> the current Maven release plugin is to fail when there are SNAPSHOT
> dependencies.
>
> Another clear requirement for any changes to Maven or its build behavior
> is
> that the current behavior must be maintained for the many projects that
> haven't embraced (or won't embrace) Continuous Delivery. So I imagine a
> Maven build's behavior being altered through configuration (or a switch
> of
> some sort) to comply with the desired behavior inherent in Continuous
> Delivery with respect to the release of artifacts. The major difference
> would appear to be with the SNAPSHOT dependencies and the Maven release
> process. Other aspects of Maven are for the most part already good as
> they
> are for CD.
>
> This is an interesting challenge that I am sure is best served by a
> thorough
> examination by the Maven and CD experts out there.
> --
> View this message in context:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-
> tp3245370p3254121.html
> Sent from the Maven - Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[stephenconnolly]

On 8 November 2010 13:30, Thiessen, Todd (Todd) <[hidden email]> wrote:
> Interestingly enough, I kind of feel that we have a different definition of continuous integration.
>

Interestingly enought, the original poster is talking about Continuous
_D_E_L_I_V_E_R_Y_ as distinct fron Continuous Integrration ;-)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Thiessen, Todd (Todd)]

True. But how does that change my statement? It still appears we may have a different definition of CI.  Because in order to do CD, you would no longer be doing CI.

Unless of course the original poster is suggesting to abandon CI.

> -----Original Message-----
> From: Stephen Connolly [mailto:[hidden email]]
> Sent: Monday, November 08, 2010 9:17 AM
> To: Maven Users List
> Subject: Re: Continuous Delivery and Maven
>
> On 8 November 2010 13:30, Thiessen, Todd (Todd) <[hidden email]>
> wrote:
> > Interestingly enough, I kind of feel that we have a different
> definition of continuous integration.
> >
>
> Interestingly enought, the original poster is talking about Continuous
> _D_E_L_I_V_E_R_Y_ as distinct fron Continuous Integrration ;-)
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Yanko, Curtis]

Jez,

Again, you have carefully crafted a loosing scenario, which are easy to
do, just lay out a bunch of bad practices a your in . You seem to be
interested in A but ignore B and C. If you can rebuild A, why not B and
C? In this scenario you have crippled Maven and would need to rely on a
Build Management system to have mapped the relationships and have
captured the meta-data. If the relationships are mapped between A,B & C
then you would have some record of which ones were used. But, lets play
devils advocate and say we weren't following our process and people were
skipping steps. Your scenario is the *we didn't know* one and *can we go
back*?

In your case you are treating B and C as *external* dependencies and you
seem to assume they aren't hitting any milestones and *releasing*. As
external deps (separate SVN roots) we would never rely on a SNAPSHOT
version of them. In a CD world we would live with the most recent
*release* they have until they delivered new value. So, if we really
wanted their latest SNAPSHOT we would get it released so we can consume
it.

If it were in our repo root, we can go back and recreate the whole repo
and rebuild everything. Only *our* stuff can be SNAPSHOT (again, we're
doing the work).

If we take your logic to it's extreme are we saying the CD can not
tolerate a broken build? It can't possible be delivered! You've
abandoned Tags and Branches so I assume you have a fall forward
mentality (or roll back). In the case of a broken build in a CI system
using a Build Management System we should be able to very quickly get
back to a good build which could then be delivered.

What's more is I feel you are taking a narrow look at things. A world of
only SCC and Maven with no Build Management System. In this world you
would need to enable uniquely ID's snapshots in your Binary Repository
and at the very least retain your build logs, then you could go back to
that build of A and look at the log to see exactly which SNAPSHOTs were
used. Now you can have big continuous *jello* view of the world and
still have a detailed bill-of-materials. Not that you'd need it mind you
since your artifact would have been uniquely ID'd and stored (presumably
as a complete package somewhere) and now you don't even have to rebuild,
just go back and grab it and go from there.

While this is not how I'd use Maven, if you are willing to throw disk
space at the problem, so you can just keep everything and you're all
set.

________________________________

Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
Making IT Happen, one build at a time, 600 times a day

-----Original Message-----
From: jhumble [mailto:[hidden email]]
Sent: Sunday, November 07, 2010 11:41 PM
To: [hidden email]
Subject: Re: Continuous Delivery and Maven


Hi Curtis

I'm the first to admit I'm no Maven expert.

So please let me just confirm. Let's assume I am working on project A
which depends on projects B and C. For the sake of argument, let's say
that the source code for A, B and C have separate roots in SVN, and are
usually checked out independently. The CI system builds A at version 50
using snapshot dependencies on B and C, which are fetched from a central
snapshot repository.

Later, there are multiple updates to projects B and C which result in
new versions of them becoming available.

Say I now check out the source code to project A to version 50, because
I want to try and debug some issue that cropped up in a performance
testing environment, and I run a maven build. Will that use the latest
versions of the snapshots from the repo, or the versions that were
originally fetched when it was run on the CI system?

Can I even find out exactly which versions from svn the snapshots of B
and C came from that were used by the CI system to generate the original
build of A?

Thanks,

Jez.

On 7 November 2010 20:10, Yanko, Curtis [via Maven] <
[hidden email]<ml-node%2B3254520-103656
[hidden email]>
> wrote:

> Very interesting discussion. With all due respect to Mr. Humble, and I

> am a big fan of CD, I am going to venture to say that you don't
> understand Maven very well. As a thought experiment, you are correct
> in saying that a build based on snapshots is not reproducible. As a
> more practical matter however, I feel it is.
>
> Dependencies come in two flavors, our and theirs (internal and 3rd
> party). If, all of *our* dependencies are SNAPSHOT (we're doing the
> developing) and all of *theirs* are 'versioned' then the build is in
> fact reproducible assuming you build everything from a particular repo

> version even with the default auto-update setting (in fact, it's
> required).
> ________________________________
>
> Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
> Making IT Happen, one build at a time, 600 times a day
>
> -----Original Message-----
> From: jhumble [mailto:[hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=0>]
>
> Sent: Sunday, November 07, 2010 11:15 AM
> To: [hidden email]
> <http://user/SendEmail.jtp?type=node&node=3254520&i=1>
> Subject: RE: Continuous Delivery and Maven
>
>
> Hey Todd
>
> The whole point of continuous delivery is that every check-in creates
> a potential release candidate.
>
> When you're doing continuous deployment, you could be releasing
> multiple times a day, so you don't bother cutting branches or tagging
> or any of that stuff because of the overhead. I'd rather not get into
> the justification for this process on this thread - but I wrote a book

> on it if you're interested:
> http://www.amazon.com/gp/product/0321601912 and many other people have

> blogged about it.
>
> You're right that creating a concrete release for each commit could
> potentially use up a lot of space - but that's fine, you can just
> delete the older ones. What this *does* mean in turn though is that it

> is essential to be able to recreate any given build given the version
> in source control it came from, and this is where Maven falls down.
> Snapshots just aren't suitable because they aren't reproducible: what
> the snapshot looks like depends not only on what versions of the
> dependencies are available at the time the snapshot is created, but
> also what Maven's configuration and plug-ins happen to be at the time
> you run it (assuming Maven is configured to auto-update - the
> default). I can't revert back to a particular revision in version
> control, run maven, and be sure that the artifact it generates is
> identical to the one it created when the commit was initially

triggered.
>
> Ideally what I'd like is for Maven to explicitly support the
> continuous delivery model and provide snapshots that are reproducible.

> Failing that, a guide to configuring Maven so that its binaries are
> reproducible (for example by switching off auto-update, and having
> sufficient metadata stored in pom files and Maven's artifacts
> repository to know what the state of each of the dependencies was at
any given time.

> --
> View this message in context:
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp32453
> 70<http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp32
> 45370?by-user=t>
> p3254090.html
> Sent from the Maven - Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=2>
> For additional commands, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=3>
>
>
> This e-mail, including attachments, may include confidential and/or
> proprietary information, and may be used only by the person or entity
> to which it is addressed. If the reader of this e-mail is not the
> intended recipient or his or her authorized agent, the reader is
> hereby notified that any dissemination, distribution or copying of
> this e-mail is prohibited. If you have received this e-mail in error,
> please notify the sender by replying to this message and delete this

e-mail immediately.

>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=4>
> For additional commands, e-mail: [hidden
> email]<http://user/SendEmail.jtp?type=node&node=3254520&i=5>
>
>
>
> ------------------------------
>  View message @
> http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp32453
> 70p3254520.html To unsubscribe from Continuous Delivery and Maven,
> click

here<http://maven.40175.n5.nabble.com/template/TplServlet.jtp?tpl=unsubs
cribe_by_code&node=3245370&code=amV6QGplemh1bWJsZS5uZXR8MzI0NTM3MHwtMTg4
MjM1NzMyNA==>.
>
>
>


--
Jez Humble
Co-author, *Continuous Delivery <http://continuousdelivery.com/>*
http://continuousdelivery.com/ http://jezhumble.net/

--
View this message in context:
http://maven.40175.n5.nabble.com/Continuous-Delivery-and-Maven-tp3245370
p3254534.html
Sent from the Maven - Users mailing list archive at Nabble.com.

This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Yanko, Curtis]

 
I would agree. CD is the logicval extension of CI, Continuous
Inspection, Continuous Testing and then and only then, Continuous
Delivery.

So I whole heartedly agree, Jez seems to have an ad hoc dev proves as an
input to his CD process and despite that, Maven can still make it work.

I also whole heartedly agree that while I am careful to protect the
foundational notion of Reproducibility, in practice it just doesn't
happen, we are always going forward and once you start treating builds
as build lives and start managing binaries in addition to code, you
never need to go back and reproduce a build.

________________________________

Curt Yanko | Continuous Integration Services | UnitedHealth Group IT
Making IT Happen, one build at a time, 600 times a day

-----Original Message-----
From: Stephen Connolly [mailto:[hidden email]]
Sent: Monday, November 08, 2010 9:17 AM
To: Maven Users List
Subject: Re: Continuous Delivery and Maven

On 8 November 2010 13:30, Thiessen, Todd (Todd) <[hidden email]>
wrote:
> Interestingly enough, I kind of feel that we have a different
definition of continuous integration.
>

Interestingly enought, the original poster is talking about Continuous
_D_E_L_I_V_E_R_Y_ as distinct fron Continuous Integrration ;-)

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]


This e-mail, including attachments, may include confidential and/or
proprietary information, and may be used only by the person or entity
to which it is addressed. If the reader of this e-mail is not the intended
recipient or his or her authorized agent, the reader is hereby notified
that any dissemination, distribution or copying of this e-mail is
prohibited. If you have received this e-mail in error, please notify the
sender by replying to this message and delete this e-mail immediately.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[stephenconnolly]

CI does not end up on production systems, so CI does not require as
rigourously reproducible a build as CD.

With CI, what you want to have happen is the latest code ends up on an
integration system and integration tests are run against the
integration system in order to identify integration issues before the
code goes to QA.

With CD you have as much of the QA process automated as you are
comfrtable with, and when you kick off a build, it will automatically
run though an entirely automated process that ends up with the build
either failing or being deployed onto the live system... (note that
you might have one or two manual verifications before deploying to
production systems, but sooner or later you will have developed enough
confidence in your CD process that you can feel confident removing the
manual blocks)

With CI, I can quite happily work with -SNAPSHOTs because all I need
to know is that something is broken.

With CD, I need releases (but perhaps more lightweight than Maven's
current release process... perhaps I do not need to create tags).

The question is what triggers the deploy in CD.

There are a number of possible triggers:

1. Manual trigger... where I click a button and the whole process starts
2. CI trigger... where a CI build passing triggers the whole process
3. Commit trigger... where any commit triggers the whole process.

The problem with #1 is that you have to remember to trigger

If you are doing CD correctly, then #2 and #3 are actually the same
thing with just a re-ordered pipeline.

#2 goes a little something like this
2.1 A commit triggers a build
2.2 The build passes and triggers a CI build
2.3 The CI build deploys to the integration system and runs the
integration tests
2.4 The CI build passes and triggers the CD build
2.5 The CD build runs a release of the software
2.6 The CD build deploys the release to the integration system and
runs the integration tests
2.7 The CD build runs the last ditch "no egg on face" tests that could
take a bit longer than integration tests (e.g. full regression)
2.8 All tests have passed and the CD build meets the release to
customer criteria
2.9 The CD build deploys the release to production systems
2.10 we are live

while #3 goes a little something like this
3.1 The CD build runs a release of the software
3.2 The CD build deploys the release to the integration system and
runs the integration tests
3.3 The CD build runs the last ditch "no egg on face" tests that could
take a bit longer than integration tests (e.g. full regression)
3.4 All tests have passed and the CD build meets the release to
customer criteria
3.5 The CD build deploys the release to production systems
3.6 we are live

#2 saves on the churn of making releases but reduces the response time
for deployment.

In any case you also want an automated process that allows you to roll
the production system back to a previous state just in case the "no
egg on face" tests let some egg slip through to your face!

-Stephen
On 8 November 2010 14:27, Thiessen, Todd (Todd) <[hidden email]> wrote:

> True. But how does that change my statement? It still appears we may have a different definition of CI.  Because in order to do CD, you would no longer be doing CI.
>
> Unless of course the original poster is suggesting to abandon CI.
>
>> -----Original Message-----
>> From: Stephen Connolly [mailto:[hidden email]]
>> Sent: Monday, November 08, 2010 9:17 AM
>> To: Maven Users List
>> Subject: Re: Continuous Delivery and Maven
>>
>> On 8 November 2010 13:30, Thiessen, Todd (Todd) <[hidden email]>
>> wrote:
>> > Interestingly enough, I kind of feel that we have a different
>> definition of continuous integration.
>> >
>>
>> Interestingly enought, the original poster is talking about Continuous
>> _D_E_L_I_V_E_R_Y_ as distinct fron Continuous Integrration ;-)
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]