Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Aldrin Leal
This is quite similar to what "go get" does to golang. I'd say its worth a
try

On May 14, 2017 09:28, "Paul Hammant" <[hidden email]> wrote:

> Article updated to eliminate misunderstandings and talk about a different
> index for 'maven central' too.
>
> - ph
>
> On Sat, May 13, 2017 at 3:04 PM, Paul Hammant <[hidden email]> wrote:
>
> > I was discussing this of the list today, and it may interest people on
> dev@
> >
> > https://paulhammant.com/2017/05/13/maven-central-as-multiple-git-repos/
> >
> >      "Maven Central as multiple Git repositories"
> >
> > Enjoy,
> >
> > - Paul
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

rfscholte
I consider this as a new repository implementation. But this also implies  
that in your pom, for every dependency you have to add a repository-entry  
as well, right?

Robert

On Wed, 17 May 2017 17:10:49 +0200, Aldrin Leal <[hidden email]> wrote:

> Just a friendly reminder that git is not optimized for large files (for
> this, they made git-lfs). Plus, when you do checkout a git repo, there's  
> no
> binary diffs - so if you've got plenty of releases, you'll be wasting a  
> lot
> of space/time in terms of transmission and storage.
>
> --
> -- Aldrin Leal, <[hidden email]> / http://about.me/aldrinleal
>
> On Wed, May 17, 2017 at 9:37 AM, Paul Hammant <[hidden email]> wrote:
>
>> We can agree to differ on what I'm suggesting and what the impact of  
>> that
>> would be :)
>>
>> On Wed, May 17, 2017 at 8:59 AM, Brian Fox <[hidden email]> wrote:
>>
>> > Even more than redefining what Central does, you're effectively
>> describing
>> > a new, unofficial java class packaging and distribution mechanism.  
>> This
>> > seems like it will violate signatures etc and make tracking of what  
>> you
>> > actually have a nightmare.
>> >
>> > On Tue, May 16, 2017 at 5:55 PM, Hervé BOUTEMY <[hidden email]>
>> > wrote:
>> >
>> > > this idea of putting everything in git is funny: not sure this will  
>> go
>> > very
>> > > far from this poc, but let's imagine...
>> > >
>> > > on classes branch, splitting the jar into individual .class has  
>> IMHO a
>> > big
>> > > drawback: we loose original jar and its signature
>> > >
>> > > On the other branches, the current poc shows commits for versions  
>> that
>> > are
>> > > perfectly linear: if there are multiple branches that are released  
>> in
>> > > parallel, the commit won't be so clean. I don't know if this will  
>> have
>> an
>> > > impact on compression efficiency.
>> > >
>> > > Another issue with this idea: during development, with SNAPSHOTs,  
>> the
>> git
>> > > repo
>> > > will be polluted: this idea IMHO could only be valid for releases
>> > >
>> > > not to speak about read concurrency when one requires to use  
>> multiple
>> > > versions
>> > > of a lib. And of course, write concurrency is even harder.
>> > >
>> > >
>> > > Definitely, the idea is funny, but I don't see how this could go  
>> very
>> far
>> > > than
>> > > this funny idea (in addition to the complexity for implementing this
>> > > format in
>> > > tooling)
>> > >
>> > > Regards,
>> > >
>> > > Hervé
>> > >
>> > > Le lundi 15 mai 2017, 21:45:00 CEST Paul Hammant a écrit :
>> > > > One more repo:
>> > > >
>> > > > https://github.com/paul-hammant/mc-xs-all/
>> > > >
>> > > > One branch for each of classes, javadoc, sources, and poms
>> > > >
>> > > > 15 javadoc original versions: 24.1M
>> > > >
>> > > > 16 sources original versions: 4.9M
>> > > >
>> > > > 27 classes original versions: 8.4M
>> > > >
>> > > > Afterwards git work the bare .git folder is: 8.4M
>> > > >
>> > > > *77.5% saving on storage*
>> > > >
>> > > > Any artifact, *including the poms,* can be pulled down via a  
>> single
>> git
>> > > > command
>> > > >
>> > > > git clone https://github.com/paul-hammant/mc-xs-classes --depth 1
>> > > --branch
>> > > > TAGNAME
>> > > >
>> > > > 74 TAGNAMEs: classes-0.1, classes-0.2, classes-0.3, classes-0.5,
>> > > > classes-0.6, classes-1.0, classes-1.0.1, classes-1.0.2,  
>> classes-1.1,
>> > > > classes-1.1.1, classes-1.1.2, classes-1.1.3, classes-1.2,
>> > classes-1.2.1,
>> > > > classes-1.2.2, classes-1.3, classes-1.3.1, classes-1.4,
>> classes-1.4.1,
>> > > > classes-1.4.2, classes-1.4.3, classes-1.4.4, classes-1.4.5,
>> > > classes-1.4.6,
>> > > > classes-1.4.7, classes-1.4.8, classes-1.4.9, javadoc-1.2,
>> > javadoc-1.2.1,
>> > > > javadoc-1.2.2, javadoc-1.3, javadoc-1.3.1, javadoc-1.4,
>> javadoc-1.4.1,
>> > > > javadoc-1.4.2, javadoc-1.4.3, javadoc-1.4.4, javadoc-1.4.5,
>> > > javadoc-1.4.6,
>> > > > javadoc-1.4.7, javadoc-1.4.8, javadoc-1.4.9, pom-1.2, pom-1.2.1,
>> > > pom-1.2.2,
>> > > > pom-1.3, pom-1.3.1, pom-1.4, pom-1.4.1, pom-1.4.2, pom-1.4.3,
>> > pom-1.4.4,
>> > > > pom-1.4.5, pom-1.4.6, pom-1.4.7, pom-1.4.8, pom-1.4.9,  
>> sources-1.1.3,
>> > > > sources-1.2, sources-1.2.1, sources-1.2.2, sources-1.3,
>> sources-1.3.1,
>> > > > sources-1.4, sources-1.4.1, sources-1.4.2, sources-1.4.3,
>> > sources-1.4.4,
>> > > > sources-1.4.5, sources-1.4.6, sources-1.4.7, sources-1.4.8,
>> > sources-1.4.9
>> > > >
>> > > >  - Paul
>> > >
>> > >
>> > >
>> > >  
>> ---------------------------------------------------------------------
>> > > To unsubscribe, e-mail: [hidden email]
>> > > For additional commands, e-mail: [hidden email]
>> > >
>> > >
>> >

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Paul Hammant
In reply to this post by Aldrin Leal
Aldrin - https://github.com/paul-hammant/mc-xs-all - no large files added
to Git. Git makes 70% saving on bytes used ('bare' mode).

On Wed, May 17, 2017 at 11:10 AM, Aldrin Leal <[hidden email]> wrote:

> Just a friendly reminder that git is not optimized for large files (for
> this, they made git-lfs). Plus, when you do checkout a git repo, there's no
> binary diffs - so if you've got plenty of releases, you'll be wasting a lot
> of space/time in terms of transmission and storage.
>
> --
> -- Aldrin Leal, <[hidden email]> / http://about.me/aldrinleal
>
> On Wed, May 17, 2017 at 9:37 AM, Paul Hammant <[hidden email]> wrote:
>
> > We can agree to differ on what I'm suggesting and what the impact of that
> > would be :)
> >
> > On Wed, May 17, 2017 at 8:59 AM, Brian Fox <[hidden email]> wrote:
> >
> > > Even more than redefining what Central does, you're effectively
> > describing
> > > a new, unofficial java class packaging and distribution mechanism. This
> > > seems like it will violate signatures etc and make tracking of what you
> > > actually have a nightmare.
> > >
> > > On Tue, May 16, 2017 at 5:55 PM, Hervé BOUTEMY <[hidden email]>
> > > wrote:
> > >
> > > > this idea of putting everything in git is funny: not sure this will
> go
> > > very
> > > > far from this poc, but let's imagine...
> > > >
> > > > on classes branch, splitting the jar into individual .class has IMHO
> a
> > > big
> > > > drawback: we loose original jar and its signature
> > > >
> > > > On the other branches, the current poc shows commits for versions
> that
> > > are
> > > > perfectly linear: if there are multiple branches that are released in
> > > > parallel, the commit won't be so clean. I don't know if this will
> have
> > an
> > > > impact on compression efficiency.
> > > >
> > > > Another issue with this idea: during development, with SNAPSHOTs, the
> > git
> > > > repo
> > > > will be polluted: this idea IMHO could only be valid for releases
> > > >
> > > > not to speak about read concurrency when one requires to use multiple
> > > > versions
> > > > of a lib. And of course, write concurrency is even harder.
> > > >
> > > >
> > > > Definitely, the idea is funny, but I don't see how this could go very
> > far
> > > > than
> > > > this funny idea (in addition to the complexity for implementing this
> > > > format in
> > > > tooling)
> > > >
> > > > Regards,
> > > >
> > > > Hervé
> > > >
> > > > Le lundi 15 mai 2017, 21:45:00 CEST Paul Hammant a écrit :
> > > > > One more repo:
> > > > >
> > > > > https://github.com/paul-hammant/mc-xs-all/
> > > > >
> > > > > One branch for each of classes, javadoc, sources, and poms
> > > > >
> > > > > 15 javadoc original versions: 24.1M
> > > > >
> > > > > 16 sources original versions: 4.9M
> > > > >
> > > > > 27 classes original versions: 8.4M
> > > > >
> > > > > Afterwards git work the bare .git folder is: 8.4M
> > > > >
> > > > > *77.5% saving on storage*
> > > > >
> > > > > Any artifact, *including the poms,* can be pulled down via a single
> > git
> > > > > command
> > > > >
> > > > > git clone https://github.com/paul-hammant/mc-xs-classes --depth 1
> > > > --branch
> > > > > TAGNAME
> > > > >
> > > > > 74 TAGNAMEs: classes-0.1, classes-0.2, classes-0.3, classes-0.5,
> > > > > classes-0.6, classes-1.0, classes-1.0.1, classes-1.0.2,
> classes-1.1,
> > > > > classes-1.1.1, classes-1.1.2, classes-1.1.3, classes-1.2,
> > > classes-1.2.1,
> > > > > classes-1.2.2, classes-1.3, classes-1.3.1, classes-1.4,
> > classes-1.4.1,
> > > > > classes-1.4.2, classes-1.4.3, classes-1.4.4, classes-1.4.5,
> > > > classes-1.4.6,
> > > > > classes-1.4.7, classes-1.4.8, classes-1.4.9, javadoc-1.2,
> > > javadoc-1.2.1,
> > > > > javadoc-1.2.2, javadoc-1.3, javadoc-1.3.1, javadoc-1.4,
> > javadoc-1.4.1,
> > > > > javadoc-1.4.2, javadoc-1.4.3, javadoc-1.4.4, javadoc-1.4.5,
> > > > javadoc-1.4.6,
> > > > > javadoc-1.4.7, javadoc-1.4.8, javadoc-1.4.9, pom-1.2, pom-1.2.1,
> > > > pom-1.2.2,
> > > > > pom-1.3, pom-1.3.1, pom-1.4, pom-1.4.1, pom-1.4.2, pom-1.4.3,
> > > pom-1.4.4,
> > > > > pom-1.4.5, pom-1.4.6, pom-1.4.7, pom-1.4.8, pom-1.4.9,
> sources-1.1.3,
> > > > > sources-1.2, sources-1.2.1, sources-1.2.2, sources-1.3,
> > sources-1.3.1,
> > > > > sources-1.4, sources-1.4.1, sources-1.4.2, sources-1.4.3,
> > > sources-1.4.4,
> > > > > sources-1.4.5, sources-1.4.6, sources-1.4.7, sources-1.4.8,
> > > sources-1.4.9
> > > > >
> > > > >  - Paul
> > > >
> > > >
> > > >
> > > > ------------------------------------------------------------
> ---------
> > > > To unsubscribe, e-mail: [hidden email]
> > > > For additional commands, e-mail: [hidden email]
> > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Paul Hammant
In reply to this post by Aldrin Leal
Hervé,

on classes branch, splitting the jar into individual .class has IMHO a big
> drawback: we loose original jar and its signature
>

Agree. Git doesn't care about timestamps for classes in jars. Java doesn't
either, but SHA1 (etc) of the jar does.

Thus - the next iteration will reproduce even the timestamps of a resulting
jar.


>
> On the other branches, the current poc shows commits for versions that are
> perfectly linear: if there are multiple branches that are released in
> parallel, the commit won't be so clean. I don't know if this will have an
> impact on compression efficiency.
>

They can go in any random order (I tested) and Git achieves the same over
all saving.


>
> Another issue with this idea: during development, with SNAPSHOTs, the git
> repo
> will be polluted: this idea IMHO could only be valid for releases
>

That's true. I'm really only focussed on the bring-down-from-maven-central
cycle. Obviously I need an answer for the on-workstation workflow which
inserts into ~/.m2/repository/ too.


>
> not to speak about read concurrency when one requires to use multiple
> versions
> of a lib. And of course, write concurrency is even harder.
>

I'm not following, dude.

- Paul
Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Paul Hammant
In reply to this post by Paul Hammant
I'm "just" unzipping the source, and committing those sources. Sure, I
delete the previous set first, but I merge the *rm* set and *add* set into
one commit with an --amend ->

 # fn is xstream-1.4.3.jar and v is 1.4.3 for example

 git("rm", "-r", "*")

            git("commit", "-m", v)
            wget(root + url + "/" + v + "/" + fn)
            unzip(fn)
            rm(fn)
            git("add", ".")
            git("commit", "-m", v, "--amend")

On Wed, May 17, 2017 at 3:41 PM, Aldrin Leal <[hidden email]> wrote:

> thats my point: the golang approach does no magic at all. It simply stores
> the source code and bases it on a convention. Just the files, and thats it.
>
> --
> -- Aldrin Leal, <[hidden email]> / http://about.me/aldrinleal
>
> On Wed, May 17, 2017 at 2:38 PM, Paul Hammant <[hidden email]> wrote:
>
> > Aldrin, The blog entry I wrote on saturday mulled classes, javadocs and
> > sources -
> > https://paulhammant.com/2017/05/13/maven-central-as-multiple-git-repos/
> > (re
> > your "way more than" comment).
> >
> > The GH repo I linked you to earlier has all three in one repo (see the
> > branches drop-down) - https://github.com/paul-hammant/mc-xs-all
> >
> > - Paul
> >
> >
> >
> > On Wed, May 17, 2017 at 2:54 PM, Aldrin Leal <[hidden email]> wrote:
> >
> > > I understand the approach is basically general, but maven artifacts are
> > way
> > > more than binary code (there's source and javadoc). I also understand
> its
> > > an interesting option for distribution.
> > >
> > > I really would like to see something close to what "go get" does. If
> not
> > > github and bitbucket (and go get includes git, hg and bzr among scms),
> it
> > > open the URL itself and resolve it into a(by means of HTML metadata).
> > >
> > > Just distributing the source allowing it to easily updates would be
> > > awesome, since it would allow less effort to create and distribute. Of
> > > course, we could have to limit the scope of what to do to avoid abuse.
> > >
> > >
> > > --
> > > -- Aldrin Leal, <[hidden email]> / http://about.me/aldrinleal
> > >
> > > On Wed, May 17, 2017 at 1:49 PM, Paul Hammant <[hidden email]>
> wrote:
> > >
> > > > There is that, yes. And Git's general upper limits which are subject
> of
> > > "I
> > > > heard of a team that had a corruption at 2GB".  I've field tested Git
> > up
> > > to
> > > > 7GB for a git-svn-clone myself (a team considering saying bye bye to
> > > Svn),
> > > > but wouldn't put that live versus hive off history to a R/O repo, and
> > > start
> > > > over with HEAD of the old becoming the initial commit of the new.
> > > >
> > > > - Paul
> > > >
> > > > On Wed, May 17, 2017 at 2:40 PM, Aldrin Leal <[hidden email]>
> > wrote:
> > > >
> > > > > Still, once github gets an outage, our repositories are basically
> > > > > 'left-padded' (taken offline)
> > > > >
> > > > > --
> > > > > -- Aldrin Leal, <[hidden email]> / http://about.me/aldrinleal
> > > > >
> > > > > On Wed, May 17, 2017 at 1:35 PM, Paul Hammant <[hidden email]>
> > > wrote:
> > > > >
> > > > > > Aldrin - https://github.com/paul-hammant/mc-xs-all - no large
> > files
> > > > > added
> > > > > > to Git. Git makes 70% saving on bytes used ('bare' mode).
> > > > > >
> > > > > > On Wed, May 17, 2017 at 11:10 AM, Aldrin Leal <
> [hidden email]>
> > > > > wrote:
> > > > > >
> > > > > > > Just a friendly reminder that git is not optimized for large
> > files
> > > > (for
> > > > > > > this, they made git-lfs). Plus, when you do checkout a git
> repo,
> > > > > there's
> > > > > > no
> > > > > > > binary diffs - so if you've got plenty of releases, you'll be
> > > > wasting a
> > > > > > lot
> > > > > > > of space/time in terms of transmission and storage.
> > > > > > >
> > > > > > > --
> > > > > > > -- Aldrin Leal, <[hidden email]> /
> > http://about.me/aldrinleal
> > > > > > >
> > > > > > > On Wed, May 17, 2017 at 9:37 AM, Paul Hammant <
> [hidden email]>
> > > > > wrote:
> > > > > > >
> > > > > > > > We can agree to differ on what I'm suggesting and what the
> > impact
> > > > of
> > > > > > that
> > > > > > > > would be :)
> > > > > > > >
> > > > > > > > On Wed, May 17, 2017 at 8:59 AM, Brian Fox <
> [hidden email]
> > >
> > > > > wrote:
> > > > > > > >
> > > > > > > > > Even more than redefining what Central does, you're
> > effectively
> > > > > > > > describing
> > > > > > > > > a new, unofficial java class packaging and distribution
> > > > mechanism.
> > > > > > This
> > > > > > > > > seems like it will violate signatures etc and make tracking
> > of
> > > > what
> > > > > > you
> > > > > > > > > actually have a nightmare.
> > > > > > > > >
> > > > > > > > > On Tue, May 16, 2017 at 5:55 PM, Hervé BOUTEMY <
> > > > > > [hidden email]>
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > this idea of putting everything in git is funny: not sure
> > > this
> > > > > will
> > > > > > > go
> > > > > > > > > very
> > > > > > > > > > far from this poc, but let's imagine...
> > > > > > > > > >
> > > > > > > > > > on classes branch, splitting the jar into individual
> .class
> > > has
> > > > > > IMHO
> > > > > > > a
> > > > > > > > > big
> > > > > > > > > > drawback: we loose original jar and its signature
> > > > > > > > > >
> > > > > > > > > > On the other branches, the current poc shows commits for
> > > > versions
> > > > > > > that
> > > > > > > > > are
> > > > > > > > > > perfectly linear: if there are multiple branches that are
> > > > > released
> > > > > > in
> > > > > > > > > > parallel, the commit won't be so clean. I don't know if
> > this
> > > > will
> > > > > > > have
> > > > > > > > an
> > > > > > > > > > impact on compression efficiency.
> > > > > > > > > >
> > > > > > > > > > Another issue with this idea: during development, with
> > > > SNAPSHOTs,
> > > > > > the
> > > > > > > > git
> > > > > > > > > > repo
> > > > > > > > > > will be polluted: this idea IMHO could only be valid for
> > > > releases
> > > > > > > > > >
> > > > > > > > > > not to speak about read concurrency when one requires to
> > use
> > > > > > multiple
> > > > > > > > > > versions
> > > > > > > > > > of a lib. And of course, write concurrency is even
> harder.
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Definitely, the idea is funny, but I don't see how this
> > could
> > > > go
> > > > > > very
> > > > > > > > far
> > > > > > > > > > than
> > > > > > > > > > this funny idea (in addition to the complexity for
> > > implementing
> > > > > > this
> > > > > > > > > > format in
> > > > > > > > > > tooling)
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > >
> > > > > > > > > > Hervé
> > > > > > > > > >
> > > > > > > > > > Le lundi 15 mai 2017, 21:45:00 CEST Paul Hammant a écrit
> :
> > > > > > > > > > > One more repo:
> > > > > > > > > > >
> > > > > > > > > > > https://github.com/paul-hammant/mc-xs-all/
> > > > > > > > > > >
> > > > > > > > > > > One branch for each of classes, javadoc, sources, and
> > poms
> > > > > > > > > > >
> > > > > > > > > > > 15 javadoc original versions: 24.1M
> > > > > > > > > > >
> > > > > > > > > > > 16 sources original versions: 4.9M
> > > > > > > > > > >
> > > > > > > > > > > 27 classes original versions: 8.4M
> > > > > > > > > > >
> > > > > > > > > > > Afterwards git work the bare .git folder is: 8.4M
> > > > > > > > > > >
> > > > > > > > > > > *77.5% saving on storage*
> > > > > > > > > > >
> > > > > > > > > > > Any artifact, *including the poms,* can be pulled down
> > via
> > > a
> > > > > > single
> > > > > > > > git
> > > > > > > > > > > command
> > > > > > > > > > >
> > > > > > > > > > > git clone https://github.com/paul-
> hammant/mc-xs-classes
> > > > > --depth
> > > > > > 1
> > > > > > > > > > --branch
> > > > > > > > > > > TAGNAME
> > > > > > > > > > >
> > > > > > > > > > > 74 TAGNAMEs: classes-0.1, classes-0.2, classes-0.3,
> > > > > classes-0.5,
> > > > > > > > > > > classes-0.6, classes-1.0, classes-1.0.1, classes-1.0.2,
> > > > > > > classes-1.1,
> > > > > > > > > > > classes-1.1.1, classes-1.1.2, classes-1.1.3,
> classes-1.2,
> > > > > > > > > classes-1.2.1,
> > > > > > > > > > > classes-1.2.2, classes-1.3, classes-1.3.1, classes-1.4,
> > > > > > > > classes-1.4.1,
> > > > > > > > > > > classes-1.4.2, classes-1.4.3, classes-1.4.4,
> > classes-1.4.5,
> > > > > > > > > > classes-1.4.6,
> > > > > > > > > > > classes-1.4.7, classes-1.4.8, classes-1.4.9,
> javadoc-1.2,
> > > > > > > > > javadoc-1.2.1,
> > > > > > > > > > > javadoc-1.2.2, javadoc-1.3, javadoc-1.3.1, javadoc-1.4,
> > > > > > > > javadoc-1.4.1,
> > > > > > > > > > > javadoc-1.4.2, javadoc-1.4.3, javadoc-1.4.4,
> > javadoc-1.4.5,
> > > > > > > > > > javadoc-1.4.6,
> > > > > > > > > > > javadoc-1.4.7, javadoc-1.4.8, javadoc-1.4.9, pom-1.2,
> > > > > pom-1.2.1,
> > > > > > > > > > pom-1.2.2,
> > > > > > > > > > > pom-1.3, pom-1.3.1, pom-1.4, pom-1.4.1, pom-1.4.2,
> > > pom-1.4.3,
> > > > > > > > > pom-1.4.4,
> > > > > > > > > > > pom-1.4.5, pom-1.4.6, pom-1.4.7, pom-1.4.8, pom-1.4.9,
> > > > > > > sources-1.1.3,
> > > > > > > > > > > sources-1.2, sources-1.2.1, sources-1.2.2, sources-1.3,
> > > > > > > > sources-1.3.1,
> > > > > > > > > > > sources-1.4, sources-1.4.1, sources-1.4.2,
> sources-1.4.3,
> > > > > > > > > sources-1.4.4,
> > > > > > > > > > > sources-1.4.5, sources-1.4.6, sources-1.4.7,
> > sources-1.4.8,
> > > > > > > > > sources-1.4.9
> > > > > > > > > > >
> > > > > > > > > > >  - Paul
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > ------------------------------
> > ------------------------------
> > > > > > > ---------
> > > > > > > > > > To unsubscribe, e-mail: [hidden email]
> > > > > > > > > > For additional commands, e-mail:
> [hidden email]
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Paul Hammant
The Maven-using developer community cares that the dependency downloader
does its think that the uploader/deploy does too. Some new releases of
those, and some back releases for the hard breaks (if any) going back in
time - Maven1, Maven2 and whatever.

Works well enough for Homebrew (https://github.com/homebrew - the core repo
is at 125MB). Sure there's been teething troubles, and Ruby was the wrong
choice for the DSL versus Python, but it is great really.



On Wed, May 17, 2017 at 5:25 PM, Manfred Moser <[hidden email]>
wrote:

> If you would run Central on git like that in a centralized manner you
> would have to find someone that does that hosting for you and you would
> have to get buy in from the community to use that - both extremely hard or
> impossible.
>
> And if you dont do that but instead go with the distributed system you end
> up with the registry model that I think just doesnt really work in the real
> world.
>
> manfred
>
> Paul Hammant wrote on 2017-05-17 13:39:
>
> > Actually I'm proposing a predictable structure on 'central :
> >
> > [hidden email]:
> > maven2/<group-name-with-slashes-for-dots</<artifact-name>.git
> >
> > (one minor fix versus previous description of the git:// location)
> >
> > Or for the three separate variant:
> >
> > [hidden email]:
> > maven2/<group-name-with-slashes-for-dots</<artifact-name>-classes.git
> > [hidden email]:
> > maven2/<group-name-with-slashes-for-dots</<artifact-name>-javadocs.git
> > [hidden email]:
> > maven2/<group-name-with-slashes-for-dots</<artifact-name>-sources.git
> >
> > More likely though (as you mentioned in your opening line) is the way
> > homebrew works - you point at repos elsewhere, but control poms/shas etc
> on
> > 'central.
> >
> >
> >
> >
> > On Wed, May 17, 2017 at 3:59 PM, Manfred Moser <[hidden email]
> >
> > wrote:
> >
> >> Having worked with repository managers and the implementation for
> various
> >> formats on Nexus for years I think such a format is a bit like Bower.
> It is
> >> a registry format that in turn points to git repositories that have the
> >> content.
> >>
> >> From a corporate usage and implementation point of view this is a utter
> >> nightmare since you would have to open up your systems to all those
> >> different repositories and sites hosting them instead of just one.
> >>
> >> You also cant simply make a copy of the content or analyze it in the way
> >> it manifests as binaries.
> >>
> >> I am not sure what you are looking for as benefits but from my point of
> >> view this is maybe a fun experiment but not something that will ever
> take
> >> off..
> >>
> >> Manfred
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >>
> >>
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Paul Hammant
In reply to this post by Paul Hammant
>
> I'm not sure I'm not mistaken for the read part: with the current repo
> format,
> one thing that is super easy is that each artifact version is just a path
> that
> you can use simultaneously in one build or in multiple builds
>

e.g.

build #1 using --classpath ~/.m2/repo/tw/xstream-1.4.3.jar
build #2 (concurrent or soon after) using --classpath
~/.m2/repo/tw/xstream-1.4.4.jar
other builds using one of those two (again) or more, concurrently or not.

Got it.


> With the proposed git repos, I don't get how read access to a tag is done
> without changing disk state, so you can access mutliple tags at the same
> time.
>

Without a *fanciful* change to java and javac, the toolchain is stilll
going to need to make
jars as we've always known them.

The toolchain for recreating jars (notwithstanding timestamps on class
files issues previously noted) from the blog entry:

git clone https://github.com/paul-hammant/mc-xs-classes --depth 1 --branch
1.4.3
cd mc-xs-classes
rm -rf .git
jar cvfM ../xstream-1.4.3.jar .
cd ..
rm -rf mc-xs-classes

That might be faster for Java and the widely used JGit if it can be done in
memory. It took 3 secs for me, with 2 of that being the jar step.

On a development workstation, that style of use *exclusively* will mean you
can rebuild ~/.m2/repository as we know it today, jar by jar.
Specifically, you don't retain the jar .git folders per repo for things
pulled from from 'central at all.

Other people, optionally, could say then want to host their own .git/
centric cache of things pulled down from 'central.  That could be on a
network mount somehow, or (more realistically) an anemic Artifactory of
Nexus service. That could even be on their own box rather than for the
team, of they are happy to momentarily use *more* hard drive space than the
~/.m2/repository of today.

I think if had a local .git representation of things from 'central, I would
be deleting ~/.m2/repository every night when I wasn't using the machine. I
mean, in lieu of a more sophisticated solution.

Regards,

- Paul


>
>
> This "on read" question happens with "on write": but as you told the idea
> is
> more to consume releases from a central repository, and not on the desktop
> with SNAPSHOTs and so on, the write question is not really a question.
>
> Regards,
>
> Hervé
>
> >
> > - Paul
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
Reply | Threaded
Open this post in threaded view
|

Re: Silly Saturday idea - If Maven Central were a bunch of Git repos

Paul Hammant
You know, I think it would be cool if there were file systems that would
implement a form of a 'link' as part of a content-based-storage mechanism
like Git itself.  Meaning if I ran the above git-clone/jar command twice in
two different directories it would only store the result once (assuming
idempotence).  Hard and soft links are not it because they need a canonical
version that the duplicates point to.  What you want is the thing stored
once, and regardless of the order of deletion, the last deletion means it
is truly gone.  The Mac's 'alias' feature is closer because it allows you
to at least move the original/canonical.

Kind of like String.intern(..) in Java.

People are, of course, pushing the file system science in the direction of
content-based (with dire/filenames a logical overlay).

https://dev.arvados.org/projects/arvados/wiki/Keep
https://github.com/ipfs/ipfs
https://en.wikipedia.org/wiki/Content-addressable_storage


- paul