carrot-nexus-aws-s3-publish-plugin

[Andrei Pozolotin]

Hello, Nexus People;

I hope you could help me with the following 3 questions:

I need to replicate private nexus repository to amazon s3 bucket maven repo in real time,
so I wrote this proof-of-concept plugin:
https://github.com/carrot-garden/carrot-nexus/tree/master/carrot-nexus-aws-s3-publish-plugin

1) existing solution

is there by chance a solution in place which is already solving this need?
well, writing integration tests for nexus is a bit of a pain :-)
?

2) extension point

currently I process aws s3 publish request inside StorageFileItemInspector extension point,
which is "too late" - the item is already stored in nexus repo, from client point of view,
but this is no good, since aws s3 publish request still could fail after that.

I can not use RequestProcessor, since while here I can block client request,
I still do not have access to the StorageFileItem inside the
   public boolean process(repository, request, action)
invocation.

EventInspector is not good for the same reason - too late.

I am curious if there is another nexus extension point witch combines
both RequestProcessor & StorageFileItemInspector, namely, so I can
    a) block/fail client request if aws storage request fails
    b) have access to already-downloaded StorageFileItem actual File
    c) annotate StorageFileItem on the fly in case of aws storage request success
?

3) config example

nexus configuration management seems a little too much involved - at first :-)

is there a blog post or a github project which gives clear cut example
of bare bone implementation of how to :
    a) persist xml
    b) process config rest
    c) expose config in nexus gui
    d) not to use modello for all of this (or must I use it?)
?

Thank you,

Andrei

[Igor Fedorenko-3]

So you goal is to have the item stored both locally and in s3, right?
What is the expected behaviour if one of the stores fails for whatever
reason? Do you plan to implement some sort of two-phase commit or just
report the problem to the user but leave the stores in inconsistent state?

Have you looked at LocalRepositoryStorage? You should be able to write
new implementation that extends/wraps standard filesystem implementation
but also talks to s3, either before or after storing the item locally.

--
Regards,
Igor

On 12-11-29 1:35 PM, Andrei Pozolotin wrote:

>     *Hello, Nexus People;**
>     *
>     I hope you could help me with the following 3 questions:
>
>     I need to replicate private nexus repository to amazon s3 bucket
>     maven repo in real time,
>     so I wrote this proof-of-concept plugin:
>     https://github.com/carrot-garden/carrot-nexus/tree/master/carrot-nexus-aws-s3-publish-plugin
>
>     *1) existing solution**
>
>     *is there by chance a solution in place which is already solving
>     this need?
>     well, writing integration tests for nexus is a bit of a pain :-)
>     ?
>
>     *2) extension point**
>     *
>     currently I process aws s3 publish request inside
>     StorageFileItemInspector extension point,
>     which is "too late" - the item is already stored in nexus repo, from
>     client point of view,
>     but this is no good, since aws s3 publish request still could fail
>     after that.
>
>     I can not use RequestProcessor, since while here I can block client
>     request,
>     I still do not have access to the StorageFileItem inside the
>         public boolean process(repository, request, action)
>     invocation.
>
>     EventInspector is not good for the same reason - too late.
>
>     I am curious if there is another nexus extension point witch combines
>     both RequestProcessor & StorageFileItemInspector, namely, so I can
>          a) block/fail client request if aws storage request fails
>          b) have access to already-downloaded StorageFileItem actual File
>          c) annotate StorageFileItem on the fly in case of aws storage
>     request success
>     ?
>
>     *3) **config example**
>     *
>     nexus configuration management seems a little too much involved - at
>     first :-)
>
>     is there a blog post or a github project which gives clear cut example
>     of bare bone implementation of how to :
>          a) persist xml
>          b) process config rest
>          c) expose config in nexus gui
>          d) not to use modello for all of this (or must I use it?)
>     ?
>
>     Thank you,
>
>     Andrei
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Andrei Pozolotin]

Igor:

thanks for looking into this;

-------- Original Message --------
Subject: Re: [nexus-dev] carrot-nexus-aws-s3-publish-plugin
From: Igor Fedorenko [hidden email]
To: [hidden email]
Date: Thu 29 Nov 2012 09:56:01 PM CST

So you goal is to have the item stored both locally and in s3, right?

yes

What is the expected behaviour if one of the stores fails for whatever
reason? Do you plan to implement some sort of two-phase commit or just
report the problem to the user but leave the stores in inconsistent state?

I want to :
a) if both local and s3 store operation is success - report success to user
b) if any store operation is failed - do not store anything in any store, report failure to user

Have you looked at LocalRepositoryStorage? You should be able to write
new implementation that extends/wraps standard filesystem implementation
but also talks to s3, either before or after storing the item locally.

great idea; I will take a look;

given static plexus/guice injection pattern, will it be possible activate/deactivate
this new custom LocalRepositoryStorage dynamically, from gui?

I could not find any example how nexus plugin could be installed/removed on demand, w/o nexus restart.

--
Regards,
Igor

Andrei.

On 12-11-29 1:35 PM, Andrei Pozolotin wrote:

    *Hello, Nexus People;**
    *
    I hope you could help me with the following 3 questions:

    I need to replicate private nexus repository to amazon s3 bucket
    maven repo in real time,
    so I wrote this proof-of-concept plugin:
    https://github.com/carrot-garden/carrot-nexus/tree/master/carrot-nexus-aws-s3-publish-plugin

    *1) existing solution**

    *is there by chance a solution in place which is already solving
    this need?
    well, writing integration tests for nexus is a bit of a pain :-)
    ?

    *2) extension point**
    *
    currently I process aws s3 publish request inside
    StorageFileItemInspector extension point,
    which is "too late" - the item is already stored in nexus repo, from
    client point of view,
    but this is no good, since aws s3 publish request still could fail
    after that.

    I can not use RequestProcessor, since while here I can block client
    request,
    I still do not have access to the StorageFileItem inside the
        public boolean process(repository, request, action)
    invocation.

    EventInspector is not good for the same reason - too late.

    I am curious if there is another nexus extension point witch combines
    both RequestProcessor & StorageFileItemInspector, namely, so I can
         a) block/fail client request if aws storage request fails
         b) have access to already-downloaded StorageFileItem actual File
         c) annotate StorageFileItem on the fly in case of aws storage
    request success
    ?

    *3) **config example**
    *
    nexus configuration management seems a little too much involved - at
    first :-)

    is there a blog post or a github project which gives clear cut example
    of bare bone implementation of how to :
         a) persist xml
         b) process config rest
         c) expose config in nexus gui
         d) not to use modello for all of this (or must I use it?)
    ?

    Thank you,

    Andrei

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Tamás Cservenák]

Exactly, if _real time_ is a must, then this is exactly what you need, just like Igor says.

Implement "tee" LS and "s3" LS, and maybe a capability you can enable per-repo to "replicate", that would swap LS implementation on given repo.

Your "tee" LS would hold FS and S3 LS implementations and just delegate to them.

Maybe make "tee" LS smart (universal?), and have set it what calls to delegate? READ, WRITE, BOTH?

In your case, WRITE would be the way to go.

Other nx node could do READ, and "pull" from S3. In that case, FS would be a la "cache" for S3.

Thanks,

~t~

On Fri, Nov 30, 2012 at 4:56 AM, Igor Fedorenko <[hidden email]> wrote:

So you goal is to have the item stored both locally and in s3, right?
What is the expected behaviour if one of the stores fails for whatever
reason? Do you plan to implement some sort of two-phase commit or just
report the problem to the user but leave the stores in inconsistent state?

Have you looked at LocalRepositoryStorage? You should be able to write
new implementation that extends/wraps standard filesystem implementation
but also talks to s3, either before or after storing the item locally.

--
Regards,
Igor

On 12-11-29 1:35 PM, Andrei Pozolotin wrote:

    *Hello, Nexus People;**

    *
    I hope you could help me with the following 3 questions:

    I need to replicate private nexus repository to amazon s3 bucket
    maven repo in real time,
    so I wrote this proof-of-concept plugin:
    https://github.com/carrot-garden/carrot-nexus/tree/master/carrot-nexus-aws-s3-publish-plugin

    *1) existing solution**

    *is there by chance a solution in place which is already solving

    this need?
    well, writing integration tests for nexus is a bit of a pain :-)
    ?

    *2) extension point**

    *
    currently I process aws s3 publish request inside
    StorageFileItemInspector extension point,
    which is "too late" - the item is already stored in nexus repo, from
    client point of view,
    but this is no good, since aws s3 publish request still could fail
    after that.

    I can not use RequestProcessor, since while here I can block client
    request,
    I still do not have access to the StorageFileItem inside the
        public boolean process(repository, request, action)
    invocation.

    EventInspector is not good for the same reason - too late.

    I am curious if there is another nexus extension point witch combines
    both RequestProcessor & StorageFileItemInspector, namely, so I can
         a) block/fail client request if aws storage request fails
         b) have access to already-downloaded StorageFileItem actual File
         c) annotate StorageFileItem on the fly in case of aws storage
    request success
    ?

    *3) **config example**

    *
    nexus configuration management seems a little too much involved - at
    first :-)

    is there a blog post or a github project which gives clear cut example
    of bare bone implementation of how to :
         a) persist xml
         b) process config rest
         c) expose config in nexus gui
         d) not to use modello for all of this (or must I use it?)
    ?

    Thank you,

    Andrei

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Tamás Cservenák]

Look at capabilities. There is no way currently to dynamically load/unload a plugin at runtime in Nexus.

But adding/removing capability to/from repo would work just fine for you.

Also, by using capability, you'd get generic UI too.

On Fri, Nov 30, 2012 at 4:49 PM, Andrei Pozolotin <[hidden email]> wrote:

given static plexus/guice injection pattern, will it be possible activate/deactivate
this new custom LocalRepositoryStorage dynamically, from gui?

I could not find any example how nexus plugin could be installed/removed on demand, w/o nexus restart.

[Andrei Pozolotin]

Tamás:

I followed yours and Igor advice and have working prototype with LocalRepositoryStorage

Thank you

Andrei

-------- Original Message --------
Subject: Re: [nexus-dev] carrot-nexus-aws-s3-publish-plugin
From: Tamás Cservenák [hidden email]
To: [hidden email]
Date: Fri 30 Nov 2012 10:27:17 AM CST

Exactly, if _real time_ is a must, then this is exactly what you need, just like Igor says.

Implement "tee" LS and "s3" LS, and maybe a capability you can enable per-repo to "replicate", that would swap LS implementation on given repo.

Your "tee" LS would hold FS and S3 LS implementations and just delegate to them.

Maybe make "tee" LS smart (universal?), and have set it what calls to delegate? READ, WRITE, BOTH?

In your case, WRITE would be the way to go.

Other nx node could do READ, and "pull" from S3. In that case, FS would be a la "cache" for S3.

Thanks,

~t~

On Fri, Nov 30, 2012 at 4:56 AM, Igor Fedorenko <[hidden email]> wrote:

So you goal is to have the item stored both locally and in s3, right?
What is the expected behaviour if one of the stores fails for whatever
reason? Do you plan to implement some sort of two-phase commit or just
report the problem to the user but leave the stores in inconsistent state?

Have you looked at LocalRepositoryStorage? You should be able to write
new implementation that extends/wraps standard filesystem implementation
but also talks to s3, either before or after storing the item locally.

--
Regards,
Igor

On 12-11-29 1:35 PM, Andrei Pozolotin wrote:

    *Hello, Nexus People;**

    *
    I hope you could help me with the following 3 questions:

    I need to replicate private nexus repository to amazon s3 bucket
    maven repo in real time,
    so I wrote this proof-of-concept plugin:
    https://github.com/carrot-garden/carrot-nexus/tree/master/carrot-nexus-aws-s3-publish-plugin

    *1) existing solution**

    *is there by chance a solution in place which is already solving

    this need?
    well, writing integration tests for nexus is a bit of a pain :-)
    ?

    *2) extension point**

    *
    currently I process aws s3 publish request inside
    StorageFileItemInspector extension point,
    which is "too late" - the item is already stored in nexus repo, from
    client point of view,
    but this is no good, since aws s3 publish request still could fail
    after that.

    I can not use RequestProcessor, since while here I can block client
    request,
    I still do not have access to the StorageFileItem inside the
        public boolean process(repository, request, action)
    invocation.

    EventInspector is not good for the same reason - too late.

    I am curious if there is another nexus extension point witch combines
    both RequestProcessor & StorageFileItemInspector, namely, so I can
         a) block/fail client request if aws storage request fails
         b) have access to already-downloaded StorageFileItem actual File
         c) annotate StorageFileItem on the fly in case of aws storage
    request success
    ?

    *3) **config example**

    *
    nexus configuration management seems a little too much involved - at
    first :-)

    is there a blog post or a github project which gives clear cut example
    of bare bone implementation of how to :
         a) persist xml
         b) process config rest
         c) expose config in nexus gui
         d) not to use modello for all of this (or must I use it?)
    ?

    Thank you,

    Andrei

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Andrei Pozolotin]

Tamás:

re: capabilities - got it;

what about "is there a blog post or a github project which gives clear cut example"
do you feel like writing one? :-)

poking around nexus core plugins, looking at hand made javascript, modello, etc - just makes me scared! :-)
is there an easy way to get nexus config gui?

cheers,

Andrei

-------- Original Message --------
Subject: Re: [nexus-dev] carrot-nexus-aws-s3-publish-plugin
From: Tamás Cservenák [hidden email]
To: [hidden email]
Cc: Igor Fedorenko [hidden email]
Date: Fri 30 Nov 2012 10:28:47 AM CST

Look at capabilities. There is no way currently to dynamically load/unload a plugin at runtime in Nexus.

But adding/removing capability to/from repo would work just fine for you.

Also, by using capability, you'd get generic UI too.

On Fri, Nov 30, 2012 at 4:49 PM, Andrei Pozolotin <[hidden email]> wrote:

given static plexus/guice injection pattern, will it be possible activate/deactivate
this new custom LocalRepositoryStorage dynamically, from gui?

I could not find any example how nexus plugin could be installed/removed on demand, w/o nexus restart.

[Andrei Pozolotin]

I feel my new plug-in is "production quality" now.

https://github.com/carrot-garden/carrot-nexus/blob/master/carrot-nexus-aws-s3-publish-plugin/readme.md

anyone cares to do a code review?

thanks!

-------- Original Message --------
Subject: Re: [nexus-dev] carrot-nexus-aws-s3-publish-plugin
From: Andrei Pozolotin [hidden email]
To: [hidden email]
Cc: Tamás Cservenák [hidden email]
Date: Fri 30 Nov 2012 07:43:28 PM CST

Tamás:

I followed yours and Igor advice and have working prototype with LocalRepositoryStorage

Thank you

Andrei

-------- Original Message --------
Subject: Re: [nexus-dev] carrot-nexus-aws-s3-publish-plugin
From: Tamás Cservenák [hidden email]
To: [hidden email]
Date: Fri 30 Nov 2012 10:27:17 AM CST

Exactly, if _real time_ is a must, then this is exactly what you need, just like Igor says.

Implement "tee" LS and "s3" LS, and maybe a capability you can enable per-repo to "replicate", that would swap LS implementation on given repo.

Your "tee" LS would hold FS and S3 LS implementations and just delegate to them.

Maybe make "tee" LS smart (universal?), and have set it what calls to delegate? READ, WRITE, BOTH?

In your case, WRITE would be the way to go.

Other nx node could do READ, and "pull" from S3. In that case, FS would be a la "cache" for S3.

Thanks,

~t~

On Fri, Nov 30, 2012 at 4:56 AM, Igor Fedorenko <[hidden email]> wrote:

So you goal is to have the item stored both locally and in s3, right?
What is the expected behaviour if one of the stores fails for whatever
reason? Do you plan to implement some sort of two-phase commit or just
report the problem to the user but leave the stores in inconsistent state?

Have you looked at LocalRepositoryStorage? You should be able to write
new implementation that extends/wraps standard filesystem implementation
but also talks to s3, either before or after storing the item locally.

--
Regards,
Igor

On 12-11-29 1:35 PM, Andrei Pozolotin wrote:

    *Hello, Nexus People;**

    *
    I hope you could help me with the following 3 questions:

    I need to replicate private nexus repository to amazon s3 bucket
    maven repo in real time,
    so I wrote this proof-of-concept plugin:
    https://github.com/carrot-garden/carrot-nexus/tree/master/carrot-nexus-aws-s3-publish-plugin

    *1) existing solution**

    *is there by chance a solution in place which is already solving

    this need?
    well, writing integration tests for nexus is a bit of a pain :-)
    ?

    *2) extension point**

    *
    currently I process aws s3 publish request inside
    StorageFileItemInspector extension point,
    which is "too late" - the item is already stored in nexus repo, from
    client point of view,
    but this is no good, since aws s3 publish request still could fail
    after that.

    I can not use RequestProcessor, since while here I can block client
    request,
    I still do not have access to the StorageFileItem inside the
        public boolean process(repository, request, action)
    invocation.

    EventInspector is not good for the same reason - too late.

    I am curious if there is another nexus extension point witch combines
    both RequestProcessor & StorageFileItemInspector, namely, so I can
         a) block/fail client request if aws storage request fails
         b) have access to already-downloaded StorageFileItem actual File
         c) annotate StorageFileItem on the fly in case of aws storage
    request success
    ?

    *3) **config example**

    *
    nexus configuration management seems a little too much involved - at
    first :-)

    is there a blog post or a github project which gives clear cut example
    of bare bone implementation of how to :
         a) persist xml
         b) process config rest
         c) expose config in nexus gui
         d) not to use modello for all of this (or must I use it?)
    ?

    Thank you,

    Andrei

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

[Chrome2]

ที่ได้รับการรับรองและมีใบอนุญาตให้เปิดเว็บไซต์เกมส์การพนันออนไลน์จากรัฐบาลของประเทศกัมพูชา ดาวน์โหลด genting ซึ่งต้องผ่านช่วงเวลาที่เจ็บปวดแบบนี้เคุณั้น ผู้เล่นจึงจะได้รู้วิธีเล่นเสียติดต่อกัน และเวลาที่เดินผ่านไปจะช่วยเยียวยาตัวเอง มันจะท้าทายกับทุกความหวัง เทคนิคที่เราจะกล่าวในต่อไปนี้นั้น ไม่รับประกันนะ ว่าจะชนะมากขึ้นกว่าเดิมเต็ม 100% หรือไม่ แต่ถ้าหากนำไปประยุกต์ใช้ให้ถูกวิธี ลูกทีมท่านอื่นเล่นแล้วชนะติดต่อกันหลายตา ก็มีความเชื่อมั่นที่มากขึ้นเรื่อยๆ ก็ยิ่งส่งผลดีต่อการเล่น กับการคำนวนเงินที่ถูกต้องแม่นยำที่สุด เพื่อให้ลูกค้ามั่นใจกับรายละเอียดการใช้งานที่แม่นยำ และมีการตรวจสอบปัญหาฐานข้อมูลรายวัน อาจจะไม่สามารถร้องเรียนแบบไหนได้เลย หากคุณทำผิดกฎเอง ข้อนี้จึงเป็นข้อควรพิจารณาเป็นอย่างยิ่ง คาสิโน การเริ่มต้นเสี่ยงดวงกับการพนันออนไลน์ที่คิดว่าถนัดกันเยอะที่สุดของทาง ข่าวฟุตบอล อีกเทคนิคก็คือ หากคุณใดเห็นว่า ผู้ร่วมวางเดิมพันท่านอื่นกำลังดวงขึ้น อย่าได้รู้สึกประหม่า และหวั่นไหวใดๆ แต่ให้มองแต่ในมุมของตัวเอง

[Chrome2]

ที่ได้รับการรับรองและมีใบอนุญาตให้เปิดเว็บไซต์เกมส์การพนันออนไลน์จากรัฐบาลของประเทศกัมพูชา สำหรับรูปแบบผู้เล่นทดลองนี้ถือว่าให้โอกาสผู้เล่นได้ทำความรู้จักระบบการเล่น เกมส์บาคาร่า ก็ยังพอมีรายได้สำรองเอาไว้ใช้จ่ายในคราที่จำเป็นจริงๆ จะว่าไปแล้วการเกิดทางเลือกอย่างบ่อนเสมือนจริงที่ให้บริการเกมส์ goldclub slot ด้วยยุคที่อินเตอร์เน็ตกำลังครองเมือง สิ่งที่ขวางกันกั้นระหว่างคุณกับคาสิโนออนไลน์คือแค่การคลิกเบาๆ เคุณั้น ลองเปรียบเทียบกับเส้นทางอันแสนยาวไกลที่คุณต้องบากบั่นไปเพื่อเล่น ทุกคนก็จะได้รับเงินเพิ่มมากขึ้น การมีรายได้ที่ดีเลือกเสี่ยงดวงที่เว็บ ข่าวฟุตบอล จะมีให้คำแนะนำกับทุกคนได้เล่นการพนันเหล่านี้กันอย่างถั่วถึง sbobet เป็นเครื่องมือสื่อสารที่โดยทันทีและทันสมัย ที่สามารถทำได้หลายอย่างภายในมือถือเมื่อเครื่องเดียวเคุณั้น นอกจากจะใช้โทรสื่อสาร  แชท ส่งข้อความ อัพเดทเฟสบุ๊ก ทวิตเตอร์แล้ว มือถือสมัยนี้ยังสามารถเล่น