On Maven 4.0.0 and beyond

classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

On Maven 4.0.0 and beyond

stephenconnolly
Over the break I *may* get some time to work on the tooling improvements
that Robert, Hervé and I identified for the jump to 4.0.0 (namely a better
way to ensure api binary compatibility for the APIs that core exposes to
plugins)

If I do get a chance to work on this, it’ll be in a branch until we think
it’s “good enough”.

The idea is to enable large scale lambdafication and other Java 8
improvements in core without breaking plugins.

If we have good reliable tooling, then we hope to be able to encourage
contributions (as you don’t need a CLA for an “obvious” ~10-15 line patch -
intent to submit is sufficient) and get the code base more energised.

-Stephen

P.S. the existing api checker tools do not meet our needs as they check too
much, we only want those classes exposed to plugins and extensions. I think
animal sniffer might be closest to what we need - as it allows defining the
signature scope - but it needs a different execution mode to do what we
need: check that the signature is still valid for a specific classpath.

P.P.S. I also might need to think about Java modules... eg would it make
sense to use modules for Maven 5.0 as an alternative to classworlds?
--
Sent from my phone
Reply | Threaded
Open this post in threaded view
|

Re: On Maven 4.0.0 and beyond

stephenconnolly
On Sun 23 Dec 2018 at 11:32, Michael Osipov <[hidden email]> wrote:

> Am 2018-12-23 um 11:14 schrieb Stephen Connolly:
> > Over the break I *may* get some time to work on the tooling improvements
> > that Robert, Hervé and I identified for the jump to 4.0.0 (namely a
> better
> > way to ensure api binary compatibility for the APIs that core exposes to
> > plugins)
> >
> > If I do get a chance to work on this, it’ll be in a branch until we think
> > it’s “good enough”.
> >
> > The idea is to enable large scale lambdafication and other Java 8
> > improvements in core without breaking plugins.
>
> I'd rather see a big bug/PR sweep than lambafication which does not give
> a huge benefit for the users. What else I'd like to see is edge cases
> covered Christian Schulte started to work on. We can break things this
> time and clean them up.


We need to get people actually involved in the code. Lambdafication and
other modernisation refactoring enable people to get their hands dirty
without committing too much. Asking potential new blood to look at the
harder things will not get us new blood.

Actual committers should be looking at the other changes... though I don’t
believe we can change the pom until 5.0.0 which is why the scope of 4.0.0
is likely:

* modernise the codebase
* Java 8 baseline
* perhaps relocate aether to org.apache.maven
* other behaviour changes that do not affect the pom
* nicer failure message when presented with a newer modelVersion
* native consumer pom support... perhaps also PDT publishing


>
> Michael
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
> --
Sent from my phone
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Karl Heinz Marbaise-3
In reply to this post by stephenconnolly
Hi Mickael,

On 29/12/18 09:57, Mickael Istria wrote:

> On Mon, Dec 24, 2018 at 5:21 PM Karl Heinz Marbaise <[hidden email]>
> wrote:
>
>> Hi Mickael,
>>
>
> Hi,
>
> I have to summarize this simply.
>
> I was confused about how automated tests are running or not. If I look at
> https://github.com/apache/maven/pull/194 , I do not see any build/test
> report on the issue. I imagine this requires someone to manually trigger
> the tests, doesn't it?
> If it is so, then it could be one step to remove on the review process, and
> the automated build feedback would allow contributors to fix their PRs by
> themselves before wasting time of a reviewer.

I have started to discuss this with ASF Infra team ...

Lets see if we find a solution...I see currently the issue in the amount
of repos we have...but lets see...

Kind regards
Karl Heinz Marb aise

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Mickael Istria-2
In reply to this post by stephenconnolly
On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <[hidden email]>
wrote:

> But in both cases, currently, there is no automatic GitHub PR integration:
> Maven committers need to create a branch in the official repository to
> benefit
> from ASF Jenkins build
>

Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be used
to trigger builds. That sucks...

Any idea how we could have GitHub PR reviews without this branch creation
> by
> committers, be it at ASF or somewhere else?
>

Using TravisCI could be a solution.
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Enrico Olivelli
Il sab 29 dic 2018, 12:37 Mickael Istria <[hidden email]> ha scritto:

> On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <[hidden email]>
> wrote:
>
> > But in both cases, currently, there is no automatic GitHub PR
> integration:
> > Maven committers need to create a branch in the official repository to
> > benefit
> > from ASF Jenkins build
> >
>
> Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be used
> to trigger builds. That sucks...
>

Maven migrated to gitbox so actually github is an official repo for Maven.
I see the same setup in Zookeeper and Bookkeeper and github pr plugin works
like a charm (and I partecipated in setting it up)

Enrico


> Any idea how we could have GitHub PR reviews without this branch creation
> > by
> > committers, be it at ASF or somewhere else?
> >
>
> Using TravisCI could be a solution.
>
--


-- Enrico Olivelli
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Hervé BOUTEMY
In reply to this post by Mickael Istria-2
Le samedi 29 décembre 2018, 12:37:36 CET Mickael Istria a écrit :

> On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <[hidden email]>
>
> wrote:
> > But in both cases, currently, there is no automatic GitHub PR integration:
> > Maven committers need to create a branch in the official repository to
> > benefit
> > from ASF Jenkins build
>
> Ah ok, I wasn't aware the GitHub repo was "unofficial" and couldn't be used
> to trigger builds. That sucks...
GitHub repo is not unofficial: with GitBox, it's as official as ASF-hosted Git
repo
it's GitHub PRs that are not real branches of the Git repo, then the
triggerring has to be different than normal Git branches commits

>
> Any idea how we could have GitHub PR reviews without this branch creation
>
> > by
> > committers, be it at ASF or somewhere else?
>
> Using TravisCI could be a solution.





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Enrico Olivelli
In reply to this post by Enrico Olivelli
Il sab 29 dic 2018, 17:25 Stephen Connolly <[hidden email]>
ha scritto:

> On Sat 29 Dec 2018 at 16:20, Stephen Connolly <
> [hidden email]> wrote:
>
> >
> >
> > On Sat 29 Dec 2018 at 15:18, Enrico Olivelli <[hidden email]>
> wrote:
> >
> >> Il sab 29 dic 2018, 15:17 Stephen Connolly <
> >> [hidden email]>
> >> ha scritto:
> >>
> >> > There is a security issue with building PRs automatically.
> >> >
> >> > I can see about adding PR discovery to the existing ASF gitbox plugin,
> >> but
> >> > we’d need committers to ok the build and have reviewed the code as the
> >> PR
> >> > could contain attacks to be run from ASF hardware... which is why we
> >> don’t
> >> > build PRs at present.
> >> >
> >>
> >> Now I have to review and then push to ASF repo and I have to tell to the
> >> contributor that I will make CI start.
> >> IMHO a good tradeoff is:
> >> - a committer adds a 'test this please' comment, or '@asfbot test this
> >> please' and then a CI job start
> >> - this job updates the status line of the PR, with a link to the logs
> and
> >> the status of the job
> >>
> >> How does it sounds to you?
> >
> >
> > Like it’ll burn through the GitHub api rate limit like crazy.
>

I did not think we have 100 repos

> >
> > The committer goes to Jenkins and clicks the build button on the PR job
> > (which is sitting there ready and waiting), done.
> >
>
> Oh and before I forget, clicking build now I’m Jenkins will update the CI
> status in GitHub for the PR to say in progress and then provide the result
> when available.
>
>
> > Searching through comments on PRs to find commits with a magic comment
> > string that haven’t been built by Jenkins is certainly what would be
> > lovely... but right now it would burn the GitHub rate limit (which is
> 5,000
> > requests per hour across the whole ASF) right through.
> >
>
> To be clear, the hard part is efficiently finding “missed” comments and
> associating with the commit hash they belong to. We don’t want an approval
> to allow attack via an update pushed to the PR. So you have to walk all the
> comments and associate with the commit hash they applied to... gets tricky.
>

Yep, I understand, you are right.

Another option is to have a script to launch:

import-pr maven-assembly-plugin #567

Then the script + Jenkins:
- bind the pr to a JIRA by scanning git log
- push to ASF (changing 'committer') with a standard name (JIRA id)
- start a job
- add a github status line with a link to the logs
- bonus: the job will change the status line with green/red light

I already have such kind of script in my company (but for
bitbucket/JIRA/Jenkins and not for such a complex system like Maven CI),
but the hard part is the job which has to write the status line

Enrico


> We could maybe hijack approval state... but that allows merging by the
> author.
>

This part is not clear to me. Only 'commiters' can push to the repo.



>
> >
> >
> >>
> >> Enrico
> >>
> >>
> >> > Other projects at ASF probably missed this point in the video series
> >> > chronicling the development of the plugin...
> >> >
> >> > On Sat 29 Dec 2018 at 13:10, Enrico Olivelli <[hidden email]>
> >> wrote:
> >> >
> >> > > Hervè,
> >> > > This is the plugin
> >> > >
> >> > >
> >> >
> >>
> https://wiki.jenkins.io/display/JENKINS/GitHub+Branch+Source+Plugin?desktop=true&macroName=unmigrated-inline-wiki-markup
> >> > >
> >> > > I see our "maven-box" is using some special "Scan Apache Hosted Git
> >> > > Folder Triggers" source
> >> > > (https://builds.apache.org/job/maven-box/configure)
> >> > > In the Jenkins of my company in a "Multibranch Pipeline" I have a
> >> > > "Branch Sources" box with a "GitHub" option which lets me trigger
> >> > > builds by using PRs
> >> > >
> >> > >
> >> > > Enrico
> >> > >
> >> > > Il giorno sab 29 dic 2018 alle ore 13:43 Hervé BOUTEMY
> >> > > <[hidden email]> ha scritto:
> >> > > >
> >> > > > Le samedi 29 décembre 2018, 12:40:20 CET Enrico Olivelli a écrit :
> >> > > > > Il sab 29 dic 2018, 12:37 Mickael Istria <[hidden email]>
> ha
> >> > > scritto:
> >> > > > > > On Sat, Dec 29, 2018 at 12:01 PM Hervé BOUTEMY <
> >> > > [hidden email]>
> >> > > > > >
> >> > > > > > wrote:
> >> > > > > > > But in both cases, currently, there is no automatic GitHub
> PR
> >> > > > > >
> >> > > > > > integration:
> >> > > > > > > Maven committers need to create a branch in the official
> >> > > repository to
> >> > > > > > > benefit
> >> > > > > > > from ASF Jenkins build
> >> > > > > >
> >> > > > > > Ah ok, I wasn't aware the GitHub repo was "unofficial" and
> >> couldn't
> >> > > be
> >> > > > > > used
> >> > > > > > to trigger builds. That sucks...
> >> > > > >
> >> > > > > Maven migrated to gitbox so actually github is an official repo
> >> for
> >> > > Maven.
> >> > > > > I see the same setup in Zookeeper and Bookkeeper and github pr
> >> plugin
> >> > > works
> >> > > > > like a charm (and I partecipated in setting it up)
> >> > > > oh great, that would be nice to have the same setup for Maven
> repos!
> >> > > >
> >> > > > >
> >> > > > > Enrico
> >> > > > >
> >> > > > > > Any idea how we could have GitHub PR reviews without this
> branch
> >> > > creation
> >> > > > > >
> >> > > > > > > by
> >> > > > > > > committers, be it at ASF or somewhere else?
> >> > > > > >
> >> > > > > > Using TravisCI could be a solution.
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> > > >
> >> ---------------------------------------------------------------------
> >> > > > To unsubscribe, e-mail: [hidden email]
> >> > > > For additional commands, e-mail: [hidden email]
> >> > > >
> >> > >
> >> > >
> ---------------------------------------------------------------------
> >> > > To unsubscribe, e-mail: [hidden email]
> >> > > For additional commands, e-mail: [hidden email]
> >> > >
> >> > > --
> >> > Sent from my phone
> >> >
> >> --
> >>
> >>
> >> -- Enrico Olivelli
> >>
> > --
> > Sent from my phone
> >
> --
> Sent from my phone
>
--


-- Enrico Olivelli
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Karl Heinz Marbaise-3
Hi,

On 29/12/18 18:16, Enrico Olivelli wrote:

> Il sab 29 dic 2018, 17:25 Stephen Connolly <[hidden email]>
> ha scritto:
>
>> On Sat 29 Dec 2018 at 16:20, Stephen Connolly <
>> [hidden email]> wrote:
>>
>>>
>>>
>>> On Sat 29 Dec 2018 at 15:18, Enrico Olivelli <[hidden email]>
>> wrote:
>>>
>>>> Il sab 29 dic 2018, 15:17 Stephen Connolly <
>>>> [hidden email]>
>>>> ha scritto:
>>>>
>>>>> There is a security issue with building PRs automatically.
>>>>>
>>>>> I can see about adding PR discovery to the existing ASF gitbox plugin,
>>>> but
>>>>> we’d need committers to ok the build and have reviewed the code as the
>>>> PR
>>>>> could contain attacks to be run from ASF hardware... which is why we
>>>> don’t
>>>>> build PRs at present.
>>>>>
>>>>
>>>> Now I have to review and then push to ASF repo and I have to tell to the
>>>> contributor that I will make CI start.
>>>> IMHO a good tradeoff is:
>>>> - a committer adds a 'test this please' comment, or '@asfbot test this
>>>> please' and then a CI job start
>>>> - this job updates the status line of the PR, with a link to the logs
>> and
>>>> the status of the job
>>>>
>>>> How does it sounds to you?
>>>
>>>
>>> Like it’ll burn through the GitHub api rate limit like crazy.
>>
>
> I did not think we have 100 repos

I wouldn't be that sure:

https://builds.apache.org/view/M-R/view/Maven/job/maven-box/
(96 at the moment)..

or https://gitbox.apache.org/repos/asf

search for "Apache Maven"...

Apart from that the GitHub API rate limit is for the whole ASF
orga...not only for our project...

Kind regards
Karl Heinz Marbaise

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Vladimir Sitnikov
In reply to this post by Enrico Olivelli
Stephen> Nah. Once committers have approved the PR then the PR can be
merged by the
Stephen> creator of the PR even if not a committer... at least that’s
the default
Stephen> way GitHub PRs work

By default one needs push rights on the repository to merge PRs.
"Approval" changes nothing. "PR approval" does not differ from any
other comment.
In other words, non-committers can't merge PRs.

Vladimir

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

stephenconnolly
On Sat 29 Dec 2018 at 18:19, Stephen Connolly <
[hidden email]> wrote:

>
>
> On Sat 29 Dec 2018 at 18:06, Vladimir Sitnikov <
> [hidden email]> wrote:
>
>> Stephen> Nah. Once committers have approved the PR then the PR can be
>> merged by the
>> Stephen> creator of the PR even if not a committer... at least that’s
>> the default
>> Stephen> way GitHub PRs work
>>
>> By default one needs push rights on the repository to merge PRs.
>> "Approval" changes nothing. "PR approval" does not differ from any
>> other comment.
>> In other words, non-committers can't merge PRs.
>
>
> Well on other GitHub orgs i’ve seen PR author has Merge button once PR is
> approved by someone with push rights to the repo... until they add a commit
> or the merge result changes
>

Gitbox may be configuring synchronised repos differently, but for interop
with the rest of GitHub we shouldn’t assume approval of a PR by committers
will not enable self-merging


>>
>> Vladimir
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>>
>> --
> Sent from my phone
>
--
Sent from my phone
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Mickael Istria-2
I'mglad to see that PR build/merge discussed, it seems to have a good
potential value for many in simplifying it.

FWIW, at Eclipse Foundation, similar questions were faced and risks
identified. The result is that at the moment, all PRs from anyone can be
built on Eclipse infra without much visible restriction. The Foundation
might have extra guards under the hood but it's not something we developers
need to care about.
While this is probably unsafe, it's so far so good. We weren't warned by
the Foundation about malicious usage of this permissive access to build
machines.
AFAIK, there is no issue from developer POV about GitHub API rates limit.
I suggest the Apache infra team gets in touch with Eclipse Foundation one
to sort out whether similar configurations could be implemented in a
safe-enough way.

The GitHub PR builder plugin already has support for whitelisting users and
giving them ability to trigger a build from a non-whitelidsted contributor
with a single «test PR» comment or similar.
Then build progress and report are reported as expected.
And, again, TravisCI also does the same in a decent way, with the benefit
of worrying less about underlying infra and security, and only drawback of
being discoupled from a specific infra (is it really a drawback?)

Cheers,


--
Mickael Istria
Eclipse IDE <https://www.eclipse.org/downloads/eclipse-packages/>
developer, for Red Hat Developers <https://developers.redhat.com/>
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Vladimir Sitnikov
In reply to this post by stephenconnolly
Karl Heinz Marbaise > The user community is very big but unfortunately the
people who are
Karl Heinz Marbaise > willing to help is not that big...

Can I (ab)use the thread a bit?

One of the issues I run into with Maven is "it takes ages for repeated
build when nothing has changed".
Of course there are details, yet the final perception is like that.

I know Apache JMeter team sticks with Ant for a mere reason of fast builds
(even though it is way harder to configure in IDE)

Once upon a time I was bitten hard enough by Maven which always rebuilds
everything even for repeated builds, so I investigated the reason and
created https://issues.apache.org/jira/browse/CALCITE-484
The basic case is "jar is rebuilt due to resource was somehow modified".
Of course there are multiple offenders, however two cases come from
maven-resources-plugin and maven-remote-resources-plugin.

I've created a PR (in December 2014) to fix maven-remote-resources-plugin:
https://github.com/apache/maven-plugins/pull/40 (Avoid overwrite of the
destination file if the produced contents is the same), and it took just
FOUR years to release the new version of the plugin in October 2018.

The second offender is maven-filtering (
https://issues.apache.org/jira/browse/MSHARED-394 )
In a brief, maven-filtering rewrites the file not matter the contents. For
instance, when Maven copies resources, it compares timestamps, so it can
skip copy if the file is good enough.
However maven-filtering always writes the destination file, thus it
triggers jar rebuild, etc.

Frankly speaking, I'm proud of a trick in
https://github.com/apache/maven-plugins/pull/40 which keeps contents in
memory, and if the buffer overflows, then it writes the contents to the
destination file. In other words, no temporary files are required, and it
can skip file overwrites when the produced contents is the same.

I think similar should be done to DefaultMavenFileFilter, however Kristian
suggested he wanted to look on his own:

https://issues.apache.org/jira/browse/MSHARED-394?focusedCommentId=14449864&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14449864

>Kristian Rosenvold added a comment - 16/Dec/2014 12:19
>Haven given this issue some thought I believe I have come up with a
radically better solution than what is suggested here.
>There is no point in working on the changes suggested here,
>at least not until I give up my other plan.
>Since I am currently working on a slightly different task, it will be a
couple of weeks before I'll make any activity on this issue.


Frankly speaking, I'm very puzzled.
The net result is: my first Maven contribution (MRRESOURCES-91) took 4
years to release, and very similar one was rejected with "There is no point
in working on the changes suggested here" comment.
That is quite sad to be honest.

Vladimir
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Enrico Olivelli
In reply to this post by Mickael Istria-2
Il dom 30 dic 2018, 00:53 Mickael Istria <[hidden email]> ha scritto:

> On Saturday, December 29, 2018, Stephen Connolly <
> [hidden email]> wrote:
> >
> > I think we could easily use TravisCI or one of the cloud CI vendors to
> > perform trial builds of PRs on GitHub.
>
>
> Good. So at this point, it's only a matter of writing a .travis.yaml file
> in the repo, or is there anything else missing?
>

Probably Travis won't be able to run our ITs, at least the free version

Enrico

>
>
> > What is of importance to the ASF is
> > that the build of ASF hosted code are on ASF hosted hardware... until the
> > PR is merged it is not ASF hosted code, it exists only on GitHub.
> >
>
> From contributor and user POV, I think where the software is built and
> where the code is hosted isn't important. So no issue if it's on ASF
> Jenkins and hardware, but for PR trials, it seems like a constraint it'd be
> sane to get rid of.
>
>
> --
> Mickael Istria
> Eclipse IDE <https://www.eclipse.org/downloads/eclipse-packages/>
> developer, for Red Hat Developers <https://developers.redhat.com/>
>
--


-- Enrico Olivelli
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Hervé BOUTEMY
In reply to this post by stephenconnolly
yes, on our 100 Git repos [1], 90% should be easy to run on any CI with
minimum configuration and resources

then there is Maven core and complex plugins (like Surefire) which are not the
same story

if someone wants to test other toolings to show some useful feature we don't
get in our current ASF Jenkins configuration [2], just start with some
examples of the base 90% and make a demo of something that would be useful to
add to our configuration

Regards,

Hervé

[1] https://maven.apache.org/scm.html#Maven_Sources_Overview

[2] https://github.com/apache/maven-jenkins-lib

Le mercredi 2 janvier 2019, 15:38:12 CET Enrico Olivelli a écrit :

> Il giorno mer 2 gen 2019 alle ore 15:34 Vladimir Sitnikov
>
> <[hidden email]> ha scritto:
> > Enrico> For instance maven-surefire
> > integration tests take 2 hours.
> > In general I see Travis (free edition) does not offer many resources.
> >
> > Each Travis job is limited by 50 minutes, however one can split tests into
> > multiple jobs, so it will run fine.
>
> I guess it won't be easy.
> Maven integration testing is very complex, it's Maven launching Maven
> itself. We will need to improve maven-invoker plugin, I don't know if there
> is support for something like JUnit categories.
> We can run only "unit tests" and this will be a minimal "feedback" for
> the contributor.
> For simpler plugins, like "checkstyle", I think Travis would be a good
> choice, at least I would give it a try
>
> Enrico
>
> > Vladimir
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Enrico Olivelli
Il gio 3 gen 2019, 17:38 Mickael Istria <[hidden email]> ha scritto:

> Hi,
>
> I think this discussion is diverging into "trying TravisCI for some
> plugins" and is loosing focus on the initial question of how to improve the
> build+test flow to get faster feedback as a contributor or as a reviewer.
> Can the GitHub PR builder plugin be enabled on Maven Core ? Committers
> would be allowed to trigger a test with a single comment once they checked
> it doesn't cause a security flaw.
>

It turns out that it is not simple as it could seem because we have custom
Jenkins plugins to scan all the repos and have a common configuration, so
in order to achieve such goal we need to enhance that plugin, please
Stephen correct me if I am wrong.
Travis jumped on this train because it is easy to enable and it is very
widespread, and it leaves security problems out of ASF infra.
But a check with Travis won't ever cover all jobs we are actually starting
per-branch.
It can be a good compromise to start, but if we have resources to improve
current integration this will be the best choice for the mid term.

Enrico

>
> Cheers,
>
--


-- Enrico Olivelli
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Tibor Digana
I already lost the point of this thread.
Still the disk space problem on Windows executors in ASF Jenkins?
If it's this issue, then why the workspace is not investigated directly on
the system with remote access?
Jenkins is very good tool and I do not want to use Travis but Travis is one
step after finding the root cause.
T

On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
[hidden email]> wrote:

> On Thu 3 Jan 2019 at 18:03, Enrico Olivelli <[hidden email]> wrote:
>
> > Il gio 3 gen 2019, 17:38 Mickael Istria <[hidden email]> ha scritto:
> >
> > > Hi,
> > >
> > > I think this discussion is diverging into "trying TravisCI for some
> > > plugins" and is loosing focus on the initial question of how to improve
> > the
> > > build+test flow to get faster feedback as a contributor or as a
> reviewer.
>
>
> Travis will get the build+test flow faster for non-committers.
>
>
> > > Can the GitHub PR builder plugin be enabled on Maven Core ?
>
>
> No, it’s not compatible with how we build, as currently we only build from
> gitbox not from GitHub so there is no link for Jenkins to see
>
>
> Committers
> > > would be allowed to trigger a test with a single comment once they
> > checked
> > > it doesn't cause a security flaw.
> > >
> >
> > It turns out that it is not simple as it could seem because we have
> custom
> > Jenkins plugins to scan all the repos and have a common configuration, so
> > in order to achieve such goal we need to enhance that plugin, please
> > Stephen correct me if I am wrong.
>
>
> Yep I need to enhance the plugin a little... just trying to clear stuff off
> my plate
>
>
> > Travis jumped on this train because it is easy to enable and it is very
> > widespread, and it leaves security problems out of ASF infra.
> > But a check with Travis won't ever cover all jobs we are actually
> starting
> > per-branch.
> > It can be a good compromise to start, but if we have resources to improve
> > current integration this will be the best choice for the mid term.
> >
> > Enrico
> >
> > >
> > > Cheers,
> > >
> > --
> >
> >
> > -- Enrico Olivelli
> >
> --
> Sent from my phone
>
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

stephenconnolly
On Thu 3 Jan 2019 at 20:52, Tibor Digana <[hidden email]> wrote:

> I already lost the point of this thread.
> Still the disk space problem on Windows executors in ASF Jenkins?
> If it's this issue, then why the workspace is not investigated directly on
> the system with remote access?
> Jenkins is very good tool and I do not want to use Travis but Travis is one
> step after finding the root cause.


We’re not going to be able to use Travis for Surefire (unless we have a
“fast” suite of tests)

That doesn’t mean we cannot use Travis for the 90 other repos we have in
order to get test results of PRs from non-committers.


> T
>
> On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> [hidden email]> wrote:
>
> > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli <[hidden email]> wrote:
> >
> > > Il gio 3 gen 2019, 17:38 Mickael Istria <[hidden email]> ha
> scritto:
> > >
> > > > Hi,
> > > >
> > > > I think this discussion is diverging into "trying TravisCI for some
> > > > plugins" and is loosing focus on the initial question of how to
> improve
> > > the
> > > > build+test flow to get faster feedback as a contributor or as a
> > reviewer.
> >
> >
> > Travis will get the build+test flow faster for non-committers.
> >
> >
> > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> >
> >
> > No, it’s not compatible with how we build, as currently we only build
> from
> > gitbox not from GitHub so there is no link for Jenkins to see
> >
> >
> > Committers
> > > > would be allowed to trigger a test with a single comment once they
> > > checked
> > > > it doesn't cause a security flaw.
> > > >
> > >
> > > It turns out that it is not simple as it could seem because we have
> > custom
> > > Jenkins plugins to scan all the repos and have a common configuration,
> so
> > > in order to achieve such goal we need to enhance that plugin, please
> > > Stephen correct me if I am wrong.
> >
> >
> > Yep I need to enhance the plugin a little... just trying to clear stuff
> off
> > my plate
> >
> >
> > > Travis jumped on this train because it is easy to enable and it is very
> > > widespread, and it leaves security problems out of ASF infra.
> > > But a check with Travis won't ever cover all jobs we are actually
> > starting
> > > per-branch.
> > > It can be a good compromise to start, but if we have resources to
> improve
> > > current integration this will be the best choice for the mid term.
> > >
> > > Enrico
> > >
> > > >
> > > > Cheers,
> > > >
> > > --
> > >
> > >
> > > -- Enrico Olivelli
> > >
> > --
> > Sent from my phone
> >
>
--
Sent from my phone
Reply | Threaded
Open this post in threaded view
|

Re: New Committers - community

Enrico Olivelli
Il gio 3 gen 2019, 22:37 Tibor Digana <[hidden email]> ha scritto:

> Why the build cannot be triggered on GitHub or PR?
> It's only a URL.
>

You are right, github gives special refs which represents the branch of the
forked repo, like pull/123/head.
The  problem is that our CI is configured for gitbox and not for github, so
we don't have those special refs

Enrico

If you see the frequency of PRs in Surefire, 1 or 2 PRs/month, this should

> not be a problem since the Windows build takes 1.5 hour and Linux 1 hour.
> Taking JDK 7 and 11, and one Maven version (3.5) is usually enough on Git
> branches and should be fine for PRs as well.
>
> On Thu, Jan 3, 2019 at 10:19 PM Stephen Connolly <
> [hidden email]> wrote:
>
> > On Thu 3 Jan 2019 at 20:52, Tibor Digana <[hidden email]> wrote:
> >
> > > I already lost the point of this thread.
> > > Still the disk space problem on Windows executors in ASF Jenkins?
> > > If it's this issue, then why the workspace is not investigated directly
> > on
> > > the system with remote access?
> > > Jenkins is very good tool and I do not want to use Travis but Travis is
> > one
> > > step after finding the root cause.
> >
> >
> > We’re not going to be able to use Travis for Surefire (unless we have a
> > “fast” suite of tests)
> >
> > That doesn’t mean we cannot use Travis for the 90 other repos we have in
> > order to get test results of PRs from non-committers.
> >
> >
> > > T
> > >
> > > On Thu, Jan 3, 2019 at 8:39 PM Stephen Connolly <
> > > [hidden email]> wrote:
> > >
> > > > On Thu 3 Jan 2019 at 18:03, Enrico Olivelli <[hidden email]>
> > wrote:
> > > >
> > > > > Il gio 3 gen 2019, 17:38 Mickael Istria <[hidden email]> ha
> > > scritto:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I think this discussion is diverging into "trying TravisCI for
> some
> > > > > > plugins" and is loosing focus on the initial question of how to
> > > improve
> > > > > the
> > > > > > build+test flow to get faster feedback as a contributor or as a
> > > > reviewer.
> > > >
> > > >
> > > > Travis will get the build+test flow faster for non-committers.
> > > >
> > > >
> > > > > > Can the GitHub PR builder plugin be enabled on Maven Core ?
> > > >
> > > >
> > > > No, it’s not compatible with how we build, as currently we only build
> > > from
> > > > gitbox not from GitHub so there is no link for Jenkins to see
> > > >
> > > >
> > > > Committers
> > > > > > would be allowed to trigger a test with a single comment once
> they
> > > > > checked
> > > > > > it doesn't cause a security flaw.
> > > > > >
> > > > >
> > > > > It turns out that it is not simple as it could seem because we have
> > > > custom
> > > > > Jenkins plugins to scan all the repos and have a common
> > configuration,
> > > so
> > > > > in order to achieve such goal we need to enhance that plugin,
> please
> > > > > Stephen correct me if I am wrong.
> > > >
> > > >
> > > > Yep I need to enhance the plugin a little... just trying to clear
> stuff
> > > off
> > > > my plate
> > > >
> > > >
> > > > > Travis jumped on this train because it is easy to enable and it is
> > very
> > > > > widespread, and it leaves security problems out of ASF infra.
> > > > > But a check with Travis won't ever cover all jobs we are actually
> > > > starting
> > > > > per-branch.
> > > > > It can be a good compromise to start, but if we have resources to
> > > improve
> > > > > current integration this will be the best choice for the mid term.
> > > > >
> > > > > Enrico
> > > > >
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > --
> > > > >
> > > > >
> > > > > -- Enrico Olivelli
> > > > >
> > > > --
> > > > Sent from my phone
> > > >
> > >
> > --
> > Sent from my phone
> >
>
--


-- Enrico Olivelli