Maven builds out of control

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Maven builds out of control

gmcdonald
Hi Maven folks,

Over 800 builds in the Jenkins queue right now, most of them Maven - and
its been above 500 , mostly Maven for over 12 hours now - Please take a
look and see what is going on ASAP, or I might have to cancel all Maven
jobs to allow other projects to proceed

--

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Reply | Threaded
Open this post in threaded view
|

Re: Maven builds out of control

rfscholte
Looking at it.
It seems like most are waiting for a Windows executor, but these are all taken too.
Need to investigate a bit more, but it looks like all branches are triggered for some reason.
That's weird, because we didn't touch are shared Jenkins scripts

Robert
On 21-7-2020 18:22:12, Gavin McDonald <[hidden email]> wrote:
Hi Maven folks,

Over 800 builds in the Jenkins queue right now, most of them Maven - and
its been above 500 , mostly Maven for over 12 hours now - Please take a
look and see what is going on ASAP, or I might have to cancel all Maven
jobs to allow other projects to proceed

--

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team
Reply | Threaded
Open this post in threaded view
|

Re: Maven builds out of control

gmcdonald
Hi!

On Wed, Jul 22, 2020 at 10:29 AM Robert Scholte <[hidden email]>
wrote:

>
>
> *Robert Scholte:*
> That's very unlikely. I expect that Jenkins itself got an update of that
> plugin and that our MultiBranch setup picked up that change.
>

hm, we don't normally do out of band plugin upgrades unless its' necessary
, I'll investigate this end, I have added a plugin or two
recently, and this may have come in as a dependant upgrade, but fairly sure
I did not do any plugins on the 21st.

Gav...


>
>
>
> >
> >
> > This caused new configuration for *all* our branches and triggered new
> > builds.
> > The Ubuntu builds were often ready, but a lot were still waiting for
> > Windows Executors.
> >
>
> Ack
>
>
> >
> > Most of these builds are quite short so under normal circumstances there
> > shouldn't be any issues.
> > However, I already noticed we're having way too much branches, so either
> > we'll clean these up or remove/disable its JenkinsFile, so it won't be
> > triggered anymore.
> >
>
> That would be awesome, things are looking good at the moment, thanks for
> looking into it
>
> Gav...
>
>
> >
> > thanks,
> > Robert
> >
> > On 21-7-2020 19:49:13, Robert Scholte wrote:
> > Looking at it.
> > It seems like most are waiting for a Windows executor, but these are all
> > taken too.
> > Need to investigate a bit more, but it looks like all branches are
> > triggered for some reason.
> > That's weird, because we didn't touch are shared Jenkins scripts
> >
> > Robert
> >
> > On 21-7-2020 18:22:12, Gavin McDonald wrote:
> > Hi Maven folks,
> >
> > Over 800 builds in the Jenkins queue right now, most of them Maven - and
> > its been above 500 , mostly Maven for over 12 hours now - Please take a
> > look and see what is going on ASAP, or I might have to cancel all Maven
> > jobs to allow other projects to proceed
> >
> > --
> >
> > *Gavin McDonald*
> > Systems Administrator
> > ASF Infrastructure Team
> >
> >
>
> --
>
> *Gavin McDonald*
> Systems Administrator
> ASF Infrastructure Team
>
>

--

*Gavin McDonald*
Systems Administrator
ASF Infrastructure Team