Re: Maven Runtime Metrics System - MNG-6899

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Maven Runtime Metrics System - MNG-6899

Enrico Olivelli
Il giorno dom 10 mag 2020 alle ore 11:53 Robert Scholte <
[hidden email]> ha scritto:

> Maybe I'm still missing a detail, but this is what I had in mind:
> Maven already fires events.
>

Metrics are for a distinct use case.
Metrics are like Logs, the developer instruments its own code to have
important information available during the execution.
For instance you want to see the evolution of a cache, see how much it is
useful,
or you can instrument the HTTP client inside wagon to see how well the
network is performing.
Usually they are for fine grained instrumentation of hot points in code.
For instance you do not want to fire an event every time an entry of a
cache is evicted and expect some code to intercept that event.

Metrics are not extension points like current "events" mechanism.
The MetricsProvider will be an extension, in a way that it is separate from
Maven Core, it only provides the way to gather the information,
usually in form of time series and make it available to the tools (like
Dashboards, reports...).

Metrics will be useful for:
- Maven developers (core and plugins): see how well code behaves and where
it is better to put enhancements/optimizations
- Maven users: some of the metrics could be useful for users, especially
about the system in which Maven is executing (network, disk speed, usage of
memory during peaks....)
- Automatics tools, like Jenkins, to display information about the
execution of long running builds


Enrico





> The extension listens to these events and can do its metrics.
> I see no need for a hard coupling between these two.
>
> Robert
>
> On 10-5-2020 10:28:42, Enrico Olivelli <[hidden email]> wrote:
> Il Dom 10 Mag 2020, 09:21 Romain Manni-Bucau ha
> scritto:
>
> > Le dim. 10 mai 2020 à 07:43, Enrico Olivelli a
> > écrit :
> >
> > > Robert
> > > Maybe I was misunderstood.
> > > We need a new repo only for the Metrics API.
> > >
> > > Maven core (3.7?) provides the noop implementation.
> > >
> > > Implementations of MetricsProvider will be extensions. I feel it is
> > better
> > > that we don't deal with implementations as a first step.
> > > I will leave a couple of implementations in my personal repo.
> > >
> >
> > Think we need at least a trivial memory impl with maybe a "log at
> > shutdown" feature otherwise it will not bring anything to end users
> >
>
>
> Yep
>
> I have already implemented it :)
>
>
> Robert,
> I saw the github notification about the new two repositories
>
> Thank you all
>
>
> Enrico
>
>
>
> >
> > >
> > > Enrico
> > >
> > >
> > > Il Sab 9 Mag 2020, 22:51 Robert Scholte ha
> > scritto:
> > >
> > > >
> > > > On 9-5-2020 17:58:47, Enrico Olivelli wrote:
> > > > Robert
> > > >
> > > > Il Sab 9 Mag 2020, 10:54 Robert Scholte ha scritto:
> > > >
> > > > > Although I'm disappointed there's no spec group for this, having
> our
> > > own
> > > > > seems indeed the best choice, but here we will likely face similar
> > > issues
> > > > > where the API needs adjustments.
> > > > > So lets create the maven-metrics git repository
> > > > >
> > > >
> > > >
> > > > Shall I do it by myself or do I need help from a PMC?
> > > >
> > > > For JIRA I think we can keep using Maven core project
> > > > Robert Scholte:
> > > > No, I'll create a separate project for it too. This extension will
> have
> > > > its own lifecycle, unrelated to the Maven lifecycle (and it won't be
> > > > bundled)
> > > >
> > > >
> > > >
> > > > >
> > > > >
> > > > > There are a few things I want to address:
> > > > > - Consider returning Optional when methods might return null.
> > > > >
> > > > Okay
> > > >
> > > > > - Specify how to handle Collections and Maps, I think we should
> > assume
> > > > > these are never null
> > > > >
> > > > Okay
> > > >
> > > > > - Abstract classes versus default methods. Default methods were
> > > > introduced
> > > > > to extend existing interfaces without breaking the contract. I
> > already
> > > > > expected patterns to appear where default methods would replace
> > > abstract
> > > > > classes. I'm not saying it is a bad thing, but just to have
> consensus
> > > > that
> > > > > we can do this.
> > > > >
> > > >
> > > > I will think more about this
> > > >
> > > > Thanks
> > > > Enrico
> > > >
> > > >
> > > > > thanks,
> > > > > Robert
> > > > >
> > > > > On 4-5-2020 17:00:39, Romain Manni-Bucau wrote:
> > > > > Le lun. 4 mai 2020 à 16:55, Slawomir Jaranowski a
> > > > > écrit :
> > > > >
> > > > > > Hi,
> > > > > > In my humble opinion it is not the best way to implement own api
> > when
> > > > > > similar api is already ready and maintained.
> > > > > >
> > > > > > There is another project used for metrics: micrometer, as we can
> > see
> > > it
> > > > > is
> > > > > > a quite popular 2.3K stars, 500 forks on github
> > > > > > https://github.com/micrometer-metrics/micrometer
> > > > > >
> > > > > > Please consider similar situation with logging api used in maven,
> > we
> > > > have
> > > > > > different logging in plexus component, maven plugin api, maven
> > core,
> > > > ...
> > > > > > and now slf4j is try to replace old
> > > > > >
> > > > > > Why it is so important to you to be independent in this case?
> > > > > >
> > > > >
> > > > > Cause none is stable and it will be user facing (mojo dev) so it is
> > key
> > > > to
> > > > > create an API we - maven - can assume in time and not depend on
> > > vendors.
> > > > > Microprofile just proved it would have been a bad choice cause they
> > > broke
> > > > > the API quite drastically (I'm not blaming them, it is the
> > microprofile
> > > > > contract for now but at maven stage it would have been a bad
> choice).
> > > > > This is not a ton of API and impl can rely on anything you want
> while
> > > > fully
> > > > > isolated (proxy on API?) from the mojo/extensions classloader
> > > (otherwise
> > > > it
> > > > > will conflict for sure).
> > > > >
> > > > >
> > > > > >
> > > > > > sob., 2 maj 2020 o 15:20 Enrico Olivelli napisał(a):
> > > > > >
> > > > > > > Robert
> > > > > > >
> > > > > > > Il Sab 2 Mag 2020, 15:11 Robert Scholte ha
> > > > > > scritto:
> > > > > > >
> > > > > > > > If I take a look at the pom of maven-metrics, I see no
> > dependency
> > > > on
> > > > > > > Maven.
> > > > > > > > And looking at
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics/src/main/java/org/apache/maven/metrics
> > > > > > > > [
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics/src/main/java/org/apache/maven/metrics
> > > > > > > > ]
> > > > > > > > This looks a lot like
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/eclipse/microprofile-metrics/tree/master/api/src/main/java/org/eclipse/microprofile/metrics
> > > > > > > > [
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/eclipse/microprofile-metrics/tree/master/api/src/main/java/org/eclipse/microprofile/metrics
> > > > > > > > ]
> > > > > > > >
> > > > > > > > So do we need to maintain our own Metrics API?
> > > > > > > >
> > > > > > >
> > > > > > > Yes it is really better.
> > > > > > >
> > > > > > > We will be in charge for this API, it will be a new API on
> which
> > we
> > > > > will
> > > > > > > depend in many part of Maven core and in plugins.
> > > > > > > It is better to not depend on third party.
> > > > > > >
> > > > > > > There are other initiatives like microprofile metrics.
> > > > > > >
> > > > > > > The API itself is very small and we could add an implementation
> > > that
> > > > > uses
> > > > > > > micro profile. But we must be independent.
> > > > > > >
> > > > > > > Enrico
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > thanks,
> > > > > > > > Robert
> > > > > > > > On 2-5-2020 10:26:19, Enrico Olivelli wrote:
> > > > > > > > Hello community,
> > > > > > > > I am now ready to move forward with concrete steps for the
> > > > > > implementation
> > > > > > > > of Maven Runtime Metrics.
> > > > > > > >
> > > > > > > > This is the JIRA
> > > > > > > > https://issues.apache.org/jira/browse/MNG-6899
> > > > > > > >
> > > > > > > > It links to my proof-of-concept branch on maven studies.
> > > > > > > > https://github.com/apache/maven-studies/tree/maven-metrics
> > > > > > > >
> > > > > > > > In order to move forward I have to create an independent
> > > module/git
> > > > > > > > repository for the Maven Metrics Runtime API.
> > > > > > > > Currently I have it on maven-studies inside Maven Core but
> this
> > > is
> > > > > not
> > > > > > > > good, because I would like to use it in Plugins independently
> > > from
> > > > > the
> > > > > > > > version of Maven Core.
> > > > > > > > When you run the plugin on an old version of Maven all of the
> > > data
> > > > > will
> > > > > > > be
> > > > > > > > simply ignored.
> > > > > > > >
> > > > > > > > My plan:
> > > > > > > > - create a git repository
> > > > > > > > - put there the first version of the API (maybe we can put
> > there
> > > a
> > > > > > simple
> > > > > > > > implementation of the API, but I could leave it off for the
> > first
> > > > > > > release)
> > > > > > > > - release it to the public
> > > > > > > > - use it in Maven 3.7
> > > > > > > > - use it in Wagon and in Resolver and in other "interesting"
> > > > > > modules/core
> > > > > > > > plugins
> > > > > > > >
> > > > > > > > Best regards
> > > > > > > > Enrico
> > > > > > > >
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Sławomir Jaranowski
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Maven Runtime Metrics System - MNG-6899

rfscholte
Just as an example:
metricsSystem
.getMetricsContext()
.getSummary( "resolvePluginDependency", "Time to resolve dependencies of a plugin (ms)" )
.add( MetricsUtils.elapsedMillis( startResolve ) );


Here you are adding something to the Metric System, but you could also but this in a single event.
In this case the event contains the time diff, but it could also contain the start and/or end time, so you won't get any delays based.

Robert


On 10-5-2020 19:32:15, Enrico Olivelli <[hidden email]> wrote:
Il Dom 10 Mag 2020, 17:49 Robert Scholte ha scritto:

> Hi Enrico,
>
> These could all be solved by firing events.
>

Yes
I see your point.
Unfortunately those are the first points that I have instrumented in order
to see some data.


> And there's another benefit: since there won't be an API, we don't need to
> maintain it,or be afraid of mistakes.
>

Exposed metrics won't be an API, or at least something to be maintained.
Each Maven version, component version, plugin version may expose any metric
is valuable for the component developer or for the users.

Exposed metrics are not an extension point, it is not expected that a
MetricsProvider is able to alter the behaviour of Maven.
Events/Extension point must be maintained forever, but you can drop/add a
metric at each release. They are very low level tracking points in code.

This is because I say that metrics are like loggers: you add logs where you
think it is useful, you can change log point at every version, there is not
strong contract with the user, logging implementation is totally pluggable.

The Metrics API is mostly the contract with Metrics Provider developers.
It will be very stable, I don't think there will be many changes in the mid
term.
It is only about the ability to give a name to counters, summaries and
gauges.

Having a simple implementation that dumps the final values at the end is
not the full power of this mechanism.

You can attach Prometheus or any other time series based metrics database
and see the evolution of the build, this will be very useful for long
running builds and especially on CI or during component development.

I will finish the implementation with Prometheus and try to create a
Grafana dashboard.

It would be super useful that some Maven-internals experts could suggest
useful values to track during the execution of the build.

I hope that explains better the value of the Metrics system.

Thank you Robert for your feedback and help

Enrico


To me this is the wrong approach to embed metrics.

>
> thanks,
> Robert
> On 10-5-2020 16:58:59, Enrico Olivelli wrote:
> Il giorno dom 10 mag 2020 alle ore 13:42 Robert Scholte
> [hidden email]> ha scritto:
>
> > sometimes code says more than a thousand words: can you share the changes
> > in Maven Core you had in mind?
> >
>
> Maven Metrics Extensions API:
>
> https://github.com/apache/maven-metric/blob/master/maven-metrics-api/src/main/java/org/apache/maven/metrics/MetricsSystem.java
> An Extension is expected to implement that Named component
>
> Example implementation:
>
> https://github.com/eolivelli/simplemavenmetrics/blob/master/src/main/java/org/apache/maven/metrics/SimpleMetricsSystem.java
> This is mostly what Romain said, if you add this extension to your project
> (and use Maven 3.7. from my branch on maven studies)
> mvn -Dmetrics.dumpAtEnd=true verify
> You will see the "final" values of every metric, it does not track any time
> series
>
> This is a sample instrumentation of Maven Core
> https://github.com/apache/maven/pull/344/files
>
> I am not sure that those are the most interesting points but it is enough
> to see interesting data on some project:
> - time to resolve the pom
> - time to resolve dependencies of plugins
> - time to "install" an artifact locally
> - time to perform the execution of each mojo
>
> I wanted to instrument more components but I have to tweak Wagon and other
> lower level external components
>
> Enrico
>
> On 10-5-2020 12:07:33, Enrico Olivelli wrote:
> > Il giorno dom 10 mag 2020 alle ore 11:53 Robert Scholte
> > [hidden email]> ha scritto:
> >
> > > Maybe I'm still missing a detail, but this is what I had in mind:
> > > Maven already fires events.
> > >
> >
> > Metrics are for a distinct use case.
> > Metrics are like Logs, the developer instruments its own code to have
> > important information available during the execution.
> > For instance you want to see the evolution of a cache, see how much it is
> > useful,
> > or you can instrument the HTTP client inside wagon to see how well the
> > network is performing.
> > Usually they are for fine grained instrumentation of hot points in code.
> > For instance you do not want to fire an event every time an entry of a
> > cache is evicted and expect some code to intercept that event.
> >
> > Metrics are not extension points like current "events" mechanism.
> > The MetricsProvider will be an extension, in a way that it is separate
> from
> > Maven Core, it only provides the way to gather the information,
> > usually in form of time series and make it available to the tools (like
> > Dashboards, reports...).
> >
> > Metrics will be useful for:
> > - Maven developers (core and plugins): see how well code behaves and
> where
> > it is better to put enhancements/optimizations
> > - Maven users: some of the metrics could be useful for users, especially
> > about the system in which Maven is executing (network, disk speed, usage
> of
> > memory during peaks....)
> > - Automatics tools, like Jenkins, to display information about the
> > execution of long running builds
> >
> >
> > Enrico
> >
> >
> >
> >
> >
> > > The extension listens to these events and can do its metrics.
> > > I see no need for a hard coupling between these two.
> > >
> > > Robert
> > >
> > > On 10-5-2020 10:28:42, Enrico Olivelli wrote:
> > > Il Dom 10 Mag 2020, 09:21 Romain Manni-Bucau ha
> > > scritto:
> > >
> > > > Le dim. 10 mai 2020 à 07:43, Enrico Olivelli a
> > > > écrit :
> > > >
> > > > > Robert
> > > > > Maybe I was misunderstood.
> > > > > We need a new repo only for the Metrics API.
> > > > >
> > > > > Maven core (3.7?) provides the noop implementation.
> > > > >
> > > > > Implementations of MetricsProvider will be extensions. I feel it is
> > > > better
> > > > > that we don't deal with implementations as a first step.
> > > > > I will leave a couple of implementations in my personal repo.
> > > > >
> > > >
> > > > Think we need at least a trivial memory impl with maybe a "log at
> > > > shutdown" feature otherwise it will not bring anything to end users
> > > >
> > >
> > >
> > > Yep
> > >
> > > I have already implemented it :)
> > >
> > >
> > > Robert,
> > > I saw the github notification about the new two repositories
> > >
> > > Thank you all
> > >
> > >
> > > Enrico
> > >
> > >
> > >
> > > >
> > > > >
> > > > > Enrico
> > > > >
> > > > >
> > > > > Il Sab 9 Mag 2020, 22:51 Robert Scholte ha
> > > > scritto:
> > > > >
> > > > > >
> > > > > > On 9-5-2020 17:58:47, Enrico Olivelli wrote:
> > > > > > Robert
> > > > > >
> > > > > > Il Sab 9 Mag 2020, 10:54 Robert Scholte ha scritto:
> > > > > >
> > > > > > > Although I'm disappointed there's no spec group for this,
> having
> > > our
> > > > > own
> > > > > > > seems indeed the best choice, but here we will likely face
> > similar
> > > > > issues
> > > > > > > where the API needs adjustments.
> > > > > > > So lets create the maven-metrics git repository
> > > > > > >
> > > > > >
> > > > > >
> > > > > > Shall I do it by myself or do I need help from a PMC?
> > > > > >
> > > > > > For JIRA I think we can keep using Maven core project
> > > > > > Robert Scholte:
> > > > > > No, I'll create a separate project for it too. This extension
> will
> > > have
> > > > > > its own lifecycle, unrelated to the Maven lifecycle (and it won't
> > be
> > > > > > bundled)
> > > > > >
> > > > > >
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > There are a few things I want to address:
> > > > > > > - Consider returning Optional when methods might return null.
> > > > > > >
> > > > > > Okay
> > > > > >
> > > > > > > - Specify how to handle Collections and Maps, I think we should
> > > > assume
> > > > > > > these are never null
> > > > > > >
> > > > > > Okay
> > > > > >
> > > > > > > - Abstract classes versus default methods. Default methods were
> > > > > > introduced
> > > > > > > to extend existing interfaces without breaking the contract. I
> > > > already
> > > > > > > expected patterns to appear where default methods would replace
> > > > > abstract
> > > > > > > classes. I'm not saying it is a bad thing, but just to have
> > > consensus
> > > > > > that
> > > > > > > we can do this.
> > > > > > >
> > > > > >
> > > > > > I will think more about this
> > > > > >
> > > > > > Thanks
> > > > > > Enrico
> > > > > >
> > > > > >
> > > > > > > thanks,
> > > > > > > Robert
> > > > > > >
> > > > > > > On 4-5-2020 17:00:39, Romain Manni-Bucau wrote:
> > > > > > > Le lun. 4 mai 2020 à 16:55, Slawomir Jaranowski a
> > > > > > > écrit :
> > > > > > >
> > > > > > > > Hi,
> > > > > > > > In my humble opinion it is not the best way to implement own
> > api
> > > > when
> > > > > > > > similar api is already ready and maintained.
> > > > > > > >
> > > > > > > > There is another project used for metrics: micrometer, as we
> > can
> > > > see
> > > > > it
> > > > > > > is
> > > > > > > > a quite popular 2.3K stars, 500 forks on github
> > > > > > > > https://github.com/micrometer-metrics/micrometer
> > > > > > > >
> > > > > > > > Please consider similar situation with logging api used in
> > maven,
> > > > we
> > > > > > have
> > > > > > > > different logging in plexus component, maven plugin api,
> maven
> > > > core,
> > > > > > ...
> > > > > > > > and now slf4j is try to replace old
> > > > > > > >
> > > > > > > > Why it is so important to you to be independent in this case?
> > > > > > > >
> > > > > > >
> > > > > > > Cause none is stable and it will be user facing (mojo dev) so
> it
> > is
> > > > key
> > > > > > to
> > > > > > > create an API we - maven - can assume in time and not depend on
> > > > > vendors.
> > > > > > > Microprofile just proved it would have been a bad choice cause
> > they
> > > > > broke
> > > > > > > the API quite drastically (I'm not blaming them, it is the
> > > > microprofile
> > > > > > > contract for now but at maven stage it would have been a bad
> > > choice).
> > > > > > > This is not a ton of API and impl can rely on anything you want
> > > while
> > > > > > fully
> > > > > > > isolated (proxy on API?) from the mojo/extensions classloader
> > > > > (otherwise
> > > > > > it
> > > > > > > will conflict for sure).
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > sob., 2 maj 2020 o 15:20 Enrico Olivelli napisał(a):
> > > > > > > >
> > > > > > > > > Robert
> > > > > > > > >
> > > > > > > > > Il Sab 2 Mag 2020, 15:11 Robert Scholte ha
> > > > > > > > scritto:
> > > > > > > > >
> > > > > > > > > > If I take a look at the pom of maven-metrics, I see no
> > > > dependency
> > > > > > on
> > > > > > > > > Maven.
> > > > > > > > > > And looking at
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics/src/main/java/org/apache/maven/metrics
> > > > > > > > > > [
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/apache/maven-studies/tree/maven-metrics/maven-metrics/src/main/java/org/apache/maven/metrics
> > > > > > > > > > ]
> > > > > > > > > > This looks a lot like
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/eclipse/microprofile-metrics/tree/master/api/src/main/java/org/eclipse/microprofile/metrics
> > > > > > > > > > [
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> https://github.com/eclipse/microprofile-metrics/tree/master/api/src/main/java/org/eclipse/microprofile/metrics
> > > > > > > > > > ]
> > > > > > > > > >
> > > > > > > > > > So do we need to maintain our own Metrics API?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Yes it is really better.
> > > > > > > > >
> > > > > > > > > We will be in charge for this API, it will be a new API on
> > > which
> > > > we
> > > > > > > will
> > > > > > > > > depend in many part of Maven core and in plugins.
> > > > > > > > > It is better to not depend on third party.
> > > > > > > > >
> > > > > > > > > There are other initiatives like microprofile metrics.
> > > > > > > > >
> > > > > > > > > The API itself is very small and we could add an
> > implementation
> > > > > that
> > > > > > > uses
> > > > > > > > > micro profile. But we must be independent.
> > > > > > > > >
> > > > > > > > > Enrico
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > > thanks,
> > > > > > > > > > Robert
> > > > > > > > > > On 2-5-2020 10:26:19, Enrico Olivelli wrote:
> > > > > > > > > > Hello community,
> > > > > > > > > > I am now ready to move forward with concrete steps for
> the
> > > > > > > > implementation
> > > > > > > > > > of Maven Runtime Metrics.
> > > > > > > > > >
> > > > > > > > > > This is the JIRA
> > > > > > > > > > https://issues.apache.org/jira/browse/MNG-6899
> > > > > > > > > >
> > > > > > > > > > It links to my proof-of-concept branch on maven studies.
> > > > > > > > > >
> https://github.com/apache/maven-studies/tree/maven-metrics
> > > > > > > > > >
> > > > > > > > > > In order to move forward I have to create an independent
> > > > > module/git
> > > > > > > > > > repository for the Maven Metrics Runtime API.
> > > > > > > > > > Currently I have it on maven-studies inside Maven Core
> but
> > > this
> > > > > is
> > > > > > > not
> > > > > > > > > > good, because I would like to use it in Plugins
> > independently
> > > > > from
> > > > > > > the
> > > > > > > > > > version of Maven Core.
> > > > > > > > > > When you run the plugin on an old version of Maven all of
> > the
> > > > > data
> > > > > > > will
> > > > > > > > > be
> > > > > > > > > > simply ignored.
> > > > > > > > > >
> > > > > > > > > > My plan:
> > > > > > > > > > - create a git repository
> > > > > > > > > > - put there the first version of the API (maybe we can
> put
> > > > there
> > > > > a
> > > > > > > > simple
> > > > > > > > > > implementation of the API, but I could leave it off for
> the
> > > > first
> > > > > > > > > release)
> > > > > > > > > > - release it to the public
> > > > > > > > > > - use it in Maven 3.7
> > > > > > > > > > - use it in Wagon and in Resolver and in other
> > "interesting"
> > > > > > > > modules/core
> > > > > > > > > > plugins
> > > > > > > > > >
> > > > > > > > > > Best regards
> > > > > > > > > > Enrico
> > > > > > > > > >
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > Sławomir Jaranowski
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Maven Runtime Metrics System - MNG-6899

Romain Manni-Bucau
In reply to this post by Enrico Olivelli
Hmm, it shouldn't be:

metricsSystem
.getMetricsContext()
.getSummary( "resolvePluginDependency", "Time to resolve dependencies of a
plugin (ms)" )
.add( MetricsUtils.elapsedMillis( startResolve ) );

but

counter.add(duration).

Resolution of the counter can be either (just in terms of inputs, then api
can be fluent or not, it is a detail for me):

Counter counter = metricSystem.get(new CounterSpec("my counter", "ms"));

or, since any operation in maven has a scope (mojo, artifact resolution,
...):

@Inject @MetricSpec("my-counter", "ms") Counter counter;

Concretely metrics system should enable to resolve a counter from a few
meta (at least name, likely also the unit) and counter is just a long (then
in terms of impl it is a LongAdder to be concurrency friendly I guess but
it is a detail).
We likely don't want to pay any other overhead otherwise I fully agree
events are almost 1-1 in terms of feature but totally opposed in terms of
design:

1. A counter is a unified view of data where data are pushed from
contributors
2. Event are an heterogeneous set of data where consumers are interpreting
the value

If you want to be iso you create a Counter event but then you have this
overhead we should avoid IMO + you just recreated metric system with
another API (likely slower due to the bus usage which requires a lot of
caching and JIT to be iso in terms of perf but it is really after some
dozen of thousands of executions).

Romain Manni-Bucau
@rmannibucau <https://twitter.com/rmannibucau> |  Blog
<https://rmannibucau.metawerx.net/> | Old Blog
<http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> |
LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book
<https://www.packtpub.com/application-development/java-ee-8-high-performance>


Le dim. 10 mai 2020 à 22:23, Enrico Olivelli <[hidden email]> a écrit :

> Il giorno dom 10 mag 2020 alle ore 22:10 Robert Scholte <
> [hidden email]> ha scritto:
>
> > This looks more like a bug, so this should be analyzed.
> > It sounds to me, that if this information was accurate, there wouldn't be
> > a need for the a separate MetricSystem.
> >
>
> Sorry, I have added that metric just to have some data to retrieve and
> monitor.
> Maybe this is not interesting.
>
> The point is that if I want to track this duration, now I can do it just by
> adding a bunch of lines locally.
> There is no need to introduce a new formal "event", that would be part of
> some public API.
> If the information is not useful I can remove it in the next release, there
> is no strong contract with the end user
>
> Enrico
>
>
>
>
>
>
>
>
> >
> > Robert
> > On 10-5-2020 20:38:39, Romain Manni-Bucau <[hidden email]> wrote:
> > Le dim. 10 mai 2020 à 20:11, Karl Heinz Marbaise a
> > écrit :
> >
> > >
> > > On 10.05.20 19:59, Romain Manni-Bucau wrote:
> > > > Events vs metrics is an old topic.
> > > > The choice between both is not only design but also about perf. In
> > terms
> > > of
> > > > design it brings the ability to export data without exposing it in
> code
> > > > (events are public). It also avoid to expose a stable api of events
> and
> > > > create coupling between plugin/exts and only require a single stable
> > api
> > > > which is very minimal (lot of projects kept their metric abstraction
> > > > stable).
> > > >
> > > > Typically wagon should export metrics but if you fire an event per
> > > download
> > > > progress it will be quite slow currently and you will allocate a lot
> of
> > > > objects (or create contentions) just to increase a monotonic counter
> (a
> > > > longadder concretely).
> > > >
> > > > So I think Enrico is right and we cant avoid metrics (I have to admit
> > > > events are overcomplex to use for plugin/ext dev, see all profiling
> > > > extensions,
> > >
> > > > not a single one works and report accurate data whereas metrics
> > > > can miss some drilldown data but would be right).
> > >
> > > Interesting point cause can you give some more details on which the
> > > testimony is based that none of them works?
> > >
> >
> > They all measure bus events, mainly mojo_start, mojo_end but at the end
> > several mojo have a duration of 0ms...even if they take minutes.
> >
> > What I saw last time I checked the two top rated by github (and google I
> > think), some events were missing (one ex where metrics are easier to
> > handle, you push your data, no computation on consumer side).
> > Out of my head in the project i was working on, gmaven plugin was totally
> > missed.
> >
> >
> > > Can you explain a accurate implementation?
> > >
> >
> > If my build take 5mn and i sum all not overlapping times of the report
> of a
> > monothreaded (T1) build then i should be close to 5mn (+- some sec for
> the
> > boot, ioc, logging, final report).
> > If I cant extract where time is spent these tools are uselesd.
> >
> > For the story i used a javaagent (based on sirona) to instrument the
> build
> > i was optimizing, was using in memory counters with dump at shutdown (in
> > csv). Indeed bytecode instrumentation overhead was not low enough but i
> > managed to explain the build duration properly with such a tool.
> >
> > If we put these basic metrics in maven core (executor for ex) I'm pretty
> > confident results will be less random than events interpretation and
> > required code is very low and trivial to maintain for any dev so guess it
> > is worth it even if off by default (noop metrics system).
> >
> >
> > > Kind regards
> > > Karl Heinz Marbaise
> > >
> > >
> > >
> > > >
> > > > Now we can limit the surfacing api and start only by counters and
> > gauges
> > > > maybe and later add histogram, meters etc only if needed (not sure
> for
> > > mvn).
> > > >
> > > > Le dim. 10 mai 2020 à 19:32, Enrico Olivelli a
> > > écrit :
> > > >
> > > >> Il Dom 10 Mag 2020, 17:49 Robert Scholte ha
> > > >> scritto:
> > > >>
> > > >>> Hi Enrico,
> > > >>>
> > > >>> These could all be solved by firing events.
> > > >>>
> > > >>
> > > >> Yes
> > > >> I see your point.
> > > >> Unfortunately those are the first points that I have instrumented in
> > > order
> > > >> to see some data.
> > > >>
> > > >>
> > > >>> And there's another benefit: since there won't be an API, we don't
> > need
> > > >> to
> > > >>> maintain it,or be afraid of mistakes.
> > > >>>
> > > >>
> > > >> Exposed metrics won't be an API, or at least something to be
> > maintained.
> > > >> Each Maven version, component version, plugin version may expose any
> > > metric
> > > >> is valuable for the component developer or for the users.
> > > >>
> > > >> Exposed metrics are not an extension point, it is not expected that
> a
> > > >> MetricsProvider is able to alter the behaviour of Maven.
> > > >> Events/Extension point must be maintained forever, but you can
> > drop/add
> > > a
> > > >> metric at each release. They are very low level tracking points in
> > code.
> > > >>
> > > >> This is because I say that metrics are like loggers: you add logs
> > where
> > > you
> > > >> think it is useful, you can change log point at every version, there
> > is
> > > not
> > > >> strong contract with the user, logging implementation is totally
> > > pluggable.
> > > >>
> > > >> The Metrics API is mostly the contract with Metrics Provider
> > developers.
> > > >> It will be very stable, I don't think there will be many changes in
> > the
> > > mid
> > > >> term.
> > > >> It is only about the ability to give a name to counters, summaries
> and
> > > >> gauges.
> > > >>
> > > >> Having a simple implementation that dumps the final values at the
> end
> > is
> > > >> not the full power of this mechanism.
> > > >>
> > > >> You can attach Prometheus or any other time series based metrics
> > > database
> > > >> and see the evolution of the build, this will be very useful for
> long
> > > >> running builds and especially on CI or during component development.
> > > >>
> > > >> I will finish the implementation with Prometheus and try to create a
> > > >> Grafana dashboard.
> > > >>
> > > >> It would be super useful that some Maven-internals experts could
> > suggest
> > > >> useful values to track during the execution of the build.
> > > >>
> > > >> I hope that explains better the value of the Metrics system.
> > > >>
> > > >> Thank you Robert for your feedback and help
> > > >>
> > > >> Enrico
> > > >>
> > > >>
> > > >> To me this is the wrong approach to embed metrics.
> > > >>>
> > > 7
> > >
> >
>