Re: [Proposal] When Old Meets New: Turning Maven into a High Scalable, Resource Efficient, Cloud Ready Microservice

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: [Proposal] When Old Meets New: Turning Maven into a High Scalable, Resource Efficient, Cloud Ready Microservice

Massimiliano Dessì
Hi Robert

2018-08-07 18:35 GMT+02:00 Robert Scholte <[hidden email]>:

> Hi Max,
>
> regarding the contribution, I would like to propose the following:
> - At least create a JIRA issue for every feature.
> - Due to the large amount of code I can imagine there are some
> dependencies between the features, i.e some features should be implemented
> first.
> - Based on that we can decide what we want to adopt, prioritize/order and
> start working on the implementations.
> - I assume CLA[1] is not an issue ;)
>

+1


>
> Do you have statistics of the memory consumption?
>

Nope,

>
> It looks to me we need to introduce some sort of profiles, so people can
> choose the preferred setup.
> I see both the memory and concurrency as solutions that come with "costs"
> not everybody wants to pay.
> Did you solve this with a specific builder?
>

I've created the entire "compiler" to use with only one instance or
multiple, but now if multiple running in the same time they working
correctly,
the two major problems arised during the concurrent builds was the random
build failed to changed order of the parameters and the output interleaved
in the same output.


> Also, did you have to break APIs or are all current plugins and extensions
> still usable?
>

No API breaks, two aspects was mandatory, easy (or at least with few
changes) update of Maven version
and the same behaviour compared to the MavenCLI,
for this reason the changes on Maven API are limited to few classes of the
Maven embedder, some was suggested by the debugging experience,
everytime the major part of the time was spent on the creation of the
Plexus/Sisu Ioc Container and this suggest to reuse the Container and some
internal objects if the pom isn't changed between builds,
because in our workbench the editing is on the Java files and on drools
rules mostly,
in this way after the first build, when the maven infrastructure is created
and takari build all the buildable stuffs,
the others following builds are fast, the Maven infrastructure used by the
embedder is ready and only the changes classes needs to be recompiled,
in our workbench we cache the compiler created in this way with the project
associated, and this association is alive for the entire working session,
a step furter could be the changes of Maven internal but this mean breaks
the Maven API.
Then when we have writed some concurrent build test to see if the Maven
runtime was happy we have discovered some problems and fixed,
specific slf4j in memory appenders are used to write the specific build
output, since the embedder redirects the entire System.out to a single file.

Here the Mojo used to share the drools runtime objects with the client
caller, it's the culprit of all these stuffs :)
https://github.com/kiegroup/droolsjbpm-integration/blob/master/kie-maven-plugin/src/main/java/org/kie/maven/plugin/BuildMojo.java#L183



>
> thanks,
> Robert
>
> [1] https://www.apache.org/dev/new-committers-guide


Best
Max


>
>
>
> On Mon, 06 Aug 2018 11:16:36 +0200, Massimiliano Dessì <
> [hidden email]> wrote:
>
> Hi all,
>> as a part of my daily job in Red Hat
>> I've worked on a "customization" of Maven for our Kie Workbench used with
>> Drools, JBPM and Optaplanner.
>> The starting point was exports Objects created by Drools inside our Maven
>> plugin, but the features are growed a lot.
>>
>> A brief and not exhaustive list of the features:
>> -From static file producer to in memory producer and exporter
>> -Reusable internal components to optimize memory footprint and time of
>> execution
>> -Maven like a builder Daemon
>> -Configurable behaviour using a pipeline of decorators
>> -Stateless (when possible)
>> -Concurrent builds and concurrent logs
>> -Cloud enabled
>> -Local and remote executions
>> -Plugins turned from FileSystem based to in memory based
>> -Async API to consume build result
>> -Incremental builds
>>
>> The current version is based on the Maven 3.3.9 but could be easily
>> updated
>> and is in our plans in the next months.
>> Currently we have four modules plus other testing module and the
>> offprocess
>> module is under development.
>>
>> core
>> service
>> maven-plugins (plugins turned from FS to in memory)
>> distribution
>>
>> Preso with details:
>> https://www.slideshare.net/desmax74/when-old-meets-new-turni
>> ng-maven-into-a-high-scalable-resource-efficient-cloud-ready-microservice
>>
>> I'd like to contribute this code, but since isn't a simple patch I'd like
>> discuss about how to contribute,
>> code plus tests is around 18k loc.
>>
>> Best
>> Max
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>


--
http://twitter.com/desmax74
http://www.linkedin.com/in/desmax74
http://www.slideshare.net/desmax74
Author of: Spring 2.5 Aspect Oriented Programming
"http://www.packtpub.com/aspect-oriented-programming-with-spring-2-5/book"