Re: Filetrasformer design

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Re: Filetrasformer design

Slawomir Jaranowski
ok, duplicate transformer execution will be resolved by MNG-5667.

Of course the performance of signing with an external tool and internal is
not a good comparison, but with  external tool process was run once and now
will be run twice. So I thought about performance in this way that we will
do the same job many times.

I see that transformation should be changed, so can we define (if possible)
small steps which get us to the goal, but first we need this goal.

What I see:
- possibility to plug transformations for poms
- possibility to plug transformations for other artifacts (do we want?)
- possibility to plug transformations for signing/checksumming

So never mind we will have transformation in two levels:
pom -> signing/consuming
each of this level can produce new artefakt or something else which should
be attached / installed /deployed

When we use transformation like now we need to define all transformation
and output items from them before start, because the list of project
artifacts which will be installed/deployed is built before the
transformation process is started.


niedz., 11 paź 2020 o 11:59 Robert Scholte <[hidden email]>
napisał(a):

> It is true that for both install and deploy the transformers will be
> executed, however in general there is no need to do both, this is how Maven
> still works, but should be solved with MNG-5667[1].
> This might be a good reason to give this a higher priority. (and to get
> "mvn clean install" out of everybody's system)
>
> I don't think there's a lot of need for transformers, so I could image
> Maven will predefine the available transformation hooks, i.e. pom
> transformers and checksum/signature transformers.
>
> I would like to see numbers of performance differences. (keep in mind it
> is probably an unfair comparison, because signing is now done by a third
> party program).
> To reduce memory consumption the transfomer is using PipedStreams, which
> should imply that size doesn't matter anymore.
>
> thanks,
> Robert
>
>
> [1] https://issues.apache.org/jira/browse/MNG-5667
> On 11-10-2020 09:06:11, Slawomir Jaranowski <[hidden email]>
> wrote:
> Hi,
>
>
> During a spike on maven-sign-plugin we discovered the chain feature in the
> Transformation process will be useful.
>
> Current transformation solution is dedicated to pom transformation, it will
> be also useful to prepare more flexible solutions for another type of
> transformation like preparing signatures for artefacts.
>
> Present implementation of transformers cause of calling the whole
> transformation process from beginning for all requests.
>
> We process it twice - first for install and second for deploy.
> So when we add more complicated and more resource demand (like computation
> of signature) we can decrease performance in a visible way.
>
> Now try to build a process with a chined transformer, we will have a
> PomTransformer which can produce two kinds of pom.
> Next transformer will be for signing SigTransformer, so we will have:
>
> - pom.xml
> -> PomTransformer
> - pom.xml FileTransformer
> -> SigTransformer
> - pom.xml.asc FileTransformer
> - pom5.xml FileTransformer
> -> SigTransformer
> - pom5.xml.asc FileTransformer
>
>
> In this example we can see that in output we need to produce 4 artifacts:
> pom.xml, pom5.xml and pom.xml.asc, pom 5.xml.asc so we need 4 final
> transformers and transformer from previous level was run twice.
>
> The biggest problem which I see is that we need results from all
> Transformers level.
>
> I'm afraid that when we start to use transformers for all artifacts we can
> decrease performance for projects with big artifacts.
>
> And the end - summary:
> 1. problem of running transformers many times during project build
> 2. multilevel / chained transformers - how to reuse output from previous
> level
>
> --
> Sławomir Jaranowski
>


--
Sławomir Jaranowski