Process maven mdo files via ModelloCLI

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Process maven mdo files via ModelloCLI

William L. Thomson Jr.
I have been working on packaging and building Maven from source on
Gentoo. Without using Maven, just straight javac, etc.  It is looking
like I cannot do this without using pre-generated sources on Maven
Central. I am having issues with ModelloCLI loading plugins.
https://github.com/codehaus-plexus/modello/issues/15

My gut says MetadataPlugins are not the same as Generators. Maybe need
to package more of Modello plugins and/or have them on the classpath. I
am really not clear how Modello loads its plugins via Plexus. Once I
can get the plugin loaded. I should be able to proceed. This is where I
am stuck, Modello loading a plugin to generate Java from mdo files.
I do not need any other formats, just Java.

Even more interesting, seems to be a chicken/egg situation with parts
of Plexus. Which also has mdo files.
https://github.com/codehaus-plexus/plexus-containers/tree/plexus-containers-1.x/plexus-container-default/src/main/mdo

Mailing lists for Modello and Plexus seem MIA. It does not seem like
anything else uses this other than Maven. This is not a normal user
issue, thus asking on Maven development list.

If someone here does not know about Modello and Plexus. Not sure where
I can find someone who does... Thanks for any help!

--
William L. Thomson Jr.

attachment0 (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Process maven mdo files via ModelloCLI

Robert Scholte-8
Hi William,

I fully agree with you that Maven developers should know about Plexus and  
Modello, and in fact if you compare the names of the developers of both  
projects, you should recognize an overlap.
For me this case is too specific, it would take quite some time to dive  
into, while many, many other issues are waiting as well to be fixed. And  
in fact, your issue starts with this sentence "I am doing abnormal  
things!" :)

Based on the analysis in the comments the issue does seem to be having 2  
instances instead of 1. I would try to figure out where and why this is  
happening.

If you find the bug and are able to fix it, please provide a PR so we can  
improve Modello.

thanks,
Robert

On Sun, 24 Jun 2018 02:35:28 +0200, William L. Thomson Jr.  
<[hidden email]> wrote:

> I guess no one is familiar with Plexus or Modello. If Maven developers
> are not familiar with what technologies Maven is built atop. Not sure
> who else would have a clue about Plexus or Modello.
>
> I can look to building Maven via other means. But does not reflect to
> well on Maven IMHO. Plexus and Modello are part of how Maven is built.
> Someone should have a clue about both.
>
> If your a Maven developer, I highly recommend getting to know Plexus
> and Modello. It is the underlying foundation to Maven it seems.
>
> On Thu, 14 Jun 2018 10:41:14 -0400
> "William L. Thomson Jr." <[hidden email]> wrote:
>
>> I have been working on packaging and building Maven from source on
>> Gentoo. Without using Maven, just straight javac, etc.  It is looking
>> like I cannot do this without using pre-generated sources on Maven
>> Central. I am having issues with ModelloCLI loading plugins.
>> https://github.com/codehaus-plexus/modello/issues/15
>>
>> My gut says MetadataPlugins are not the same as Generators. Maybe need
>> to package more of Modello plugins and/or have them on the classpath.
>> I am really not clear how Modello loads its plugins via Plexus. Once I
>> can get the plugin loaded. I should be able to proceed. This is where
>> I am stuck, Modello loading a plugin to generate Java from mdo files.
>> I do not need any other formats, just Java.
>>
>> Even more interesting, seems to be a chicken/egg situation with parts
>> of Plexus. Which also has mdo files.
>> https://github.com/codehaus-plexus/plexus-containers/tree/plexus-containers-1.x/plexus-container-default/src/main/mdo
>>
>> Mailing lists for Modello and Plexus seem MIA. It does not seem like
>> anything else uses this other than Maven. This is not a normal user
>> issue, thus asking on Maven development list.
>>
>> If someone here does not know about Modello and Plexus. Not sure where
>> I can find someone who does... Thanks for any help!
>>
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Process maven mdo files via ModelloCLI

William L. Thomson Jr.
On Mon, 25 Jun 2018 10:28:20 -0400
"William L. Thomson Jr." <[hidden email]> wrote:

> Hi Robert,
>
> > Based on the analysis in the comments the issue does seem to be
> > having 2 instances instead of 1. I would try to figure out where and
> > why this is happening.  
>
> That I have been doing for sometime. Its rare for me to reach out for
> help in building stuff.

Having spent some time debugging this, I came to realize the it is not
initializing twice. It initializes the MetaDataPlugin as part of Plexus
startup. Which precedes Modello startup. But those are not Generators.
For what ever reason no Generator plugins are loaded, or show up.
But the HashMap has elements.

I can provide the patch, but it basically just does System.out.println
in places where I added text.

In initialize I loop through plugins.

System.out.println("plugins = "+plugins.size());
plugins.forEach((k, v) -> System.out.println(k + " : " + v));

https://github.com/codehaus-plexus/modello/blob/master/modello-core/src/main/java/org/codehaus/modello/plugin/AbstractPluginManager.java#L44

Where plugins is
private Map<String, T> plugins = new HashMap<String, T>()

What is interesting there, is that there is a plugin count, with empty
key or value for Generators . I can try to print out null elements.
Though likely get a NPE or something in that process.

Here is some example output with java, xml, and xsd plugins. I see the
MetaDataPlugins, but never any Generators. The plugin ModelloCLI wants
is a generator not a MetaDataPlugin.

--------------------------------------------------------------------------------------------------------------

$ modello  settings.mdo modello ../java/ 4.0.0 false true
DefaultPlexusContainer
AbstractPluginManager.initialize()
plugins = 4
xsd : org.codehaus.modello.plugin.xsd.metadata.XsdMetadataPlugin@275710fc
model : org.codehaus.modello.plugin.model.ModelMetadataPlugin@4de5031f
java : org.codehaus.modello.plugin.java.metadata.JavaMetadataPlugin@5d47c63f
xml : org.codehaus.modello.plugins.xml.metadata.XmlMetadataPlugin@3bbc39f8
AbstractPluginManager.initialize()
plugins = 6
ModelloCore
new Modello()
parsed args
generatorId = modello
plugins = 6
getPlugin = modello
Exception in thread "main"
org.codehaus.modello.ModelloRuntimeException: No such plugin: modello

--------------------------------------------------------------------------------------------------------------

Not sure how it sees 6 plugins from 4 before. Or why I cannot access any.
Seems like some Generator plugins exist. But the HashMap has some issues.
Not sure if its improper types being passed or what.

--
William L. Thomson Jr.

attachment0 (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Process maven mdo files via ModelloCLI

William L. Thomson Jr.
In reply to this post by Robert Scholte-8
Robert,

        Nice timing, our posts literally crossed at the same time! Also
on the same topic, Generators vs MetaDataPlugins.

On Mon, 25 Jun 2018 22:10:48 +0200
"Robert Scholte" <[hidden email]> wrote:

> I've got some answers for you:
>
> you're seeing 2 times init because there are 2 AbstractPluginManager  
> instances: org.codehaus.modello.core.MetadataPluginManager and  
> org.codehaus.modello.core.GeneratorPluginManager

That part I have figured out. I also played around with components.xml
and in an incorrect way got a false Generator to show.
https://github.com/codehaus-plexus/modello/issues/15#issuecomment-397433691

But that is not correct, as when I go to use I get a class cast
exception.
org.codehaus.modello.core.DefaultGeneratorPluginManager cannot be cast
to org.codehaus.modello.plugin.ModelloGenerator

That at least loaded a Generator that could be found via ModelloCLI.
The furthest I have gotten, and it is not correct. Only sign of any
progress thus far :(

> They are initiated when calling the Modello()-constructor. It creates
> the DefaultPlexusContainer, which is the CDI container used. Based on
> the META-INF/plexus/components.xml it'll instantiate and bind all
> the instances.

For some reasons it seems like the Generators are not being loaded or
bound. Or maybe they are, and something is messing up the HashMap. In
my debug output, I get a count from plugin Generators. HashMap has a
size, but I cannot loop through the elements in any normal means.

Which I believe is also causing the default lookup to fail. If I cannot
loop through and print out Generator plugins in the Generators plugin
HashMap. That would also explain why ModelloCLI cannot lookup the
plugin using the same method, getting value based on key.

Not sure which is null, key or value, or why. Or why hacking the
components.xml loads a bad Generator. When I cannot get it to load
valid ones.

It maybe a bug somewhere.

--
William L. Thomson Jr.

attachment0 (201 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Process maven mdo files via ModelloCLI

William L. Thomson Jr.
In reply to this post by William L. Thomson Jr.
Hi Hervé,

        Thanks for the response!

On Sun, 26 Aug 2018 19:35:02 +0200
Hervé BOUTEMY <[hidden email]> wrote:

> Hi William,
>
> Debian already worked on this: I don't precisely know how they did,
> but you should probably have a look.

Well they can cut some corners as they need to build it just once for
their package. I have to make it such that anyone can build it from
source. Given Gentoo is a from source distro. That is what makes things
a bit harder to package on Gentoo vs binary distros.

> A few hints on your investigations:
> - on the .mdo files in plexus-containers, I added them only to
> generate documentation [1], then you can just ignore that ones

Good to know. That is one of the differences with sisu-plexus. It does
not have that mdo file, and some other changes. But is basically the
same as plexus-containers. Why I think any other stuff using
plexus-containers may work as well with sisu-plexus.

Not sure if using plexus-containers rather than sisu-plexus, is the
cause for ModelloCli to not be able to load/find the Java plugin. Just
one thing I have not tried with Modello. I have tried a bunch of other
stuff. Packaging the other plugins, various hackish debug code to see
whats being loaded and not. No clue how Modello loads its plugins.
Seems ModelloCli is not really functional or something.

> - are you using the old Ant build file, that was used to bootstrap
> Maven?

No I am using straight javac. Gentoo ebuilds are basically bash
scripts. We have eclasses which are a libary of bash functions for
common re-use.

I have most all of Maven packaged. I just had to use sources from Maven
Central for some packages that had mdo files. Ideally I would like to
use regular sources. Since that also allows live building from git.

Right now one could not use an ebuild to build from git, as I cannot
process the mdo files. Using Maven Central sources works for packaging,
but not for say development. If someone wanted to further maven, or
test out changes in git.


--
William L. Thomson Jr.

attachment0 (201 bytes) Download Attachment