Re: main-class + module-version

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: main-class + module-version

rfscholte
On Wed, 20 Dec 2017 07:40:24 +0100, Plamen Totev  
<[hidden email]> wrote:

> Hi,
>
> I've started working on implementing support for modular JARs in
> Plexus Archiver but as I research the matter I found that they are
> more complex than I expected. I've updated the GitHub issue[1] with
> the details but I'll post the findings here as well. The modular JAR
> files contain not only version and main class but also:
>
> * List of the packages contained - usually populated by the compiler,
> but there are some caveats here. The most obvious one is that for jars
> created by the Shade Plugin this attribute will almost certainly be
> incorrect - the plugin will most likely introduce additional packages
> and the list produced by the compiler will be out of date. The other
> caveat are the resources - they could be located in packages and
> encapsulated the same way as the classes. The jar tools records not
> only the class packages but the resource ones too. Packaging resources
> together with the classes may result in incorrect module descriptor if
> this attribute is not updated.
>
> * The module main class - passed as argument to the jar tool and it
> seems that it cannot be set using the compiler.
>
> * The module version - passed as argument to the jar tool. Could be
> set using the compiler(again passed as argument) as well.
>
> * Hashes - calculates the hashes of given external modules that depend
> on this one. During runtime java checks the recorded hashes against
> the resolved module and and if don't match it will fail. The primary
> use case is a module A that exports packages to B and C. An easy way
> to break the encapsulation of A is to create module named B. To
> prevent that you can record the hash of B and C inside A. Then A will
> export the packages only to the intended modules. It is used by the
> OpenJDK but could be useful for platforms and frameworks as well.
>
> * Module resolution - marks the module to not be resolved by default or
> marks it as deprecated or incubating so at run time a warning is
> displayed if it is resolved. I don't think you can set it using the
> JDK 9 jar tool but it's present in the code. Maybe it will be
> implemented in future version of the jar tool.
>
> To me it looks like the packaging of modular JAR file is more complex
> than just packing module-info.class file and the jar tool is no longer
> just a ZIP program. Something similar is stated in the tool
> documentation[2]:
>
>> The jar command is a general-purpose archiving and compression tool,
>> based on the ZIP and ZLIB compression formats.
>> Initially, the jar command was designed to package Java applets
>> or applications; however, beginning with JDK 9, users can use the
>> jar command to create modular JARs.
>> For transportation and deployment, it’s usually more convenient to
>> package modules as modular JARs.
>
> While not at all that complex from technical point of view, I don't
> think it is worth implementing, and whats more maintaining, all this
> functionality. Probably it would be better if the plugins that produce
> JAR files use the jar tool, the same way the compiler plugin does not
> compile by itself but uses the Java compiler.
>
> What do you think?

Based on this message it seems worth implementing a JarToolArchiver, using  
the jar tool via the ToolProvider[1]
I hope it can still be a org.codehaus.plexus.archiver.Archiver, otherwise  
I'll contact the openjdk team about the details of the specifications. I  
don't think we need them all, good to know the reason for the extra files.

thanks so far,
Robert


[1]  
https://docs.oracle.com/javase/9/docs/api/jdk.jartool-summary.html#module.description

>
>
> [1]  
> https://github.com/codehaus-plexus/plexus-archiver/issues/69#issuecomment-349095101
> [2] https://docs.oracle.com/javase/9/tools/jar.htm
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: main-class + module-version

Plamen Totev-2
Hi,

Actually that might just work. Good idea. I'll take a look at it.

Thanks,
Plamen Totev

On Wed, Dec 20, 2017 at 9:53 PM, Robert Scholte <[hidden email]> wrote:

> On Wed, 20 Dec 2017 07:40:24 +0100, Plamen Totev <[hidden email]>
> wrote:
>
>> Hi,
>>
>> I've started working on implementing support for modular JARs in
>> Plexus Archiver but as I research the matter I found that they are
>> more complex than I expected. I've updated the GitHub issue[1] with
>> the details but I'll post the findings here as well. The modular JAR
>> files contain not only version and main class but also:
>>
>> * List of the packages contained - usually populated by the compiler,
>> but there are some caveats here. The most obvious one is that for jars
>> created by the Shade Plugin this attribute will almost certainly be
>> incorrect - the plugin will most likely introduce additional packages
>> and the list produced by the compiler will be out of date. The other
>> caveat are the resources - they could be located in packages and
>> encapsulated the same way as the classes. The jar tools records not
>> only the class packages but the resource ones too. Packaging resources
>> together with the classes may result in incorrect module descriptor if
>> this attribute is not updated.
>>
>> * The module main class - passed as argument to the jar tool and it
>> seems that it cannot be set using the compiler.
>>
>> * The module version - passed as argument to the jar tool. Could be
>> set using the compiler(again passed as argument) as well.
>>
>> * Hashes - calculates the hashes of given external modules that depend
>> on this one. During runtime java checks the recorded hashes against
>> the resolved module and and if don't match it will fail. The primary
>> use case is a module A that exports packages to B and C. An easy way
>> to break the encapsulation of A is to create module named B. To
>> prevent that you can record the hash of B and C inside A. Then A will
>> export the packages only to the intended modules. It is used by the
>> OpenJDK but could be useful for platforms and frameworks as well.
>>
>> * Module resolution - marks the module to not be resolved by default or
>> marks it as deprecated or incubating so at run time a warning is
>> displayed if it is resolved. I don't think you can set it using the
>> JDK 9 jar tool but it's present in the code. Maybe it will be
>> implemented in future version of the jar tool.
>>
>> To me it looks like the packaging of modular JAR file is more complex
>> than just packing module-info.class file and the jar tool is no longer
>> just a ZIP program. Something similar is stated in the tool
>> documentation[2]:
>>
>>> The jar command is a general-purpose archiving and compression tool,
>>> based on the ZIP and ZLIB compression formats.
>>> Initially, the jar command was designed to package Java applets
>>> or applications; however, beginning with JDK 9, users can use the
>>> jar command to create modular JARs.
>>> For transportation and deployment, it’s usually more convenient to
>>> package modules as modular JARs.
>>
>>
>> While not at all that complex from technical point of view, I don't
>> think it is worth implementing, and whats more maintaining, all this
>> functionality. Probably it would be better if the plugins that produce
>> JAR files use the jar tool, the same way the compiler plugin does not
>> compile by itself but uses the Java compiler.
>>
>> What do you think?
>
>
> Based on this message it seems worth implementing a JarToolArchiver, using
> the jar tool via the ToolProvider[1]
> I hope it can still be a org.codehaus.plexus.archiver.Archiver, otherwise
> I'll contact the openjdk team about the details of the specifications. I
> don't think we need them all, good to know the reason for the extra files.
>
> thanks so far,
> Robert
>
>
> [1]
> https://docs.oracle.com/javase/9/docs/api/jdk.jartool-summary.html#module.description
>
>>
>>
>> [1]
>> https://github.com/codehaus-plexus/plexus-archiver/issues/69#issuecomment-349095101
>> [2] https://docs.oracle.com/javase/9/tools/jar.htm
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: main-class + module-version

Plamen Totev-2
In reply to this post by rfscholte
Hi,

On 12/20/2017 9:53 PM, Robert Scholte wrote:
>
> Based on this message it seems worth implementing a JarToolArchiver,
> using the jar tool via the ToolProvider[1]
> I hope it can still be a org.codehaus.plexus.archiver.Archiver,
> otherwise I'll contact the openjdk team about the details of the
> specifications. I don't think we need them all, good to know the reason
> for the extra files.

So I did some experiments and definitely it is possible to implement it
as org.codehaus.plexus.archiver.Archiver. I think it would be best to
reuse JarArchiver to create the  non-modular JAR file and then use the
JDK jar tool to update it to modular JAR file. The downside of this
approach is that it involves additional work(there is some performance
penalty). The JDK jar tool updates files the same as way Plexus Archiver
- creates new file, copies the old entries and adds the new ones. Do you
think this is really an issue? I think for small JAR files the
difference would not be noticeable.

Of course we could create the jar file directly using only the JDK jar
tool. But the Plexus Archiver is quite advanced tool compared to it. If
we have to implement all of its functionality using only the JDK jar
tool, it would be easier to update the module descriptors using asm (IMHO).

About the ToolProvider - maybe I'm missing something but it is available
only for Java 9 and does not allow the use of tool chains, does it?

Regards,
Plamen Totev

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: main-class + module-version

Plamen Totev-2
Hi Robert,

On 1/15/2018 10:10 PM, Robert Scholte wrote:
 > Hi Plamen,
 >
 > Alan Bateman has provided some valuable information:

Thank you for sharing this. It's really helpful.

On 1/15/2018 10:10 PM, Robert Scholte wrote:
> 1. MR JARs weren't mentioned. The jar tool does validation to ensure
> that the API provided by the classes in the JAR file is the same for all
> versions.

Validation is indeed something I haven't dig in yet. It is interesting
to know what is the impact of Maven not doing validation of modular JAR
files.

On 1/15/2018 10:10 PM, Robert Scholte wrote:
> 2. The Module, ModulePackages, and ModuleMainClass class file attributes
> are specified in the JVMS. ASM supports them so the Archiver could use
> that. The ModulePackages attribute is optional. If this attribute is not
> present then the JAR file will be scanned to get the set of packages in
> the module.

If the JAR is scanned when ModulePackages is not present then for the
time being is ok to not implement it.

On 1/15/2018 10:10 PM, Robert Scholte wrote:
> The ModuleTarget, ModuleResolution, and ModuleHashes class file
> attributes are JDK-specific so you won't find these in the JVMS. The
> ModuleTarget attribute is documented in JEP 261, the others aren't there
> yet but ASM has support in org.objectweb.asm.commons for these
> attributes so you should be okay.

All of those could be set with ASM (I think) but setting them is the
easy part. Calculating the hashes for ModuleHashes will require some
work though.

On 1/15/2018 10:09 PM, Andreas Sewe wrote:
 > Also, if two separate tools modify the JAR, the goal of reproducible
 > builds is again a bit harder to accomplish. So, please try to use an
 > internal, written-in-Java solution if possible.

This is a valid point. I don't think it would be possible to set the
entries timestamps using the JDK jar tool. But a bit harder does not
mean impossible. It could be achieved by "post-processing" the resulting
jar file.

Another point I think we should have in mind is the fact that every time
the class format version is increased we should update the ASM library
and release a new version of the JAR plugin otherwise the build will
fail. Also the users should update the JAR plugin version. That may
prove to be a bit inconvenient.

 From my point of view the trade-off is ease of maintainability vs
flexibility and control.

Regards,
Plamen Totev

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]