New standard directory layout for better multi-languages and jigsaw support?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

New standard directory layout for better multi-languages and jigsaw support?

Martin Desruisseaux
Hello all

Note: this email mixes two issues: Java 9 support and support of
languages other than Java. I discuss those two issues together because
the proposed experiment tries to address them together. This email is
also reworded as a wiki page at [1].

Last April we had an email thread about "Folder structure for Java 9
project". After 4 months of thinking and doing some experiments, I came
to question whether the Maven standard directory layout is still suited
to the current development landscape. My issues are:

  * Maven standard directory layout is incompatible with the
    --module-path options of JDK 9 tools (javac, javadoc, jlink,
    /etc./). It has been argued that those options are only useful for
    building more than one module at once. But I want this capability
    because:
      o I'm not aware of other way to generate satisfying aggregated
        javadoc (MJAVADOC-449).
      o It concerns not only aggregated javadoc, but most JDK tools
        (e.g. annotation processors).
      o It simplifies the handling of some circular dependencies.
  * When using more than one language (in my case Java, Python and
    C/C++) as components of the same project, I find the
    "src/<main_or_test>/<language>" directory order tedious to use because:
      o The tests are not close to the classes they are testing (less
        than if the order was "src/<language>/<main_or_test>"). They are
        interleaved with directories of other languages, thus
        distracting navigation with less relevant content (from the
        perspective of target language).
      o Maven directory layout assumes that all languages split their
        files in the same set of directories ("main" and "test"). But
        Python and other languages have their own tools and practices.
      o If a language needs another directory (e.g. "doc"), creating
        "src/doc/<language>" imposes to all languages in that module a
        directory relevant to only one language.
      o "src/main/resources" is Java resources in practice. It would
        have been more obvious if the path was "src/java/resources".
      o Is the "src" level really necessary? After 15 years of using
        Maven in various projects, it still appears like noise to me.
  * The /"one pom.xml = one module"/ paradigm is not always convenient
    because:
      o If transposed to Python where a module is basically a single
        file, it creates at least 4 files or directories boilerplate for
        each Python file.
      o It does not suit well to the build of many modules at once with
        JDK 9 --module-path option; still workable, but not natural.

I realize that the /"one pom.xml = one module"/ paradigm is deeply
anchored in Maven architecture, but I came to a point where alternatives
seem yet more complicated (at least as a user). A directory layout
solving the above issues could be:

    MyProject
     ├─ pom.xml
     ├─ MavenModule1
     │   ├─ pom.xml
     │   ├─ java
     │   │   ├─ org.foo.bar1
     │   │   │   ├─ main
     │   │   │   │   └─ org/foo/bar1/…
     │   │   │   └─ test
     │   │   │       └─ org/foo/bar1/…
     │   │   ├─ org.foo.bar2
     │   │   │   ├─ main
     │   │   │   │   └─ org/foo/bar2/…
     │   │   │   └─ test
     │   │   │       └─ org/foo/bar2/…
     │   │   └─ org.foo.bar3
     │   └─ python
     │       ├─ foo
     │       │   ├─ bar1.py
     │       │   └─ bar2.py
     │       └─ test_bar1.py
     └─ MavenModule2
         ├─ pom.xml
        ...

A controversial point would be that a single Maven module, say
MavenModule1, could contain an arbitrary amount of Java 9 or Python
modules. The pom.xml for each Java 9 modules would either be inferred
automatically by computing the "intersection" of MavenModule1/pom.xml
with dependencies declared in the module-info of each Java module, or
specified explicitly in another way. GroupID and artifactID can be
inferred automatically too. After install or deploy phase, the JAR +
pom.xml files would be handled as today's Maven artifact.

This would require modifications of at least the Maven compiler,
javadoc, jar and deploy plugins (I realize that it would impact all
plugins, but other plugins could throw some "unsupported directory
layout" exception for now). I'm willing to experiment such layout and
Maven plugins modifications in the following months, because after 4
months of thinking I found no other satisfying solution except switching
to another build system. But comments are appreciated, and I would be
glad to propose the results back to Maven if the community consider them
acceptable.

Any though?

    Martin

[1] https://github.com/Geomatys/maven-plugins/wiki

Reply | Threaded
Open this post in threaded view
|

Re: New standard directory layout for better multi-languages and jigsaw support?

rfscholte
Hi Martin,

based on your comments I would suggest to have a look at pro[1], which is  
a buildtool comparable with Maven, but which builds on top of all Java 9  
features.
I think you'll have more success there. Any help on trying to implement  
improvements for Maven is highly appreciated, but we have other priorities  
for Maven 4 and 5. We're talking about it for quite some years, but still  
missing the time to work on it, since we're still a small group of  
volunteers.
Hence I totally forgot about MJAVADOC-449; the stream of issues,  
improvements, messages on mailinglists, etc is ongoing, I can't handle  
them all.

thanks,
Robert

[1] https://github.com/forax/pro

On Wed, 29 Aug 2018 18:08:49 +0200, Martin Desruisseaux  
<[hidden email]> wrote:

> Hello all
>
> Note: this email mixes two issues: Java 9 support and support of
> languages other than Java. I discuss those two issues together because
> the proposed experiment tries to address them together. This email is
> also reworded as a wiki page at [1].
>
> Last April we had an email thread about "Folder structure for Java 9
> project". After 4 months of thinking and doing some experiments, I came
> to question whether the Maven standard directory layout is still suited
> to the current development landscape. My issues are:
>
>   * Maven standard directory layout is incompatible with the
>     --module-path options of JDK 9 tools (javac, javadoc, jlink,
>     /etc./). It has been argued that those options are only useful for
>     building more than one module at once. But I want this capability
>     because:
>       o I'm not aware of other way to generate satisfying aggregated
>         javadoc (MJAVADOC-449).
>       o It concerns not only aggregated javadoc, but most JDK tools
>         (e.g. annotation processors).
>       o It simplifies the handling of some circular dependencies.
>   * When using more than one language (in my case Java, Python and
>     C/C++) as components of the same project, I find the
>     "src/<main_or_test>/<language>" directory order tedious to use  
> because:
>       o The tests are not close to the classes they are testing (less
>         than if the order was "src/<language>/<main_or_test>"). They are
>         interleaved with directories of other languages, thus
>         distracting navigation with less relevant content (from the
>         perspective of target language).
>       o Maven directory layout assumes that all languages split their
>         files in the same set of directories ("main" and "test"). But
>         Python and other languages have their own tools and practices.
>       o If a language needs another directory (e.g. "doc"), creating
>         "src/doc/<language>" imposes to all languages in that module a
>         directory relevant to only one language.
>       o "src/main/resources" is Java resources in practice. It would
>         have been more obvious if the path was "src/java/resources".
>       o Is the "src" level really necessary? After 15 years of using
>         Maven in various projects, it still appears like noise to me.
>   * The /"one pom.xml = one module"/ paradigm is not always convenient
>     because:
>       o If transposed to Python where a module is basically a single
>         file, it creates at least 4 files or directories boilerplate for
>         each Python file.
>       o It does not suit well to the build of many modules at once with
>         JDK 9 --module-path option; still workable, but not natural.
>
> I realize that the /"one pom.xml = one module"/ paradigm is deeply
> anchored in Maven architecture, but I came to a point where alternatives
> seem yet more complicated (at least as a user). A directory layout
> solving the above issues could be:
>
>     MyProject
>      ├─ pom.xml
>      ├─ MavenModule1
>      │   ├─ pom.xml
>      │   ├─ java
>      │   │   ├─ org.foo.bar1
>      │   │   │   ├─ main
>      │   │   │   │   └─ org/foo/bar1/…
>      │   │   │   └─ test
>      │   │   │       └─ org/foo/bar1/…
>      │   │   ├─ org.foo.bar2
>      │   │   │   ├─ main
>      │   │   │   │   └─ org/foo/bar2/…
>      │   │   │   └─ test
>      │   │   │       └─ org/foo/bar2/…
>      │   │   └─ org.foo.bar3
>      │   └─ python
>      │       ├─ foo
>      │       │   ├─ bar1.py
>      │       │   └─ bar2.py
>      │       └─ test_bar1.py
>      └─ MavenModule2
>          ├─ pom.xml
>         ...
>
> A controversial point would be that a single Maven module, say
> MavenModule1, could contain an arbitrary amount of Java 9 or Python
> modules. The pom.xml for each Java 9 modules would either be inferred
> automatically by computing the "intersection" of MavenModule1/pom.xml
> with dependencies declared in the module-info of each Java module, or
> specified explicitly in another way. GroupID and artifactID can be
> inferred automatically too. After install or deploy phase, the JAR +
> pom.xml files would be handled as today's Maven artifact.
>
> This would require modifications of at least the Maven compiler,
> javadoc, jar and deploy plugins (I realize that it would impact all
> plugins, but other plugins could throw some "unsupported directory
> layout" exception for now). I'm willing to experiment such layout and
> Maven plugins modifications in the following months, because after 4
> months of thinking I found no other satisfying solution except switching
> to another build system. But comments are appreciated, and I would be
> glad to propose the results back to Maven if the community consider them
> acceptable.
>
> Any though?
>
>     Martin
>
> [1] https://github.com/Geomatys/maven-plugins/wiki

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]