Quantcast

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
Falko Modler created MCOMPILER-170:
--------------------------------------

             Summary: Regression: Compiler Plugin fails when building with multiple threads (-T...)
                 Key: MCOMPILER-170
                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
             Project: Maven 2.x Compiler Plugin
          Issue Type: Bug
    Affects Versions: 2.4
         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
            Reporter: Falko Modler
            Priority: Critical


I just tried building my current project which is rather large and has many sub-modules.

With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
{{mvn clean compile -T4}}

Version 2.4 fails with random errors when using this command line!
Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.

A single-threaded build with:
{{mvn clean compile}}
*completes just fine*!

Unfortunately I cannot upload my project for copyright reasons.

So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297882#comment-297882 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

forking compiler can be a solution mojo configuration or -Dmaven.compiler.fork=true
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297882#comment-297882 ]

Olivier Lamy edited comment on MCOMPILER-170 at 5/4/12 12:07 PM:
-----------------------------------------------------------------

The unique compiler instance used is not thread safe (maybe we could ask sun/oracle to fix that :-) )
forking compiler can be a solution: use mojo configuration or -Dmaven.compiler.fork=true

               
      was (Author: olamy):
    forking compiler can be a solution mojo configuration or -Dmaven.compiler.fork=true
                 

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297884#comment-297884 ]

Falko Modler commented on MCOMPILER-170:
----------------------------------------

Thanks for the pointer, but I don't know whether I understood you correctly:

You mean this is a general problem and it is not specific for the changes that have been made for 2.4?
But why does everything work as expected in 2.3.2 then?

There was also no note of switching on the forking when @threadSafe was introduced, see: MCOMPILER-127

Btw: Switching on forking does indeed work around this issue but it also slows down the build a bit. And of course it can't benefit from MCOMPILER-166 either (which was the main reason for me the try 2.4).
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297884#comment-297884 ]

Falko Modler edited comment on MCOMPILER-170 at 5/4/12 12:23 PM:
-----------------------------------------------------------------

Thanks for the pointer, but I don't know whether I understood you correctly:

You mean this is a general problem and it is not specific for the changes that have been made for 2.4?
But why does everything work as expected in 2.3.2 then?

There was also no note of switching on the forking when @threadSafe was introduced, see: MCOMPILER-127

Btw: Switching on forking does indeed work around this issue but it also slows down the build a bit. And of course it can't benefit from MCOMPILER-166 either (which was the main reason for me to try 2.4).
               
      was (Author: famod):
    Thanks for the pointer, but I don't know whether I understood you correctly:

You mean this is a general problem and it is not specific for the changes that have been made for 2.4?
But why does everything work as expected in 2.3.2 then?

There was also no note of switching on the forking when @threadSafe was introduced, see: MCOMPILER-127

Btw: Switching on forking does indeed work around this issue but it also slows down the build a bit. And of course it can't benefit from MCOMPILER-166 either (which was the main reason for me the try 2.4).
                 

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297892#comment-297892 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

The optimization introduces by MCOMPILER-166 means we only use 1 instance of Javac class instead of rebuild it for all compilation.
This Javac instance is not thread safe (euh we cannot fix that :-) ).
So sure the compiler plugin is not threadSafe if you don't fork compiler.
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297902#comment-297902 ]

Falko Modler commented on MCOMPILER-170:
----------------------------------------

Ok, so in my opinion that means:

- the plugin is *not threadsafe by default* anymore which means 2.4 breaks backward compatibility to some extend
- the documentation should clearly point out that fork needs to be turned on (default is off/false) when using the parallel build feature
  see also: http://maven.apache.org/plugins/maven-compiler-plugin/compile-mojo.html -> "The goal is thread-safe and supports parallel builds."
- MCOMPILER-166 brings a *performance gain for single threaded builds only* but results in a *performance hit for parallel builds* because the forking causes more overhead

Seriously, medium to large scale projects *will* have many sub-modules (at least that's what I have seen in many projects where I work) so it is more than probable that the parallel build feature will be used. In my current project we save a lot of valuable time by using this feature.
With that in mind, it is rather disappointing that version 2.4. of the plugin will actually slow down parallel builds.

Please reconsider the changes made by MCOMPILER-166!
You could at least introduce a switch to enable/disable the caching introduced by MCOMPILER-166.
Personally, I think this caching should be off by default to ensure backward compatibility for parallel builds.
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297902#comment-297902 ]

Falko Modler edited comment on MCOMPILER-170 at 5/4/12 5:04 PM:
----------------------------------------------------------------

Ok, so in my opinion that means:

- the plugin is *not threadsafe by default* anymore which means 2.4 breaks backward compatibility to some extend
- the documentation should clearly point out that fork needs to be turned on (default is off/false) when using the parallel build feature
  see also: http://maven.apache.org/plugins/maven-compiler-plugin/compile-mojo.html -> "The goal is thread-safe and supports parallel builds."
- MCOMPILER-166 brings a *performance gain for single threaded builds only* but results in a *performance hit for parallel builds* because the forking causes more overhead

Seriously, medium to large scale projects *will* have many sub-modules (at least that's what I have seen in many projects where I work) so it is more than probable that the parallel build feature will be used. In my current project we save a lot of valuable time by using this feature.
With that in mind, it is rather disappointing that version 2.4. of the plugin will actually slow down parallel builds.

Please reconsider the changes made by MCOMPILER-166!
You could at least introduce a switch to enable/disable the caching introduced by MCOMPILER-166.
Personally, I think this caching should be off by default to ensure backward compatibility for parallel builds.

Edit:
Just realized that above-mentioned flag would have to be passed to that "external" plexus component which means this will be harder achieve.
But still - as outlined above - the current situation is really not satisfactory.
               
      was (Author: famod):
    Ok, so in my opinion that means:

- the plugin is *not threadsafe by default* anymore which means 2.4 breaks backward compatibility to some extend
- the documentation should clearly point out that fork needs to be turned on (default is off/false) when using the parallel build feature
  see also: http://maven.apache.org/plugins/maven-compiler-plugin/compile-mojo.html -> "The goal is thread-safe and supports parallel builds."
- MCOMPILER-166 brings a *performance gain for single threaded builds only* but results in a *performance hit for parallel builds* because the forking causes more overhead

Seriously, medium to large scale projects *will* have many sub-modules (at least that's what I have seen in many projects where I work) so it is more than probable that the parallel build feature will be used. In my current project we save a lot of valuable time by using this feature.
With that in mind, it is rather disappointing that version 2.4. of the plugin will actually slow down parallel builds.

Please reconsider the changes made by MCOMPILER-166!
You could at least introduce a switch to enable/disable the caching introduced by MCOMPILER-166.
Personally, I think this caching should be off by default to ensure backward compatibility for parallel builds.
                 

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297952#comment-297952 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

I'd like to introduce something like (pseudo code):
{code}
Enum CompilerStrategy {
  /** default option same today but adding a log warning in console if the current build use -T option */
  ReuseSame();
  /** create a new compiler for each use */
  AlwaysNew();
  /** a kind of pool to reuse created compiler */
  ReuseCreated()
   
}
{code}
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297956#comment-297956 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

uhm to be threadSafe  the ReuseCreated() must be default
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

     [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olivier Lamy updated MCOMPILER-170:
-----------------------------------

    Comment: was deleted

(was: uhm to be threadSafe  the ReuseCreated() must be default )
   

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=297962#comment-297962 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

You can try with compiler plugin 2.5-SNAPSHOT.
The field to configure is compilerReuseStrategy. Possible values are: reuseCreated, reuseSame or alwaysNew
Default value is reuseSame.
reuseCreated strategy will try to reuse previously created Javacc classes.
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

     [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olivier Lamy closed MCOMPILER-170.
----------------------------------

       Resolution: Fixed
    Fix Version/s: 2.5
   

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=298000#comment-298000 ]

Falko Modler commented on MCOMPILER-170:
----------------------------------------

Thanks for the quick response/implementation!

I carried out some tests with 2.5-SNAPSHOT and 2.3.2. Project contains ~160 modules and ~3600 java files. Built with {{mvn clean install}}.

Format: {{<total build time>, <compiler-plugin's proportion of the execution time of all plugins in percent> (<as before but in minutes>)}}
The last two figures have been recorded by a special maven build extension I wrote some time ago.

The results:

*Single-threaded build*

reuseSame
02:53, 75,4% (2:00)

reuseCreated
02:53, 75,5% (2:00)

alwaysNew
03:49, 81,7% (2:56)

2.3.2
03:45, 81,3% (2:52)


*Multi-threaded build (-T4)*

reuseSame
Errors as expected

reuseCreated
01:30, 74,1% (2:52)

alwaysNew
02:00, 82,3% (4:35)

2.3.2
01:57, 81,7% (4:27)


Just for the records: *-Dmaven.compiler.fork=true* with 2.5-SNAPSHOT

Single-threaded
04:20, 84,1% (3:26)

-T4
02:12, 84,0% (5:12)


As you can see, reuseCreated yields the same results for single-threaded builds as reuseSame while still maintaining backwards-compatibility for parallel builds.
So the question is: Do we really need reuseSame? If yes: Why? And in case reuseSame needs to stay, I'm very much in favor of having reuseCreated as the default.
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=298001#comment-298001 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

Thanks for the figures.
I agree reuseCreated and reuseSame are same for single thread build.
But reuseSame works for me on multi thread build with jdk 1.7 on linux or osx (sorry I don't have windauze to test).
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=298002#comment-298002 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

oops my bad I agree reuseCreated must be default. I just made a typo in code :-)
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=298003#comment-298003 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

fixed
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=298005#comment-298005 ]

Falko Modler commented on MCOMPILER-170:
----------------------------------------

Forgot to add that I did not see the warning message mentioned in the description of the ReuseSame enum instance. What is the wording I should look for?

And that reuseSame is working for you on a multi-threaded build might be related to the very "volatile" nature of concurrency problems. Seems like your setup/test just doesn't "provoke" that situation. I don't think it is related to JDK 1.7 but I might be wrong.
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

     [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Olivier Lamy reopened MCOMPILER-170:
------------------------------------


reopen as wording not implemented.

               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

(MCOMPILER-170) Regression: Compiler Plugin fails when building with multiple threads (-T...)

JIRA jira@codehaus.org
In reply to this post by JIRA jira@codehaus.org

    [ https://jira.codehaus.org/browse/MCOMPILER-170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=298007#comment-298007 ]

Olivier Lamy commented on MCOMPILER-170:
----------------------------------------

my test is same build: fail with 1.6 but works with 1.7.
Previously I was using fork compiler.
               

> Regression: Compiler Plugin fails when building with multiple threads (-T...)
> -----------------------------------------------------------------------------
>
>                 Key: MCOMPILER-170
>                 URL: https://jira.codehaus.org/browse/MCOMPILER-170
>             Project: Maven 2.x Compiler Plugin
>          Issue Type: Bug
>    Affects Versions: 2.4
>         Environment: Windows 7 x64, JDK 1.6.0_31, Maven 3.0.4
>            Reporter: Falko Modler
>            Assignee: Olivier Lamy
>            Priority: Critical
>             Fix For: 2.5
>
>
> I just tried building my current project which is rather large and has many sub-modules.
> With version 2.3.2 (and below) of the plugin I was able to build the whole project with multiple threads, let's say:
> {{mvn clean compile -T4}}
> Version 2.4 fails with random errors when using this command line!
> Errors include missing closing brackets, "cannot find symbol" and so on. It's also not always the same module where the errors occur.
> A single-threaded build with:
> {{mvn clean compile}}
> *completes just fine*!
> Unfortunately I cannot upload my project for copyright reasons.
> So for me it looks like a "classic" concurrency problem. Maybe side effect of MCOMPILER-166?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

       
12
Loading...