Avoid guava version conflict with hbase by shading

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Avoid guava version conflict with hbase by shading

Debraj Manna
I have posted more details in stackoverflow
<https://stackoverflow.com/questions/48140339/guava-23-5-conflict-with-hbase-testing-util-1-2>.
I could not add all the details here because of size limitation.

In a project I am using Guava 23.5 but some of the dependencies (hadoop &
hbase) are using old Guava 14. This is causing an exception during runtime

As mentioned here <https://www.elastic.co/blog/to-shade-or-not-to-shade> I
tried to shade the Hbase dependency in a new module named shadedcdh. pom.xml
looks like below

?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>main</artifactId>
        <groupId>com.vnera</groupId>
        <version>0.001-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>shaded-cdh</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-testing-util</artifactId>
            <version>1.2.0-cdh5.7.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <!-- This must stay in sync with hbase version we deploy. -->
            <version>1.2.0-cdh5.7.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.4.1</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <relocations>
                                <relocation>
                                    <pattern>com.google.common</pattern>

<shadedPattern>shaded.com.google.common</shadedPattern>
                                </relocation>
                            </relocations>
                            <transformers>
                                <transformer
implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"
/>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
        </repository>
    </repositories>
</project>

I excluded hbase and hadoop dependency from my project and added
shadedcdh as dependency. But this is still giving me the same
exception. The dependency tree I have posted in the stackoverflow. I
could not post here because of size limitation. Can someone let me
know how can I avoid the conflict?
Reply | Threaded
Open this post in threaded view
|

Re: Avoid guava version conflict with hbase by shading

Mark Prins
It seems that the guava versions are not API compatible, so shading is
unlikely to help. You will need to downgrade to a compatible version of
guava or get the other projects to upgrade.
-M

Op 20 jan. 2018 07:51 schreef "Debraj Manna" <[hidden email]>:

I have posted more details in stackoverflow
<https://stackoverflow.com/questions/48140339/guava-23-5-
conflict-with-hbase-testing-util-1-2>.
I could not add all the details here because of size limitation.

In a project I am using Guava 23.5 but some of the dependencies (hadoop &
hbase) are using old Guava 14. This is causing an exception during runtime

As mentioned here <https://www.elastic.co/blog/to-shade-or-not-to-shade> I
tried to shade the Hbase dependency in a new module named shadedcdh. pom.xml
looks like below

?xml version="1.0" encoding="UTF-8"?>

<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>main</artifactId>
        <groupId>com.vnera</groupId>
        <version>0.001-SNAPSHOT</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>shaded-cdh</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-testing-util</artifactId>
            <version>1.2.0-cdh5.7.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>${hadoop.version}</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hbase</groupId>
            <artifactId>hbase-client</artifactId>
            <!-- This must stay in sync with hbase version we deploy. -->
            <version>1.2.0-cdh5.7.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>2.4.1</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <relocations>
                                <relocation>
                                    <pattern>com.google.common</pattern>

<shadedPattern>shaded.com.google.common</shadedPattern>
                                </relocation>
                            </relocations>
                            <transformers>
                                <transformer
implementation="org.apache.maven.plugins.shade.resource.
ManifestResourceTransformer"
/>
                            </transformers>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <repositories>
        <repository>
            <id>cloudera</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/
</url>
        </repository>
    </repositories>
</project>

I excluded hbase and hadoop dependency from my project and added
shadedcdh as dependency. But this is still giving me the same
exception. The dependency tree I have posted in the stackoverflow. I
could not post here because of size limitation. Can someone let me
know how can I avoid the conflict?
Reply | Threaded
Open this post in threaded view
|

Re: Avoid guava version conflict with hbase by shading

Debraj Manna
Mark

hbase and hadoop is using the old guava. Is there a way I can use the
latest guava in my code and let hadoop and hbase use the old guava.

On 20-Jan-2018 1:19 PM, "Mark Prins" <[hidden email]> wrote:

> It seems that the guava versions are not API compatible, so shading is
> unlikely to help. You will need to downgrade to a compatible version of
> guava or get the other projects to upgrade.
> -M
>
> Op 20 jan. 2018 07:51 schreef "Debraj Manna" <[hidden email]>:
>
> I have posted more details in stackoverflow
> <https://stackoverflow.com/questions/48140339/guava-23-5-
> conflict-with-hbase-testing-util-1-2>.
> I could not add all the details here because of size limitation.
>
> In a project I am using Guava 23.5 but some of the dependencies (hadoop &
> hbase) are using old Guava 14. This is causing an exception during runtime
>
> As mentioned here <https://www.elastic.co/blog/to-shade-or-not-to-shade> I
> tried to shade the Hbase dependency in a new module named shadedcdh.
> pom.xml
> looks like below
>
> ?xml version="1.0" encoding="UTF-8"?>
>
> <project xmlns="http://maven.apache.org/POM/4.0.0"
>          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> http://maven.apache.org/xsd/maven-4.0.0.xsd">
>     <parent>
>         <artifactId>main</artifactId>
>         <groupId>com.vnera</groupId>
>         <version>0.001-SNAPSHOT</version>
>     </parent>
>     <modelVersion>4.0.0</modelVersion>
>
>     <artifactId>shaded-cdh</artifactId>
>     <packaging>jar</packaging>
>
>     <dependencies>
>         <dependency>
>             <groupId>org.apache.hbase</groupId>
>             <artifactId>hbase-testing-util</artifactId>
>             <version>1.2.0-cdh5.7.0</version>
>         </dependency>
>         <dependency>
>             <groupId>org.apache.hadoop</groupId>
>             <artifactId>hadoop-client</artifactId>
>             <version>${hadoop.version}</version>
>         </dependency>
>         <dependency>
>             <groupId>org.apache.hadoop</groupId>
>             <artifactId>hadoop-common</artifactId>
>             <version>${hadoop.version}</version>
>         </dependency>
>         <dependency>
>             <groupId>org.apache.hbase</groupId>
>             <artifactId>hbase-client</artifactId>
>             <!-- This must stay in sync with hbase version we deploy. -->
>             <version>1.2.0-cdh5.7.0</version>
>         </dependency>
>     </dependencies>
>
>     <build>
>         <plugins>
>             <plugin>
>                 <groupId>org.apache.maven.plugins</groupId>
>                 <artifactId>maven-shade-plugin</artifactId>
>                 <version>2.4.1</version>
>                 <executions>
>                     <execution>
>                         <phase>package</phase>
>                         <goals>
>                             <goal>shade</goal>
>                         </goals>
>                         <configuration>
>                             <relocations>
>                                 <relocation>
>                                     <pattern>com.google.common</pattern>
>
> <shadedPattern>shaded.com.google.common</shadedPattern>
>                                 </relocation>
>                             </relocations>
>                             <transformers>
>                                 <transformer
> implementation="org.apache.maven.plugins.shade.resource.
> ManifestResourceTransformer"
> />
>                             </transformers>
>                         </configuration>
>                     </execution>
>                 </executions>
>             </plugin>
>         </plugins>
>     </build>
>
>     <repositories>
>         <repository>
>             <id>cloudera</id>
>             <url>https://repository.cloudera.com/artifactory/
> cloudera-repos/
> </url>
>         </repository>
>     </repositories>
> </project>
>
> I excluded hbase and hadoop dependency from my project and added
> shadedcdh as dependency. But this is still giving me the same
> exception. The dependency tree I have posted in the stackoverflow. I
> could not post here because of size limitation. Can someone let me
> know how can I avoid the conflict?
>
Reply | Threaded
Open this post in threaded view
|

Re: Avoid guava version conflict with hbase by shading

Mark Prins
You could try shading the new guava to a different package and use that in
your code.
E.g. jena-shaded-guava does that

Op 20 jan. 2018 09:02 schreef "Debraj Manna" <[hidden email]>:

> Mark
>
> hbase and hadoop is using the old guava. Is there a way I can use the
> latest guava in my code and let hadoop and hbase use the old guava.
>
> On 20-Jan-2018 1:19 PM, "Mark Prins" <[hidden email]> wrote:
>
> > It seems that the guava versions are not API compatible, so shading is
> > unlikely to help. You will need to downgrade to a compatible version of
> > guava or get the other projects to upgrade.
> > -M
> >
> > Op 20 jan. 2018 07:51 schreef "Debraj Manna" <[hidden email]>:
> >
> > I have posted more details in stackoverflow
> > <https://stackoverflow.com/questions/48140339/guava-23-5-
> > conflict-with-hbase-testing-util-1-2>.
> > I could not add all the details here because of size limitation.
> >
> > In a project I am using Guava 23.5 but some of the dependencies (hadoop &
> > hbase) are using old Guava 14. This is causing an exception during
> runtime
> >
> > As mentioned here <https://www.elastic.co/blog/to-shade-or-not-to-shade>
> I
> > tried to shade the Hbase dependency in a new module named shadedcdh.
> > pom.xml
> > looks like below
> >
> > ?xml version="1.0" encoding="UTF-8"?>
> >
> > <project xmlns="http://maven.apache.org/POM/4.0.0"
> >          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> > http://maven.apache.org/xsd/maven-4.0.0.xsd">
> >     <parent>
> >         <artifactId>main</artifactId>
> >         <groupId>com.vnera</groupId>
> >         <version>0.001-SNAPSHOT</version>
> >     </parent>
> >     <modelVersion>4.0.0</modelVersion>
> >
> >     <artifactId>shaded-cdh</artifactId>
> >     <packaging>jar</packaging>
> >
> >     <dependencies>
> >         <dependency>
> >             <groupId>org.apache.hbase</groupId>
> >             <artifactId>hbase-testing-util</artifactId>
> >             <version>1.2.0-cdh5.7.0</version>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hadoop</groupId>
> >             <artifactId>hadoop-client</artifactId>
> >             <version>${hadoop.version}</version>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hadoop</groupId>
> >             <artifactId>hadoop-common</artifactId>
> >             <version>${hadoop.version}</version>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hbase</groupId>
> >             <artifactId>hbase-client</artifactId>
> >             <!-- This must stay in sync with hbase version we deploy. -->
> >             <version>1.2.0-cdh5.7.0</version>
> >         </dependency>
> >     </dependencies>
> >
> >     <build>
> >         <plugins>
> >             <plugin>
> >                 <groupId>org.apache.maven.plugins</groupId>
> >                 <artifactId>maven-shade-plugin</artifactId>
> >                 <version>2.4.1</version>
> >                 <executions>
> >                     <execution>
> >                         <phase>package</phase>
> >                         <goals>
> >                             <goal>shade</goal>
> >                         </goals>
> >                         <configuration>
> >                             <relocations>
> >                                 <relocation>
> >                                     <pattern>com.google.common</pattern>
> >
> > <shadedPattern>shaded.com.google.common</shadedPattern>
> >                                 </relocation>
> >                             </relocations>
> >                             <transformers>
> >                                 <transformer
> > implementation="org.apache.maven.plugins.shade.resource.
> > ManifestResourceTransformer"
> > />
> >                             </transformers>
> >                         </configuration>
> >                     </execution>
> >                 </executions>
> >             </plugin>
> >         </plugins>
> >     </build>
> >
> >     <repositories>
> >         <repository>
> >             <id>cloudera</id>
> >             <url>https://repository.cloudera.com/artifactory/
> > cloudera-repos/
> > </url>
> >         </repository>
> >     </repositories>
> > </project>
> >
> > I excluded hbase and hadoop dependency from my project and added
> > shadedcdh as dependency. But this is still giving me the same
> > exception. The dependency tree I have posted in the stackoverflow. I
> > could not post here because of size limitation. Can someone let me
> > know how can I avoid the conflict?
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Avoid guava version conflict with hbase by shading

Debraj Manna
Ok. But then I think I have to change the import of guava in all the places
in my code?

On Sat, Jan 20, 2018 at 1:44 PM, Mark Prins <[hidden email]> wrote:

> You could try shading the new guava to a different package and use that in
> your code.
> E.g. jena-shaded-guava does that
>
> Op 20 jan. 2018 09:02 schreef "Debraj Manna" <[hidden email]>:
>
> > Mark
> >
> > hbase and hadoop is using the old guava. Is there a way I can use the
> > latest guava in my code and let hadoop and hbase use the old guava.
> >
> > On 20-Jan-2018 1:19 PM, "Mark Prins" <[hidden email]> wrote:
> >
> > > It seems that the guava versions are not API compatible, so shading is
> > > unlikely to help. You will need to downgrade to a compatible version of
> > > guava or get the other projects to upgrade.
> > > -M
> > >
> > > Op 20 jan. 2018 07:51 schreef "Debraj Manna" <[hidden email]
> >:
> > >
> > > I have posted more details in stackoverflow
> > > <https://stackoverflow.com/questions/48140339/guava-23-5-
> > > conflict-with-hbase-testing-util-1-2>.
> > > I could not add all the details here because of size limitation.
> > >
> > > In a project I am using Guava 23.5 but some of the dependencies
> (hadoop &
> > > hbase) are using old Guava 14. This is causing an exception during
> > runtime
> > >
> > > As mentioned here <https://www.elastic.co/blog/
> to-shade-or-not-to-shade>
> > I
> > > tried to shade the Hbase dependency in a new module named shadedcdh.
> > > pom.xml
> > > looks like below
> > >
> > > ?xml version="1.0" encoding="UTF-8"?>
> > >
> > > <project xmlns="http://maven.apache.org/POM/4.0.0"
> > >          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> > >          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> > > http://maven.apache.org/xsd/maven-4.0.0.xsd">
> > >     <parent>
> > >         <artifactId>main</artifactId>
> > >         <groupId>com.vnera</groupId>
> > >         <version>0.001-SNAPSHOT</version>
> > >     </parent>
> > >     <modelVersion>4.0.0</modelVersion>
> > >
> > >     <artifactId>shaded-cdh</artifactId>
> > >     <packaging>jar</packaging>
> > >
> > >     <dependencies>
> > >         <dependency>
> > >             <groupId>org.apache.hbase</groupId>
> > >             <artifactId>hbase-testing-util</artifactId>
> > >             <version>1.2.0-cdh5.7.0</version>
> > >         </dependency>
> > >         <dependency>
> > >             <groupId>org.apache.hadoop</groupId>
> > >             <artifactId>hadoop-client</artifactId>
> > >             <version>${hadoop.version}</version>
> > >         </dependency>
> > >         <dependency>
> > >             <groupId>org.apache.hadoop</groupId>
> > >             <artifactId>hadoop-common</artifactId>
> > >             <version>${hadoop.version}</version>
> > >         </dependency>
> > >         <dependency>
> > >             <groupId>org.apache.hbase</groupId>
> > >             <artifactId>hbase-client</artifactId>
> > >             <!-- This must stay in sync with hbase version we deploy.
> -->
> > >             <version>1.2.0-cdh5.7.0</version>
> > >         </dependency>
> > >     </dependencies>
> > >
> > >     <build>
> > >         <plugins>
> > >             <plugin>
> > >                 <groupId>org.apache.maven.plugins</groupId>
> > >                 <artifactId>maven-shade-plugin</artifactId>
> > >                 <version>2.4.1</version>
> > >                 <executions>
> > >                     <execution>
> > >                         <phase>package</phase>
> > >                         <goals>
> > >                             <goal>shade</goal>
> > >                         </goals>
> > >                         <configuration>
> > >                             <relocations>
> > >                                 <relocation>
> > >                                     <pattern>com.google.common</
> pattern>
> > >
> > > <shadedPattern>shaded.com.google.common</shadedPattern>
> > >                                 </relocation>
> > >                             </relocations>
> > >                             <transformers>
> > >                                 <transformer
> > > implementation="org.apache.maven.plugins.shade.resource.
> > > ManifestResourceTransformer"
> > > />
> > >                             </transformers>
> > >                         </configuration>
> > >                     </execution>
> > >                 </executions>
> > >             </plugin>
> > >         </plugins>
> > >     </build>
> > >
> > >     <repositories>
> > >         <repository>
> > >             <id>cloudera</id>
> > >             <url>https://repository.cloudera.com/artifactory/
> > > cloudera-repos/
> > > </url>
> > >         </repository>
> > >     </repositories>
> > > </project>
> > >
> > > I excluded hbase and hadoop dependency from my project and added
> > > shadedcdh as dependency. But this is still giving me the same
> > > exception. The dependency tree I have posted in the stackoverflow. I
> > > could not post here because of size limitation. Can someone let me
> > > know how can I avoid the conflict?
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Avoid guava version conflict with hbase by shading

Mark Prins
That's the idea, you cannot otherwise control which classes or which
version of guava are loaded by the classloader at runtime.

Op 20 jan. 2018 14:55 schreef "Debraj Manna" <[hidden email]>:

Ok. But then I think I have to change the import of guava in all the places
in my code?

On Sat, Jan 20, 2018 at 1:44 PM, Mark Prins <[hidden email]> wrote:

> You could try shading the new guava to a different package and use that in
> your code.
> E.g. jena-shaded-guava does that
>
> Op 20 jan. 2018 09:02 schreef "Debraj Manna" <[hidden email]>:
>
> > Mark
> >
> > hbase and hadoop is using the old guava. Is there a way I can use the
> > latest guava in my code and let hadoop and hbase use the old guava.
> >
> > On 20-Jan-2018 1:19 PM, "Mark Prins" <[hidden email]> wrote:
> >
> > > It seems that the guava versions are not API compatible, so shading is
> > > unlikely to help. You will need to downgrade to a compatible version
of

> > > guava or get the other projects to upgrade.
> > > -M
> > >
> > > Op 20 jan. 2018 07:51 schreef "Debraj Manna" <[hidden email]
> >:
> > >
> > > I have posted more details in stackoverflow
> > > <https://stackoverflow.com/questions/48140339/guava-23-5-
> > > conflict-with-hbase-testing-util-1-2>.
> > > I could not add all the details here because of size limitation.
> > >
> > > In a project I am using Guava 23.5 but some of the dependencies
> (hadoop &
> > > hbase) are using old Guava 14. This is causing an exception during
> > runtime
> > >
> > > As mentioned here <https://www.elastic.co/blog/
> to-shade-or-not-to-shade>
> > I
> > > tried to shade the Hbase dependency in a new module named shadedcdh.
> > > pom.xml
> > > looks like below
> > >
> > > ?xml version="1.0" encoding="UTF-8"?>
> > >
> > > <project xmlns="http://maven.apache.org/POM/4.0.0"
> > >          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> > >          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> > > http://maven.apache.org/xsd/maven-4.0.0.xsd">
> > >     <parent>
> > >         <artifactId>main</artifactId>
> > >         <groupId>com.vnera</groupId>
> > >         <version>0.001-SNAPSHOT</version>
> > >     </parent>
> > >     <modelVersion>4.0.0</modelVersion>
> > >
> > >     <artifactId>shaded-cdh</artifactId>
> > >     <packaging>jar</packaging>
> > >
> > >     <dependencies>
> > >         <dependency>
> > >             <groupId>org.apache.hbase</groupId>
> > >             <artifactId>hbase-testing-util</artifactId>
> > >             <version>1.2.0-cdh5.7.0</version>
> > >         </dependency>
> > >         <dependency>
> > >             <groupId>org.apache.hadoop</groupId>
> > >             <artifactId>hadoop-client</artifactId>
> > >             <version>${hadoop.version}</version>
> > >         </dependency>
> > >         <dependency>
> > >             <groupId>org.apache.hadoop</groupId>
> > >             <artifactId>hadoop-common</artifactId>
> > >             <version>${hadoop.version}</version>
> > >         </dependency>
> > >         <dependency>
> > >             <groupId>org.apache.hbase</groupId>
> > >             <artifactId>hbase-client</artifactId>
> > >             <!-- This must stay in sync with hbase version we deploy.
> -->
> > >             <version>1.2.0-cdh5.7.0</version>
> > >         </dependency>
> > >     </dependencies>
> > >
> > >     <build>
> > >         <plugins>
> > >             <plugin>
> > >                 <groupId>org.apache.maven.plugins</groupId>
> > >                 <artifactId>maven-shade-plugin</artifactId>
> > >                 <version>2.4.1</version>
> > >                 <executions>
> > >                     <execution>
> > >                         <phase>package</phase>
> > >                         <goals>
> > >                             <goal>shade</goal>
> > >                         </goals>
> > >                         <configuration>
> > >                             <relocations>
> > >                                 <relocation>
> > >                                     <pattern>com.google.common</
> pattern>
> > >
> > > <shadedPattern>shaded.com.google.common</shadedPattern>
> > >                                 </relocation>
> > >                             </relocations>
> > >                             <transformers>
> > >                                 <transformer
> > > implementation="org.apache.maven.plugins.shade.resource.
> > > ManifestResourceTransformer"
> > > />
> > >                             </transformers>
> > >                         </configuration>
> > >                     </execution>
> > >                 </executions>
> > >             </plugin>
> > >         </plugins>
> > >     </build>
> > >
> > >     <repositories>
> > >         <repository>
> > >             <id>cloudera</id>
> > >             <url>https://repository.cloudera.com/artifactory/
> > > cloudera-repos/
> > > </url>
> > >         </repository>
> > >     </repositories>
> > > </project>
> > >
> > > I excluded hbase and hadoop dependency from my project and added
> > > shadedcdh as dependency. But this is still giving me the same
> > > exception. The dependency tree I have posted in the stackoverflow. I
> > > could not post here because of size limitation. Can someone let me
> > > know how can I avoid the conflict?
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Avoid guava version conflict with hbase by shading

Debraj Manna
Mark

One more query if some other dependency (let's say elasticsearch for
example) leaks its usage of guava then it may happen I end up again with
two incompatible versions of guava. What is the recommended way of handling
this?

Thanks,


On Sun, Jan 21, 2018 at 3:19 AM, Mark Prins <[hidden email]> wrote:

> That's the idea, you cannot otherwise control which classes or which
> version of guava are loaded by the classloader at runtime.
>
> Op 20 jan. 2018 14:55 schreef "Debraj Manna" <[hidden email]>:
>
> Ok. But then I think I have to change the import of guava in all the places
> in my code?
>
> On Sat, Jan 20, 2018 at 1:44 PM, Mark Prins <[hidden email]> wrote:
>
> > You could try shading the new guava to a different package and use that
> in
> > your code.
> > E.g. jena-shaded-guava does that
> >
> > Op 20 jan. 2018 09:02 schreef "Debraj Manna" <[hidden email]>:
> >
> > > Mark
> > >
> > > hbase and hadoop is using the old guava. Is there a way I can use the
> > > latest guava in my code and let hadoop and hbase use the old guava.
> > >
> > > On 20-Jan-2018 1:19 PM, "Mark Prins" <[hidden email]> wrote:
> > >
> > > > It seems that the guava versions are not API compatible, so shading
> is
> > > > unlikely to help. You will need to downgrade to a compatible version
> of
> > > > guava or get the other projects to upgrade.
> > > > -M
> > > >
> > > > Op 20 jan. 2018 07:51 schreef "Debraj Manna" <
> [hidden email]
> > >:
> > > >
> > > > I have posted more details in stackoverflow
> > > > <https://stackoverflow.com/questions/48140339/guava-23-5-
> > > > conflict-with-hbase-testing-util-1-2>.
> > > > I could not add all the details here because of size limitation.
> > > >
> > > > In a project I am using Guava 23.5 but some of the dependencies
> > (hadoop &
> > > > hbase) are using old Guava 14. This is causing an exception during
> > > runtime
> > > >
> > > > As mentioned here <https://www.elastic.co/blog/
> > to-shade-or-not-to-shade>
> > > I
> > > > tried to shade the Hbase dependency in a new module named shadedcdh.
> > > > pom.xml
> > > > looks like below
> > > >
> > > > ?xml version="1.0" encoding="UTF-8"?>
> > > >
> > > > <project xmlns="http://maven.apache.org/POM/4.0.0"
> > > >          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> > > >          xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
> > > > http://maven.apache.org/xsd/maven-4.0.0.xsd">
> > > >     <parent>
> > > >         <artifactId>main</artifactId>
> > > >         <groupId>com.vnera</groupId>
> > > >         <version>0.001-SNAPSHOT</version>
> > > >     </parent>
> > > >     <modelVersion>4.0.0</modelVersion>
> > > >
> > > >     <artifactId>shaded-cdh</artifactId>
> > > >     <packaging>jar</packaging>
> > > >
> > > >     <dependencies>
> > > >         <dependency>
> > > >             <groupId>org.apache.hbase</groupId>
> > > >             <artifactId>hbase-testing-util</artifactId>
> > > >             <version>1.2.0-cdh5.7.0</version>
> > > >         </dependency>
> > > >         <dependency>
> > > >             <groupId>org.apache.hadoop</groupId>
> > > >             <artifactId>hadoop-client</artifactId>
> > > >             <version>${hadoop.version}</version>
> > > >         </dependency>
> > > >         <dependency>
> > > >             <groupId>org.apache.hadoop</groupId>
> > > >             <artifactId>hadoop-common</artifactId>
> > > >             <version>${hadoop.version}</version>
> > > >         </dependency>
> > > >         <dependency>
> > > >             <groupId>org.apache.hbase</groupId>
> > > >             <artifactId>hbase-client</artifactId>
> > > >             <!-- This must stay in sync with hbase version we deploy.
> > -->
> > > >             <version>1.2.0-cdh5.7.0</version>
> > > >         </dependency>
> > > >     </dependencies>
> > > >
> > > >     <build>
> > > >         <plugins>
> > > >             <plugin>
> > > >                 <groupId>org.apache.maven.plugins</groupId>
> > > >                 <artifactId>maven-shade-plugin</artifactId>
> > > >                 <version>2.4.1</version>
> > > >                 <executions>
> > > >                     <execution>
> > > >                         <phase>package</phase>
> > > >                         <goals>
> > > >                             <goal>shade</goal>
> > > >                         </goals>
> > > >                         <configuration>
> > > >                             <relocations>
> > > >                                 <relocation>
> > > >                                     <pattern>com.google.common</
> > pattern>
> > > >
> > > > <shadedPattern>shaded.com.google.common</shadedPattern>
> > > >                                 </relocation>
> > > >                             </relocations>
> > > >                             <transformers>
> > > >                                 <transformer
> > > > implementation="org.apache.maven.plugins.shade.resource.
> > > > ManifestResourceTransformer"
> > > > />
> > > >                             </transformers>
> > > >                         </configuration>
> > > >                     </execution>
> > > >                 </executions>
> > > >             </plugin>
> > > >         </plugins>
> > > >     </build>
> > > >
> > > >     <repositories>
> > > >         <repository>
> > > >             <id>cloudera</id>
> > > >             <url>https://repository.cloudera.com/artifactory/
> > > > cloudera-repos/
> > > > </url>
> > > >         </repository>
> > > >     </repositories>
> > > > </project>
> > > >
> > > > I excluded hbase and hadoop dependency from my project and added
> > > > shadedcdh as dependency. But this is still giving me the same
> > > > exception. The dependency tree I have posted in the stackoverflow. I
> > > > could not post here because of size limitation. Can someone let me
> > > > know how can I avoid the conflict?
> > > >
> > >
> >
>
Reply | Threaded
Open this post in threaded view
|

Re: Avoid guava version conflict with hbase by shading

Mark Prins
On 21-01-18 08:21, Debraj Manna wrote:
> Mark
>
> One more query if some other dependency (let's say elasticsearch for
> example) leaks its usage of guava then it may happen I end up again with
> two incompatible versions of guava. What is the recommended way of handling
> this?

You would need to find and settle on a guava version that is compatible
between all the versions or get the other projects to release updated
versions or make a version by yourself (of hbase or elasticsearch or...)
that is compatible

-M


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]