[GitHub] [maven-resolver] cstamas opened a new pull request #77: SyncContext implementations

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
60 messages Options
123
Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas opened a new pull request #77: SyncContext implementations

GitBox

cstamas opened a new pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77


   Just some scratches for synccontext, and added few implementations and an intermittent layer NamedLock that implements ref counting, for those implementations that needs it.
   
   So "global" synccontext is bad as it is too "coarse", and there is "redisson" synccontext. This PR adds "fine grained" local synccontexts as alternative to global, and Hazelcast ones (and copies redisson as well).
   
   Implemented:
   - JVM ReentrantReadWriteLock using, usable for MT builds
   - JVM Semaphore using, usable for MT builds
   - Hazelcast and Hazelcast client ISemaphore using, usable for MT and MP builds
   - Copy+Pasted Redisson based as well (but no tests), should be usable for MT and MP builds
   
   Note about Hazelcast: currently using _deprecated_ API as well, as HZ itself is in quite flux (in HZ3.x CP subsystem is usable only with 3+ nodes, in HZ4.0.x CP got "unsafe" operation so is usable with less than 3 nodes, but CP objects remains in heap forever as destroyed... so upgrade to latest HZ could be possible when some alternate CP structure is implemented, and code changed to use that maybe, or just avoiding to destroy DOs). Still, same code modified to use CP works if one creates a dedicated HZ cluster with CP enabled and uses hazelcast-client to connect to it.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on pull request #77: SyncContext implementations

GitBox

michael-o commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-722327873


   Wow, this is insane. What was the motivation for this?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on pull request #77: SyncContext implementations

GitBox
In reply to this post by GitBox

michael-o commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-722576829


   I will have a look this week.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] Tibor17 commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

Tibor17 commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-723502847


   It looks like the `ForkJoinPool` may substitute [SyncContext](https://maven.apache.org/resolver/maven-resolver-api/apidocs/org/eclipse/aether/SyncContext.html). If the parent Artifact sees the children > 0 then the parent Thread would fork to children threads. That's the mechanism how the locks are shared in the thread pool but it is already implemented in Java.
   btw. the usecase in `NamedLockFactoryTestSupport.java` looks like the one which exists in Java, I mean `CyclicBarrier` or  `Phaser` .


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

michael-o commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524413810



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/SyncContextFactoryAdapter.java
##########
@@ -0,0 +1,300 @@
+package org.eclipse.aether.internal.named;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.metadata.Metadata;
+import org.eclipse.aether.util.ChecksumUtils;
+import org.eclipse.aether.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Adapter to adapt {@link NamedLockFactory} and {@link NamedLock} to {@link SyncContext}.
+ */
+public final class SyncContextFactoryAdapter
+{
+    private static final String DEFAULT_DISCRIMINATOR_DIGEST = "da39a3ee5e6b4b0d3255bfef95601890afd80709";
+
+    private static final String DEFAULT_HOSTNAME = "localhost";
+
+    private static final Logger LOGGER = LoggerFactory.getLogger( SyncContextFactoryAdapter.class );
+
+    private static final long TIME = Long.getLong(
+        SyncContextFactoryAdapter.class.getName() + ".time", 10L
+    );
+
+    private static final TimeUnit TIME_UNIT = TimeUnit.valueOf( System.getProperty(
+        SyncContextFactoryAdapter.class.getName() + ".timeunit", TimeUnit.SECONDS.name()
+    ) );
+
+    private final NamedLockFactory namedLockFactory;
+
+    private final long time;
+
+    private final TimeUnit timeUnit;
+
+    private final String hostname;
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory )
+    {
+        this( namedLockFactory, TIME, TIME_UNIT );
+    }
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory,
+                                      final long time,
+                                      final TimeUnit timeUnit )
+    {
+        this.namedLockFactory = namedLockFactory;
+        this.time = time;
+        this.timeUnit = timeUnit;
+        this.hostname = getHostname();
+    }
+
+    private String getHostname()
+    {
+        try
+        {
+            return InetAddress.getLocalHost().getHostName();
+        }
+        catch ( UnknownHostException e )
+        {
+            LOGGER.warn( "Failed to get hostname, using '{}'", DEFAULT_HOSTNAME, e );
+            return DEFAULT_HOSTNAME;
+        }
+    }
+
+    public SyncContext newInstance( final RepositorySystemSession session, final boolean shared )
+    {
+        return new AdaptedLockSyncContext( session, hostname, namedLockFactory, time, timeUnit, shared );
+    }
+
+    public void shutdown()
+    {
+        namedLockFactory.shutdown();
+    }
+
+    private static class AdaptedLockSyncContext
+            implements SyncContext
+    {
+        private static final String CONFIG_PROP_DISCRIMINATOR = "aether.syncContext.named.discriminator";
+
+        private static final String KEY_PREFIX = "mvn:resolver:";
+
+        private static final Logger LOGGER = LoggerFactory.getLogger( AdaptedLockSyncContext.class );
+
+        private final RepositorySystemSession session;
+
+        private final String hostname;
+
+        private final NamedLockFactory namedLockFactory;
+
+        private final long time;
+
+        private final TimeUnit timeUnit;
+
+        private final boolean shared;
+
+        private final Map<String, NamedLock> locks = new LinkedHashMap<>();
+
+        private AdaptedLockSyncContext( final RepositorySystemSession session,
+                                        final String hostname,
+                                        final NamedLockFactory namedLockFactory,
+                                        final long time,
+                                        final TimeUnit timeUnit,
+                                        final boolean shared )
+        {
+            this.session = session;
+            this.hostname = hostname;
+            this.namedLockFactory = namedLockFactory;
+            this.time = time;
+            this.timeUnit = timeUnit;
+            this.shared = shared;
+        }
+
+        @Override
+        public void acquire( Collection<? extends Artifact> artifacts,
+                             Collection<? extends Metadata> metadatas )
+        {
+            // Deadlock prevention: https://stackoverflow.com/a/16780988/696632
+            // We must acquire multiple locks always in the same order!
+            Collection<String> keys = new TreeSet<>();
+            if ( artifacts != null )
+            {
+                for ( Artifact artifact : artifacts )
+                {
+                    // TODO Should we include extension and classifier too?
+                    String key = "artifact:" + artifact.getGroupId() + ":"
+                            + artifact.getArtifactId() + ":" + artifact.getBaseVersion();
+                    keys.add( key );
+                }
+            }
+
+            if ( metadatas != null )
+            {
+                for ( Metadata metadata : metadatas )
+                {
+                    StringBuilder key = new StringBuilder( "metadata:" );
+                    if ( !metadata.getGroupId().isEmpty() )
+                    {
+                        key.append( metadata.getGroupId() );
+                        if ( !metadata.getArtifactId().isEmpty() )
+                        {
+                            key.append( ':' ).append( metadata.getArtifactId() );
+                            if ( !metadata.getVersion().isEmpty() )
+                            {
+                                key.append( ':' ).append( metadata.getVersion() );
+                            }
+                        }
+                    }
+                    keys.add( key.toString() );
+                }
+            }
+
+            if ( keys.isEmpty() )
+            {
+                return;
+            }
+
+            String discriminator = createDiscriminator();
+            LOGGER.trace( "Using key discriminator '{}' during this session", discriminator );
+
+            LOGGER.trace( "Need {} {} lock(s) for {}", keys.size(), shared ? "read" : "write", keys );
+            int acquiredLockCount = 0;
+            int reacquiredLockCount = 0;
+            boolean doLock;
+            for ( String key : keys )
+            {
+                NamedLock namedLock = locks.get( key );
+                if ( namedLock == null )
+                {
+                    namedLock = namedLockFactory.getLock( KEY_PREFIX + discriminator + ":" + key );
+                    locks.put( key, namedLock );
+                    acquiredLockCount++;
+                    doLock = true;
+                }
+                else
+                {
+                    reacquiredLockCount++;

Review comment:
       There is no reacquisition anymore because there is no reentrancy used.

##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/synccontext/NamedSyncContextFactory.java
##########
@@ -0,0 +1,84 @@
+package org.eclipse.aether.synccontext;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.impl.SyncContextFactory;
+import org.eclipse.aether.internal.named.NamedLockFactory;
+import org.eclipse.aether.internal.named.SyncContextFactoryAdapter;
+
+import javax.annotation.PreDestroy;
+import javax.annotation.Priority;
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Provider;
+import javax.inject.Singleton;
+
+/**
+ * Named {@link SyncContextFactory} implementation that selects underlying {@link NamedLockFactory} implementation
+ * at creation. Known factory names are:
+ * <ul>
+ *   <li>rwlock-local - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>
+ *   <li>rwlock-redisson - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>

Review comment:
       This documentation seems wrong

##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/synccontext/NamedSyncContextFactory.java
##########
@@ -0,0 +1,84 @@
+package org.eclipse.aether.synccontext;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.impl.SyncContextFactory;
+import org.eclipse.aether.internal.named.NamedLockFactory;
+import org.eclipse.aether.internal.named.SyncContextFactoryAdapter;
+
+import javax.annotation.PreDestroy;
+import javax.annotation.Priority;
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Provider;
+import javax.inject.Singleton;
+
+/**
+ * Named {@link SyncContextFactory} implementation that selects underlying {@link NamedLockFactory} implementation
+ * at creation. Known factory names are:
+ * <ul>
+ *   <li>rwlock-local - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>
+ *   <li>rwlock-redisson - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>
+ *   <li>semaphore-local - uses JVM Semaphore, suitable for MT builds</li>
+ *   <li>semaphore-hazelcast - uses Hazelcast (that would form cluster with other discovered instances),
+ *   suitable for MT and MP builds</li>
+ *   <li>semaphore-hazelcast-client - uses Hazelcast Client (that would connect to some existing cluster),
+ *   suitable for MT and MP builds</li>
+ *   <li>semaphore-cp-hazelcast-client - uses Hazelcast Client (that would connect to some existing cluster),
+ *   suitable for MT and MP builds. Uses CP subsystem to get semaphore and does NOT destroy them.</li>
+ * </ul>
+ * Read Hazelcast documentation how to configure it, but tests does provide some examples.
+ */
+@Named
+@Priority( Integer.MAX_VALUE )
+@Singleton
+public final class NamedSyncContextFactory
+        implements SyncContextFactory
+{
+    private final SyncContextFactoryAdapter syncContextFactoryAdapter;
+
+    @Inject
+    public NamedSyncContextFactory( final Map<String, Provider<NamedLockFactory>> factories )
+    {
+        String name = System.getProperty( "synccontext.named,factory", "local" );

Review comment:
       Here is a typo

##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/impl/AdaptedSemaphoreNamedLock.java
##########
@@ -0,0 +1,115 @@
+package org.eclipse.aether.internal.named.impl;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.internal.named.NamedLockFactorySupport;
+import org.eclipse.aether.internal.named.NamedLockSupport;
+
+import java.util.ArrayDeque;
+import java.util.Deque;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Named lock support implementation that is using "adapted" semaphore (to be able to use semaphores not sharing common
+ * API).
+ */
+public class AdaptedSemaphoreNamedLock
+        extends NamedLockSupport
+{
+    /**
+     * Wrapper for semaphore-like stuff, that do not share common ancestor. Semaphore must be created to support
+     * {@link Integer#MAX_VALUE} permissions.
+     */
+    protected interface AdaptedSemaphore
+    {
+        boolean tryAcquire( int perms, long time, TimeUnit unit ) throws InterruptedException;
+
+        void release( int perms );
+    }
+
+    private final ThreadLocal<Deque<Integer>> threadPerms;
+
+    private final AdaptedSemaphore semaphore;
+
+    public AdaptedSemaphoreNamedLock( final String name,
+                                      final NamedLockFactorySupport factory,
+                                      final AdaptedSemaphore semaphore )
+    {
+        super( name, factory );
+        this.threadPerms = ThreadLocal.withInitial( ArrayDeque::new );
+        this.semaphore = semaphore;
+    }
+
+    @Override
+    public boolean lockShared( final long time, final TimeUnit unit ) throws InterruptedException
+    {
+        Deque<Integer> perms = threadPerms.get();

Review comment:
       How is is locking based on a name?

##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/SyncContextFactoryAdapter.java
##########
@@ -0,0 +1,300 @@
+package org.eclipse.aether.internal.named;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.metadata.Metadata;
+import org.eclipse.aether.util.ChecksumUtils;
+import org.eclipse.aether.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Adapter to adapt {@link NamedLockFactory} and {@link NamedLock} to {@link SyncContext}.
+ */
+public final class SyncContextFactoryAdapter
+{
+    private static final String DEFAULT_DISCRIMINATOR_DIGEST = "da39a3ee5e6b4b0d3255bfef95601890afd80709";
+
+    private static final String DEFAULT_HOSTNAME = "localhost";
+
+    private static final Logger LOGGER = LoggerFactory.getLogger( SyncContextFactoryAdapter.class );
+
+    private static final long TIME = Long.getLong(
+        SyncContextFactoryAdapter.class.getName() + ".time", 10L
+    );
+
+    private static final TimeUnit TIME_UNIT = TimeUnit.valueOf( System.getProperty(
+        SyncContextFactoryAdapter.class.getName() + ".timeunit", TimeUnit.SECONDS.name()
+    ) );
+
+    private final NamedLockFactory namedLockFactory;
+
+    private final long time;
+
+    private final TimeUnit timeUnit;
+
+    private final String hostname;
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory )
+    {
+        this( namedLockFactory, TIME, TIME_UNIT );
+    }
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory,
+                                      final long time,
+                                      final TimeUnit timeUnit )
+    {
+        this.namedLockFactory = namedLockFactory;
+        this.time = time;
+        this.timeUnit = timeUnit;
+        this.hostname = getHostname();
+    }
+
+    private String getHostname()
+    {
+        try
+        {
+            return InetAddress.getLocalHost().getHostName();
+        }
+        catch ( UnknownHostException e )
+        {
+            LOGGER.warn( "Failed to get hostname, using '{}'", DEFAULT_HOSTNAME, e );
+            return DEFAULT_HOSTNAME;
+        }
+    }
+
+    public SyncContext newInstance( final RepositorySystemSession session, final boolean shared )
+    {
+        return new AdaptedLockSyncContext( session, hostname, namedLockFactory, time, timeUnit, shared );
+    }
+
+    public void shutdown()
+    {
+        namedLockFactory.shutdown();
+    }
+
+    private static class AdaptedLockSyncContext
+            implements SyncContext
+    {
+        private static final String CONFIG_PROP_DISCRIMINATOR = "aether.syncContext.named.discriminator";
+
+        private static final String KEY_PREFIX = "mvn:resolver:";
+
+        private static final Logger LOGGER = LoggerFactory.getLogger( AdaptedLockSyncContext.class );
+
+        private final RepositorySystemSession session;
+
+        private final String hostname;
+
+        private final NamedLockFactory namedLockFactory;
+
+        private final long time;
+
+        private final TimeUnit timeUnit;
+
+        private final boolean shared;
+
+        private final Map<String, NamedLock> locks = new LinkedHashMap<>();
+
+        private AdaptedLockSyncContext( final RepositorySystemSession session,
+                                        final String hostname,
+                                        final NamedLockFactory namedLockFactory,
+                                        final long time,
+                                        final TimeUnit timeUnit,
+                                        final boolean shared )
+        {
+            this.session = session;
+            this.hostname = hostname;
+            this.namedLockFactory = namedLockFactory;
+            this.time = time;
+            this.timeUnit = timeUnit;
+            this.shared = shared;
+        }
+
+        @Override
+        public void acquire( Collection<? extends Artifact> artifacts,
+                             Collection<? extends Metadata> metadatas )
+        {
+            // Deadlock prevention: https://stackoverflow.com/a/16780988/696632
+            // We must acquire multiple locks always in the same order!
+            Collection<String> keys = new TreeSet<>();
+            if ( artifacts != null )
+            {
+                for ( Artifact artifact : artifacts )
+                {
+                    // TODO Should we include extension and classifier too?
+                    String key = "artifact:" + artifact.getGroupId() + ":"
+                            + artifact.getArtifactId() + ":" + artifact.getBaseVersion();
+                    keys.add( key );
+                }
+            }
+
+            if ( metadatas != null )
+            {
+                for ( Metadata metadata : metadatas )
+                {
+                    StringBuilder key = new StringBuilder( "metadata:" );
+                    if ( !metadata.getGroupId().isEmpty() )
+                    {
+                        key.append( metadata.getGroupId() );
+                        if ( !metadata.getArtifactId().isEmpty() )
+                        {
+                            key.append( ':' ).append( metadata.getArtifactId() );
+                            if ( !metadata.getVersion().isEmpty() )
+                            {
+                                key.append( ':' ).append( metadata.getVersion() );
+                            }
+                        }
+                    }
+                    keys.add( key.toString() );
+                }
+            }
+
+            if ( keys.isEmpty() )
+            {
+                return;
+            }
+
+            String discriminator = createDiscriminator();
+            LOGGER.trace( "Using key discriminator '{}' during this session", discriminator );
+
+            LOGGER.trace( "Need {} {} lock(s) for {}", keys.size(), shared ? "read" : "write", keys );
+            int acquiredLockCount = 0;
+            int reacquiredLockCount = 0;
+            boolean doLock;
+            for ( String key : keys )
+            {
+                NamedLock namedLock = locks.get( key );
+                if ( namedLock == null )
+                {
+                    namedLock = namedLockFactory.getLock( KEY_PREFIX + discriminator + ":" + key );
+                    locks.put( key, namedLock );
+                    acquiredLockCount++;
+                    doLock = true;
+                }
+                else
+                {
+                    reacquiredLockCount++;
+                    doLock = false;
+                }
+
+                if ( doLock )
+                {
+                    try
+                    {
+                        boolean locked;
+                        if ( shared )
+                        {
+                            locked = namedLock.lockShared( time, timeUnit );
+                        }
+                        else
+                        {
+                            locked = namedLock.lockExclusively( time, timeUnit );
+                        }
+
+                        if ( !locked )
+                        {
+                            throw new IllegalStateException( "Could not lock "
+                                    + namedLock.name() + " (shared=" + shared + ")" );
+                        }
+                    }
+                    catch ( InterruptedException e )
+                    {
+                        Thread.currentThread().interrupt();
+                        throw new RuntimeException( e );
+                    }
+                }
+            }
+            LOGGER.trace( "Total new locks acquired: {}, total existing locks reacquired: {}",

Review comment:
       This is now misleading. I expect a decent lock to be reentrant.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524469609



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/impl/AdaptedSemaphoreNamedLock.java
##########
@@ -0,0 +1,115 @@
+package org.eclipse.aether.internal.named.impl;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.internal.named.NamedLockFactorySupport;
+import org.eclipse.aether.internal.named.NamedLockSupport;
+
+import java.util.ArrayDeque;
+import java.util.Deque;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Named lock support implementation that is using "adapted" semaphore (to be able to use semaphores not sharing common
+ * API).
+ */
+public class AdaptedSemaphoreNamedLock
+        extends NamedLockSupport
+{
+    /**
+     * Wrapper for semaphore-like stuff, that do not share common ancestor. Semaphore must be created to support
+     * {@link Integer#MAX_VALUE} permissions.
+     */
+    protected interface AdaptedSemaphore
+    {
+        boolean tryAcquire( int perms, long time, TimeUnit unit ) throws InterruptedException;
+
+        void release( int perms );
+    }
+
+    private final ThreadLocal<Deque<Integer>> threadPerms;
+
+    private final AdaptedSemaphore semaphore;
+
+    public AdaptedSemaphoreNamedLock( final String name,
+                                      final NamedLockFactorySupport factory,
+                                      final AdaptedSemaphore semaphore )
+    {
+        super( name, factory );
+        this.threadPerms = ThreadLocal.withInitial( ArrayDeque::new );
+        this.semaphore = semaphore;
+    }
+
+    @Override
+    public boolean lockShared( final long time, final TimeUnit unit ) throws InterruptedException
+    {
+        Deque<Integer> perms = threadPerms.get();

Review comment:
       AdaptedSemaphoreNamedLock instance is created per "name" (so instance per name).




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524470046



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/synccontext/NamedSyncContextFactory.java
##########
@@ -0,0 +1,84 @@
+package org.eclipse.aether.synccontext;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.impl.SyncContextFactory;
+import org.eclipse.aether.internal.named.NamedLockFactory;
+import org.eclipse.aether.internal.named.SyncContextFactoryAdapter;
+
+import javax.annotation.PreDestroy;
+import javax.annotation.Priority;
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Provider;
+import javax.inject.Singleton;
+
+/**
+ * Named {@link SyncContextFactory} implementation that selects underlying {@link NamedLockFactory} implementation
+ * at creation. Known factory names are:
+ * <ul>
+ *   <li>rwlock-local - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>
+ *   <li>rwlock-redisson - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>
+ *   <li>semaphore-local - uses JVM Semaphore, suitable for MT builds</li>
+ *   <li>semaphore-hazelcast - uses Hazelcast (that would form cluster with other discovered instances),
+ *   suitable for MT and MP builds</li>
+ *   <li>semaphore-hazelcast-client - uses Hazelcast Client (that would connect to some existing cluster),
+ *   suitable for MT and MP builds</li>
+ *   <li>semaphore-cp-hazelcast-client - uses Hazelcast Client (that would connect to some existing cluster),
+ *   suitable for MT and MP builds. Uses CP subsystem to get semaphore and does NOT destroy them.</li>
+ * </ul>
+ * Read Hazelcast documentation how to configure it, but tests does provide some examples.
+ */
+@Named
+@Priority( Integer.MAX_VALUE )
+@Singleton
+public final class NamedSyncContextFactory
+        implements SyncContextFactory
+{
+    private final SyncContextFactoryAdapter syncContextFactoryAdapter;
+
+    @Inject
+    public NamedSyncContextFactory( final Map<String, Provider<NamedLockFactory>> factories )
+    {
+        String name = System.getProperty( "synccontext.named,factory", "local" );

Review comment:
       Fixing

##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/synccontext/NamedSyncContextFactory.java
##########
@@ -0,0 +1,84 @@
+package org.eclipse.aether.synccontext;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import java.util.Map;
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.impl.SyncContextFactory;
+import org.eclipse.aether.internal.named.NamedLockFactory;
+import org.eclipse.aether.internal.named.SyncContextFactoryAdapter;
+
+import javax.annotation.PreDestroy;
+import javax.annotation.Priority;
+import javax.inject.Inject;
+import javax.inject.Named;
+import javax.inject.Provider;
+import javax.inject.Singleton;
+
+/**
+ * Named {@link SyncContextFactory} implementation that selects underlying {@link NamedLockFactory} implementation
+ * at creation. Known factory names are:
+ * <ul>
+ *   <li>rwlock-local - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>
+ *   <li>rwlock-redisson - uses JVM ReentrantReadWriteLock, suitable for MT builds</li>

Review comment:
       Fixing




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524471145



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/SyncContextFactoryAdapter.java
##########
@@ -0,0 +1,300 @@
+package org.eclipse.aether.internal.named;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.metadata.Metadata;
+import org.eclipse.aether.util.ChecksumUtils;
+import org.eclipse.aether.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Adapter to adapt {@link NamedLockFactory} and {@link NamedLock} to {@link SyncContext}.
+ */
+public final class SyncContextFactoryAdapter
+{
+    private static final String DEFAULT_DISCRIMINATOR_DIGEST = "da39a3ee5e6b4b0d3255bfef95601890afd80709";
+
+    private static final String DEFAULT_HOSTNAME = "localhost";
+
+    private static final Logger LOGGER = LoggerFactory.getLogger( SyncContextFactoryAdapter.class );
+
+    private static final long TIME = Long.getLong(
+        SyncContextFactoryAdapter.class.getName() + ".time", 10L
+    );
+
+    private static final TimeUnit TIME_UNIT = TimeUnit.valueOf( System.getProperty(
+        SyncContextFactoryAdapter.class.getName() + ".timeunit", TimeUnit.SECONDS.name()
+    ) );
+
+    private final NamedLockFactory namedLockFactory;
+
+    private final long time;
+
+    private final TimeUnit timeUnit;
+
+    private final String hostname;
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory )
+    {
+        this( namedLockFactory, TIME, TIME_UNIT );
+    }
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory,
+                                      final long time,
+                                      final TimeUnit timeUnit )
+    {
+        this.namedLockFactory = namedLockFactory;
+        this.time = time;
+        this.timeUnit = timeUnit;
+        this.hostname = getHostname();
+    }
+
+    private String getHostname()
+    {
+        try
+        {
+            return InetAddress.getLocalHost().getHostName();
+        }
+        catch ( UnknownHostException e )
+        {
+            LOGGER.warn( "Failed to get hostname, using '{}'", DEFAULT_HOSTNAME, e );
+            return DEFAULT_HOSTNAME;
+        }
+    }
+
+    public SyncContext newInstance( final RepositorySystemSession session, final boolean shared )
+    {
+        return new AdaptedLockSyncContext( session, hostname, namedLockFactory, time, timeUnit, shared );
+    }
+
+    public void shutdown()
+    {
+        namedLockFactory.shutdown();
+    }
+
+    private static class AdaptedLockSyncContext
+            implements SyncContext
+    {
+        private static final String CONFIG_PROP_DISCRIMINATOR = "aether.syncContext.named.discriminator";
+
+        private static final String KEY_PREFIX = "mvn:resolver:";
+
+        private static final Logger LOGGER = LoggerFactory.getLogger( AdaptedLockSyncContext.class );
+
+        private final RepositorySystemSession session;
+
+        private final String hostname;
+
+        private final NamedLockFactory namedLockFactory;
+
+        private final long time;
+
+        private final TimeUnit timeUnit;
+
+        private final boolean shared;
+
+        private final Map<String, NamedLock> locks = new LinkedHashMap<>();
+
+        private AdaptedLockSyncContext( final RepositorySystemSession session,
+                                        final String hostname,
+                                        final NamedLockFactory namedLockFactory,
+                                        final long time,
+                                        final TimeUnit timeUnit,
+                                        final boolean shared )
+        {
+            this.session = session;
+            this.hostname = hostname;
+            this.namedLockFactory = namedLockFactory;
+            this.time = time;
+            this.timeUnit = timeUnit;
+            this.shared = shared;
+        }
+
+        @Override
+        public void acquire( Collection<? extends Artifact> artifacts,
+                             Collection<? extends Metadata> metadatas )
+        {
+            // Deadlock prevention: https://stackoverflow.com/a/16780988/696632
+            // We must acquire multiple locks always in the same order!
+            Collection<String> keys = new TreeSet<>();
+            if ( artifacts != null )
+            {
+                for ( Artifact artifact : artifacts )
+                {
+                    // TODO Should we include extension and classifier too?
+                    String key = "artifact:" + artifact.getGroupId() + ":"
+                            + artifact.getArtifactId() + ":" + artifact.getBaseVersion();
+                    keys.add( key );
+                }
+            }
+
+            if ( metadatas != null )
+            {
+                for ( Metadata metadata : metadatas )
+                {
+                    StringBuilder key = new StringBuilder( "metadata:" );
+                    if ( !metadata.getGroupId().isEmpty() )
+                    {
+                        key.append( metadata.getGroupId() );
+                        if ( !metadata.getArtifactId().isEmpty() )
+                        {
+                            key.append( ':' ).append( metadata.getArtifactId() );
+                            if ( !metadata.getVersion().isEmpty() )
+                            {
+                                key.append( ':' ).append( metadata.getVersion() );
+                            }
+                        }
+                    }
+                    keys.add( key.toString() );
+                }
+            }
+
+            if ( keys.isEmpty() )
+            {
+                return;
+            }
+
+            String discriminator = createDiscriminator();
+            LOGGER.trace( "Using key discriminator '{}' during this session", discriminator );
+
+            LOGGER.trace( "Need {} {} lock(s) for {}", keys.size(), shared ? "read" : "write", keys );
+            int acquiredLockCount = 0;
+            int reacquiredLockCount = 0;
+            boolean doLock;
+            for ( String key : keys )
+            {
+                NamedLock namedLock = locks.get( key );
+                if ( namedLock == null )
+                {
+                    namedLock = namedLockFactory.getLock( KEY_PREFIX + discriminator + ":" + key );
+                    locks.put( key, namedLock );
+                    acquiredLockCount++;
+                    doLock = true;
+                }
+                else
+                {
+                    reacquiredLockCount++;

Review comment:
       So IIUC, acquire method is no more invoked with Artifact/Metadata that was once invoked with?




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524471892



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/SyncContextFactoryAdapter.java
##########
@@ -0,0 +1,300 @@
+package org.eclipse.aether.internal.named;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.metadata.Metadata;
+import org.eclipse.aether.util.ChecksumUtils;
+import org.eclipse.aether.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Adapter to adapt {@link NamedLockFactory} and {@link NamedLock} to {@link SyncContext}.
+ */
+public final class SyncContextFactoryAdapter
+{
+    private static final String DEFAULT_DISCRIMINATOR_DIGEST = "da39a3ee5e6b4b0d3255bfef95601890afd80709";
+
+    private static final String DEFAULT_HOSTNAME = "localhost";
+
+    private static final Logger LOGGER = LoggerFactory.getLogger( SyncContextFactoryAdapter.class );
+
+    private static final long TIME = Long.getLong(
+        SyncContextFactoryAdapter.class.getName() + ".time", 10L
+    );
+
+    private static final TimeUnit TIME_UNIT = TimeUnit.valueOf( System.getProperty(
+        SyncContextFactoryAdapter.class.getName() + ".timeunit", TimeUnit.SECONDS.name()
+    ) );
+
+    private final NamedLockFactory namedLockFactory;
+
+    private final long time;
+
+    private final TimeUnit timeUnit;
+
+    private final String hostname;
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory )
+    {
+        this( namedLockFactory, TIME, TIME_UNIT );
+    }
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory,
+                                      final long time,
+                                      final TimeUnit timeUnit )
+    {
+        this.namedLockFactory = namedLockFactory;
+        this.time = time;
+        this.timeUnit = timeUnit;
+        this.hostname = getHostname();
+    }
+
+    private String getHostname()
+    {
+        try
+        {
+            return InetAddress.getLocalHost().getHostName();
+        }
+        catch ( UnknownHostException e )
+        {
+            LOGGER.warn( "Failed to get hostname, using '{}'", DEFAULT_HOSTNAME, e );
+            return DEFAULT_HOSTNAME;
+        }
+    }
+
+    public SyncContext newInstance( final RepositorySystemSession session, final boolean shared )
+    {
+        return new AdaptedLockSyncContext( session, hostname, namedLockFactory, time, timeUnit, shared );
+    }
+
+    public void shutdown()
+    {
+        namedLockFactory.shutdown();
+    }
+
+    private static class AdaptedLockSyncContext
+            implements SyncContext
+    {
+        private static final String CONFIG_PROP_DISCRIMINATOR = "aether.syncContext.named.discriminator";
+
+        private static final String KEY_PREFIX = "mvn:resolver:";
+
+        private static final Logger LOGGER = LoggerFactory.getLogger( AdaptedLockSyncContext.class );
+
+        private final RepositorySystemSession session;
+
+        private final String hostname;
+
+        private final NamedLockFactory namedLockFactory;
+
+        private final long time;
+
+        private final TimeUnit timeUnit;
+
+        private final boolean shared;
+
+        private final Map<String, NamedLock> locks = new LinkedHashMap<>();
+
+        private AdaptedLockSyncContext( final RepositorySystemSession session,
+                                        final String hostname,
+                                        final NamedLockFactory namedLockFactory,
+                                        final long time,
+                                        final TimeUnit timeUnit,
+                                        final boolean shared )
+        {
+            this.session = session;
+            this.hostname = hostname;
+            this.namedLockFactory = namedLockFactory;
+            this.time = time;
+            this.timeUnit = timeUnit;
+            this.shared = shared;
+        }
+
+        @Override
+        public void acquire( Collection<? extends Artifact> artifacts,
+                             Collection<? extends Metadata> metadatas )
+        {
+            // Deadlock prevention: https://stackoverflow.com/a/16780988/696632
+            // We must acquire multiple locks always in the same order!
+            Collection<String> keys = new TreeSet<>();
+            if ( artifacts != null )
+            {
+                for ( Artifact artifact : artifacts )
+                {
+                    // TODO Should we include extension and classifier too?
+                    String key = "artifact:" + artifact.getGroupId() + ":"
+                            + artifact.getArtifactId() + ":" + artifact.getBaseVersion();
+                    keys.add( key );
+                }
+            }
+
+            if ( metadatas != null )
+            {
+                for ( Metadata metadata : metadatas )
+                {
+                    StringBuilder key = new StringBuilder( "metadata:" );
+                    if ( !metadata.getGroupId().isEmpty() )
+                    {
+                        key.append( metadata.getGroupId() );
+                        if ( !metadata.getArtifactId().isEmpty() )
+                        {
+                            key.append( ':' ).append( metadata.getArtifactId() );
+                            if ( !metadata.getVersion().isEmpty() )
+                            {
+                                key.append( ':' ).append( metadata.getVersion() );
+                            }
+                        }
+                    }
+                    keys.add( key.toString() );
+                }
+            }
+
+            if ( keys.isEmpty() )
+            {
+                return;
+            }
+
+            String discriminator = createDiscriminator();
+            LOGGER.trace( "Using key discriminator '{}' during this session", discriminator );
+
+            LOGGER.trace( "Need {} {} lock(s) for {}", keys.size(), shared ? "read" : "write", keys );
+            int acquiredLockCount = 0;
+            int reacquiredLockCount = 0;
+            boolean doLock;
+            for ( String key : keys )
+            {
+                NamedLock namedLock = locks.get( key );
+                if ( namedLock == null )
+                {
+                    namedLock = namedLockFactory.getLock( KEY_PREFIX + discriminator + ":" + key );
+                    locks.put( key, namedLock );
+                    acquiredLockCount++;
+                    doLock = true;
+                }
+                else
+                {
+                    reacquiredLockCount++;
+                    doLock = false;
+                }
+
+                if ( doLock )
+                {
+                    try
+                    {
+                        boolean locked;
+                        if ( shared )
+                        {
+                            locked = namedLock.lockShared( time, timeUnit );
+                        }
+                        else
+                        {
+                            locked = namedLock.lockExclusively( time, timeUnit );
+                        }
+
+                        if ( !locked )
+                        {
+                            throw new IllegalStateException( "Could not lock "
+                                    + namedLock.name() + " (shared=" + shared + ")" );
+                        }
+                    }
+                    catch ( InterruptedException e )
+                    {
+                        Thread.currentThread().interrupt();
+                        throw new RuntimeException( e );
+                    }
+                }
+            }
+            LOGGER.trace( "Total new locks acquired: {}, total existing locks reacquired: {}",

Review comment:
       What do you mean by "decent lock"? And why this matters, as you said above reentrancy is not used anymore? (unless I misunderstood you). Is this only about the logger message "reacquired"? Will remove it...




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

michael-o commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524477115



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/SyncContextFactoryAdapter.java
##########
@@ -0,0 +1,300 @@
+package org.eclipse.aether.internal.named;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.metadata.Metadata;
+import org.eclipse.aether.util.ChecksumUtils;
+import org.eclipse.aether.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Adapter to adapt {@link NamedLockFactory} and {@link NamedLock} to {@link SyncContext}.
+ */
+public final class SyncContextFactoryAdapter
+{
+    private static final String DEFAULT_DISCRIMINATOR_DIGEST = "da39a3ee5e6b4b0d3255bfef95601890afd80709";
+
+    private static final String DEFAULT_HOSTNAME = "localhost";
+
+    private static final Logger LOGGER = LoggerFactory.getLogger( SyncContextFactoryAdapter.class );
+
+    private static final long TIME = Long.getLong(
+        SyncContextFactoryAdapter.class.getName() + ".time", 10L
+    );
+
+    private static final TimeUnit TIME_UNIT = TimeUnit.valueOf( System.getProperty(
+        SyncContextFactoryAdapter.class.getName() + ".timeunit", TimeUnit.SECONDS.name()
+    ) );
+
+    private final NamedLockFactory namedLockFactory;
+
+    private final long time;
+
+    private final TimeUnit timeUnit;
+
+    private final String hostname;
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory )
+    {
+        this( namedLockFactory, TIME, TIME_UNIT );
+    }
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory,
+                                      final long time,
+                                      final TimeUnit timeUnit )
+    {
+        this.namedLockFactory = namedLockFactory;
+        this.time = time;
+        this.timeUnit = timeUnit;
+        this.hostname = getHostname();
+    }
+
+    private String getHostname()
+    {
+        try
+        {
+            return InetAddress.getLocalHost().getHostName();
+        }
+        catch ( UnknownHostException e )
+        {
+            LOGGER.warn( "Failed to get hostname, using '{}'", DEFAULT_HOSTNAME, e );
+            return DEFAULT_HOSTNAME;
+        }
+    }
+
+    public SyncContext newInstance( final RepositorySystemSession session, final boolean shared )
+    {
+        return new AdaptedLockSyncContext( session, hostname, namedLockFactory, time, timeUnit, shared );
+    }
+
+    public void shutdown()
+    {
+        namedLockFactory.shutdown();
+    }
+
+    private static class AdaptedLockSyncContext
+            implements SyncContext
+    {
+        private static final String CONFIG_PROP_DISCRIMINATOR = "aether.syncContext.named.discriminator";
+
+        private static final String KEY_PREFIX = "mvn:resolver:";
+
+        private static final Logger LOGGER = LoggerFactory.getLogger( AdaptedLockSyncContext.class );
+
+        private final RepositorySystemSession session;
+
+        private final String hostname;
+
+        private final NamedLockFactory namedLockFactory;
+
+        private final long time;
+
+        private final TimeUnit timeUnit;
+
+        private final boolean shared;
+
+        private final Map<String, NamedLock> locks = new LinkedHashMap<>();
+
+        private AdaptedLockSyncContext( final RepositorySystemSession session,
+                                        final String hostname,
+                                        final NamedLockFactory namedLockFactory,
+                                        final long time,
+                                        final TimeUnit timeUnit,
+                                        final boolean shared )
+        {
+            this.session = session;
+            this.hostname = hostname;
+            this.namedLockFactory = namedLockFactory;
+            this.time = time;
+            this.timeUnit = timeUnit;
+            this.shared = shared;
+        }
+
+        @Override
+        public void acquire( Collection<? extends Artifact> artifacts,
+                             Collection<? extends Metadata> metadatas )
+        {
+            // Deadlock prevention: https://stackoverflow.com/a/16780988/696632
+            // We must acquire multiple locks always in the same order!
+            Collection<String> keys = new TreeSet<>();
+            if ( artifacts != null )
+            {
+                for ( Artifact artifact : artifacts )
+                {
+                    // TODO Should we include extension and classifier too?
+                    String key = "artifact:" + artifact.getGroupId() + ":"
+                            + artifact.getArtifactId() + ":" + artifact.getBaseVersion();
+                    keys.add( key );
+                }
+            }
+
+            if ( metadatas != null )
+            {
+                for ( Metadata metadata : metadatas )
+                {
+                    StringBuilder key = new StringBuilder( "metadata:" );
+                    if ( !metadata.getGroupId().isEmpty() )
+                    {
+                        key.append( metadata.getGroupId() );
+                        if ( !metadata.getArtifactId().isEmpty() )
+                        {
+                            key.append( ':' ).append( metadata.getArtifactId() );
+                            if ( !metadata.getVersion().isEmpty() )
+                            {
+                                key.append( ':' ).append( metadata.getVersion() );
+                            }
+                        }
+                    }
+                    keys.add( key.toString() );
+                }
+            }
+
+            if ( keys.isEmpty() )
+            {
+                return;
+            }
+
+            String discriminator = createDiscriminator();
+            LOGGER.trace( "Using key discriminator '{}' during this session", discriminator );
+
+            LOGGER.trace( "Need {} {} lock(s) for {}", keys.size(), shared ? "read" : "write", keys );
+            int acquiredLockCount = 0;
+            int reacquiredLockCount = 0;
+            boolean doLock;
+            for ( String key : keys )
+            {
+                NamedLock namedLock = locks.get( key );
+                if ( namedLock == null )
+                {
+                    namedLock = namedLockFactory.getLock( KEY_PREFIX + discriminator + ":" + key );
+                    locks.put( key, namedLock );
+                    acquiredLockCount++;
+                    doLock = true;
+                }
+                else
+                {
+                    reacquiredLockCount++;
+                    doLock = false;
+                }
+
+                if ( doLock )
+                {
+                    try
+                    {
+                        boolean locked;
+                        if ( shared )
+                        {
+                            locked = namedLock.lockShared( time, timeUnit );
+                        }
+                        else
+                        {
+                            locked = namedLock.lockExclusively( time, timeUnit );
+                        }
+
+                        if ( !locked )
+                        {
+                            throw new IllegalStateException( "Could not lock "
+                                    + namedLock.name() + " (shared=" + shared + ")" );
+                        }
+                    }
+                    catch ( InterruptedException e )
+                    {
+                        Thread.currentThread().interrupt();
+                        throw new RuntimeException( e );
+                    }
+                }
+            }
+            LOGGER.trace( "Total new locks acquired: {}, total existing locks reacquired: {}",

Review comment:
       See the documentation: https://maven.apache.org/resolver/apidocs/org/eclipse/aether/SyncContext.html It requires reentrancy.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524505161



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/SyncContextFactoryAdapter.java
##########
@@ -0,0 +1,300 @@
+package org.eclipse.aether.internal.named;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.metadata.Metadata;
+import org.eclipse.aether.util.ChecksumUtils;
+import org.eclipse.aether.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Adapter to adapt {@link NamedLockFactory} and {@link NamedLock} to {@link SyncContext}.
+ */
+public final class SyncContextFactoryAdapter
+{
+    private static final String DEFAULT_DISCRIMINATOR_DIGEST = "da39a3ee5e6b4b0d3255bfef95601890afd80709";
+
+    private static final String DEFAULT_HOSTNAME = "localhost";
+
+    private static final Logger LOGGER = LoggerFactory.getLogger( SyncContextFactoryAdapter.class );
+
+    private static final long TIME = Long.getLong(
+        SyncContextFactoryAdapter.class.getName() + ".time", 10L
+    );
+
+    private static final TimeUnit TIME_UNIT = TimeUnit.valueOf( System.getProperty(
+        SyncContextFactoryAdapter.class.getName() + ".timeunit", TimeUnit.SECONDS.name()
+    ) );
+
+    private final NamedLockFactory namedLockFactory;
+
+    private final long time;
+
+    private final TimeUnit timeUnit;
+
+    private final String hostname;
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory )
+    {
+        this( namedLockFactory, TIME, TIME_UNIT );
+    }
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory,
+                                      final long time,
+                                      final TimeUnit timeUnit )
+    {
+        this.namedLockFactory = namedLockFactory;
+        this.time = time;
+        this.timeUnit = timeUnit;
+        this.hostname = getHostname();
+    }
+
+    private String getHostname()
+    {
+        try
+        {
+            return InetAddress.getLocalHost().getHostName();
+        }
+        catch ( UnknownHostException e )
+        {
+            LOGGER.warn( "Failed to get hostname, using '{}'", DEFAULT_HOSTNAME, e );
+            return DEFAULT_HOSTNAME;
+        }
+    }
+
+    public SyncContext newInstance( final RepositorySystemSession session, final boolean shared )
+    {
+        return new AdaptedLockSyncContext( session, hostname, namedLockFactory, time, timeUnit, shared );
+    }
+
+    public void shutdown()
+    {
+        namedLockFactory.shutdown();
+    }
+
+    private static class AdaptedLockSyncContext
+            implements SyncContext
+    {
+        private static final String CONFIG_PROP_DISCRIMINATOR = "aether.syncContext.named.discriminator";
+
+        private static final String KEY_PREFIX = "mvn:resolver:";
+
+        private static final Logger LOGGER = LoggerFactory.getLogger( AdaptedLockSyncContext.class );
+
+        private final RepositorySystemSession session;
+
+        private final String hostname;
+
+        private final NamedLockFactory namedLockFactory;
+
+        private final long time;
+
+        private final TimeUnit timeUnit;
+
+        private final boolean shared;
+
+        private final Map<String, NamedLock> locks = new LinkedHashMap<>();
+
+        private AdaptedLockSyncContext( final RepositorySystemSession session,
+                                        final String hostname,
+                                        final NamedLockFactory namedLockFactory,
+                                        final long time,
+                                        final TimeUnit timeUnit,
+                                        final boolean shared )
+        {
+            this.session = session;
+            this.hostname = hostname;
+            this.namedLockFactory = namedLockFactory;
+            this.time = time;
+            this.timeUnit = timeUnit;
+            this.shared = shared;
+        }
+
+        @Override
+        public void acquire( Collection<? extends Artifact> artifacts,
+                             Collection<? extends Metadata> metadatas )
+        {
+            // Deadlock prevention: https://stackoverflow.com/a/16780988/696632
+            // We must acquire multiple locks always in the same order!
+            Collection<String> keys = new TreeSet<>();
+            if ( artifacts != null )
+            {
+                for ( Artifact artifact : artifacts )
+                {
+                    // TODO Should we include extension and classifier too?
+                    String key = "artifact:" + artifact.getGroupId() + ":"
+                            + artifact.getArtifactId() + ":" + artifact.getBaseVersion();
+                    keys.add( key );
+                }
+            }
+
+            if ( metadatas != null )
+            {
+                for ( Metadata metadata : metadatas )
+                {
+                    StringBuilder key = new StringBuilder( "metadata:" );
+                    if ( !metadata.getGroupId().isEmpty() )
+                    {
+                        key.append( metadata.getGroupId() );
+                        if ( !metadata.getArtifactId().isEmpty() )
+                        {
+                            key.append( ':' ).append( metadata.getArtifactId() );
+                            if ( !metadata.getVersion().isEmpty() )
+                            {
+                                key.append( ':' ).append( metadata.getVersion() );
+                            }
+                        }
+                    }
+                    keys.add( key.toString() );
+                }
+            }
+
+            if ( keys.isEmpty() )
+            {
+                return;
+            }
+
+            String discriminator = createDiscriminator();
+            LOGGER.trace( "Using key discriminator '{}' during this session", discriminator );
+
+            LOGGER.trace( "Need {} {} lock(s) for {}", keys.size(), shared ? "read" : "write", keys );
+            int acquiredLockCount = 0;
+            int reacquiredLockCount = 0;
+            boolean doLock;
+            for ( String key : keys )
+            {
+                NamedLock namedLock = locks.get( key );
+                if ( namedLock == null )
+                {
+                    namedLock = namedLockFactory.getLock( KEY_PREFIX + discriminator + ":" + key );
+                    locks.put( key, namedLock );
+                    acquiredLockCount++;
+                    doLock = true;
+                }
+                else
+                {
+                    reacquiredLockCount++;
+                    doLock = false;
+                }
+
+                if ( doLock )
+                {
+                    try
+                    {
+                        boolean locked;
+                        if ( shared )
+                        {
+                            locked = namedLock.lockShared( time, timeUnit );
+                        }
+                        else
+                        {
+                            locked = namedLock.lockExclusively( time, timeUnit );
+                        }
+
+                        if ( !locked )
+                        {
+                            throw new IllegalStateException( "Could not lock "
+                                    + namedLock.name() + " (shared=" + shared + ")" );
+                        }
+                    }
+                    catch ( InterruptedException e )
+                    {
+                        Thread.currentThread().interrupt();
+                        throw new RuntimeException( e );
+                    }
+                }
+            }
+            LOGGER.trace( "Total new locks acquired: {}, total existing locks reacquired: {}",

Review comment:
       See ec58eac66c88ad7c607e56e3c740e794785bea0e
   IMO, we misunderstood each other: lock upgrade is what SyncContext does not support (and NamedLocks did not either). I just tried to follow logic in adapter based on it's origin, the redisson. But this commit now simplifies things, and added tests for "nested" case as well.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on a change in pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

michael-o commented on a change in pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#discussion_r524505861



##########
File path: maven-resolver-synccontext-named/src/main/java/org/eclipse/aether/internal/named/SyncContextFactoryAdapter.java
##########
@@ -0,0 +1,300 @@
+package org.eclipse.aether.internal.named;
+
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import org.eclipse.aether.RepositorySystemSession;
+import org.eclipse.aether.SyncContext;
+import org.eclipse.aether.artifact.Artifact;
+import org.eclipse.aether.metadata.Metadata;
+import org.eclipse.aether.util.ChecksumUtils;
+import org.eclipse.aether.util.ConfigUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.net.InetAddress;
+import java.net.UnknownHostException;
+import java.nio.charset.StandardCharsets;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.Deque;
+import java.util.Iterator;
+import java.util.LinkedHashMap;
+import java.util.LinkedList;
+import java.util.Map;
+import java.util.TreeSet;
+import java.util.concurrent.TimeUnit;
+
+/**
+ * Adapter to adapt {@link NamedLockFactory} and {@link NamedLock} to {@link SyncContext}.
+ */
+public final class SyncContextFactoryAdapter
+{
+    private static final String DEFAULT_DISCRIMINATOR_DIGEST = "da39a3ee5e6b4b0d3255bfef95601890afd80709";
+
+    private static final String DEFAULT_HOSTNAME = "localhost";
+
+    private static final Logger LOGGER = LoggerFactory.getLogger( SyncContextFactoryAdapter.class );
+
+    private static final long TIME = Long.getLong(
+        SyncContextFactoryAdapter.class.getName() + ".time", 10L
+    );
+
+    private static final TimeUnit TIME_UNIT = TimeUnit.valueOf( System.getProperty(
+        SyncContextFactoryAdapter.class.getName() + ".timeunit", TimeUnit.SECONDS.name()
+    ) );
+
+    private final NamedLockFactory namedLockFactory;
+
+    private final long time;
+
+    private final TimeUnit timeUnit;
+
+    private final String hostname;
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory )
+    {
+        this( namedLockFactory, TIME, TIME_UNIT );
+    }
+
+    public SyncContextFactoryAdapter( final NamedLockFactory namedLockFactory,
+                                      final long time,
+                                      final TimeUnit timeUnit )
+    {
+        this.namedLockFactory = namedLockFactory;
+        this.time = time;
+        this.timeUnit = timeUnit;
+        this.hostname = getHostname();
+    }
+
+    private String getHostname()
+    {
+        try
+        {
+            return InetAddress.getLocalHost().getHostName();
+        }
+        catch ( UnknownHostException e )
+        {
+            LOGGER.warn( "Failed to get hostname, using '{}'", DEFAULT_HOSTNAME, e );
+            return DEFAULT_HOSTNAME;
+        }
+    }
+
+    public SyncContext newInstance( final RepositorySystemSession session, final boolean shared )
+    {
+        return new AdaptedLockSyncContext( session, hostname, namedLockFactory, time, timeUnit, shared );
+    }
+
+    public void shutdown()
+    {
+        namedLockFactory.shutdown();
+    }
+
+    private static class AdaptedLockSyncContext
+            implements SyncContext
+    {
+        private static final String CONFIG_PROP_DISCRIMINATOR = "aether.syncContext.named.discriminator";
+
+        private static final String KEY_PREFIX = "mvn:resolver:";
+
+        private static final Logger LOGGER = LoggerFactory.getLogger( AdaptedLockSyncContext.class );
+
+        private final RepositorySystemSession session;
+
+        private final String hostname;
+
+        private final NamedLockFactory namedLockFactory;
+
+        private final long time;
+
+        private final TimeUnit timeUnit;
+
+        private final boolean shared;
+
+        private final Map<String, NamedLock> locks = new LinkedHashMap<>();
+
+        private AdaptedLockSyncContext( final RepositorySystemSession session,
+                                        final String hostname,
+                                        final NamedLockFactory namedLockFactory,
+                                        final long time,
+                                        final TimeUnit timeUnit,
+                                        final boolean shared )
+        {
+            this.session = session;
+            this.hostname = hostname;
+            this.namedLockFactory = namedLockFactory;
+            this.time = time;
+            this.timeUnit = timeUnit;
+            this.shared = shared;
+        }
+
+        @Override
+        public void acquire( Collection<? extends Artifact> artifacts,
+                             Collection<? extends Metadata> metadatas )
+        {
+            // Deadlock prevention: https://stackoverflow.com/a/16780988/696632
+            // We must acquire multiple locks always in the same order!
+            Collection<String> keys = new TreeSet<>();
+            if ( artifacts != null )
+            {
+                for ( Artifact artifact : artifacts )
+                {
+                    // TODO Should we include extension and classifier too?
+                    String key = "artifact:" + artifact.getGroupId() + ":"
+                            + artifact.getArtifactId() + ":" + artifact.getBaseVersion();
+                    keys.add( key );
+                }
+            }
+
+            if ( metadatas != null )
+            {
+                for ( Metadata metadata : metadatas )
+                {
+                    StringBuilder key = new StringBuilder( "metadata:" );
+                    if ( !metadata.getGroupId().isEmpty() )
+                    {
+                        key.append( metadata.getGroupId() );
+                        if ( !metadata.getArtifactId().isEmpty() )
+                        {
+                            key.append( ':' ).append( metadata.getArtifactId() );
+                            if ( !metadata.getVersion().isEmpty() )
+                            {
+                                key.append( ':' ).append( metadata.getVersion() );
+                            }
+                        }
+                    }
+                    keys.add( key.toString() );
+                }
+            }
+
+            if ( keys.isEmpty() )
+            {
+                return;
+            }
+
+            String discriminator = createDiscriminator();
+            LOGGER.trace( "Using key discriminator '{}' during this session", discriminator );
+
+            LOGGER.trace( "Need {} {} lock(s) for {}", keys.size(), shared ? "read" : "write", keys );
+            int acquiredLockCount = 0;
+            int reacquiredLockCount = 0;
+            boolean doLock;
+            for ( String key : keys )
+            {
+                NamedLock namedLock = locks.get( key );
+                if ( namedLock == null )
+                {
+                    namedLock = namedLockFactory.getLock( KEY_PREFIX + discriminator + ":" + key );
+                    locks.put( key, namedLock );
+                    acquiredLockCount++;
+                    doLock = true;
+                }
+                else
+                {
+                    reacquiredLockCount++;
+                    doLock = false;
+                }
+
+                if ( doLock )
+                {
+                    try
+                    {
+                        boolean locked;
+                        if ( shared )
+                        {
+                            locked = namedLock.lockShared( time, timeUnit );
+                        }
+                        else
+                        {
+                            locked = namedLock.lockExclusively( time, timeUnit );
+                        }
+
+                        if ( !locked )
+                        {
+                            throw new IllegalStateException( "Could not lock "
+                                    + namedLock.name() + " (shared=" + shared + ")" );
+                        }
+                    }
+                    catch ( InterruptedException e )
+                    {
+                        Thread.currentThread().interrupt();
+                        throw new RuntimeException( e );
+                    }
+                }
+            }
+            LOGGER.trace( "Total new locks acquired: {}, total existing locks reacquired: {}",

Review comment:
       OK, I don't expect lock upgrade nor does `SyncContext`.




----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-728320815


   @michael-o Once you resolve all remaining conversations and have no more remark/change req, I'll squash the commits into one, until then am leaving them to be able to ref them from comments.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

michael-o commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-731601299


   @cstamas Do you want to perform a final review yourself before I go over again?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-731601543


   @michael-o Just go for it


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

michael-o commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-731601588


   When I read your note about HZ, does it make sense at all when it requires at least three nodes to work? This seems like overkill. I was able to achieve very decent peformance with Redis on an old `CPU: Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz`. On a new server even better.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-731602192


   3 nodes are required for CP only w/ Hazelcast 3.x cluster (RAFT consensus backed structures), and yes, is overkill (but someone may choose to run a "dedicated cluster" on CI nodes anyway). For non-CP stuff, single "node" that is started by provider itself is completely okay.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas edited a comment on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas edited a comment on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-731602192


   3 nodes are required for CP only w/ Hazelcast 3.x cluster (RAFT consensus backed structures), and yes, is overkill (but someone may choose to run a "dedicated cluster" on CI nodes anyway, as here I use HZ client only that connects to some existing cluster). For non-CP stuff, single "node" that is started by provider itself is completely okay.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] michael-o commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

michael-o commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-731603812


   So no code changes are required to do a single node setup like with Redis?


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


Reply | Threaded
Open this post in threaded view
|

[GitHub] [maven-resolver] cstamas commented on pull request #77: [MRESOLVER-145] SyncContext implementations

GitBox
In reply to this post by GitBox

cstamas commented on pull request #77:
URL: https://github.com/apache/maven-resolver/pull/77#issuecomment-731606060


   That's right


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]


123