ScmProvider.list()

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

ScmProvider.list()

Ilya Basin
Hi.
We need your opinion on the following topic. While Svn and Cvs providers perform the list() operation remotely and don't need a checkout directory, Git and some others
simply list the local files (generally, because their SCMs don't provide a remote list method). Arguments passed to the list() method also have different meanings for those
providers.

I think this is a problem for Scm provider clients unaware of the underlying SCM:
The most severe problem is the differing meaning of the fileSet.getBasedir() parameter. It may mean the remote path or the local CWD.
Secondly, the clients sometimes need to create a temporary folder to run the list() operation in, otherwise the files can be checked out in the process CWD.
And lastly, the clients expect the list operation to be fast, while it not always is.

We should specify in the ScmProvider.list() javadoc that if the SCM does not support remote listing, then the method should just fail and also modify the existing
ListCommand implementations.

The clients desiring to quickly list the repository contents could try list() and if it fails, fallback to ScmProvider.checkout() plus java.io.File.listFiles().



On 30.03.2018 0:05, Michael Osipov (JIRA) wrote:

>
>     [ https://issues.apache.org/jira/browse/SCM-869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16419757#comment-16419757 ]
>
> Michael Osipov commented on SCM-869:
> ------------------------------------
>
> The {{list}} function has been introduced with SCM-183 12 years ago with b158cf35d05271942a10bbef564366ab6a3059ad. I have checked few other providers. They have been incorrectly implemented also. The Git provider has been incorrectly implemented eversince with {{git ls-files}}.
>
> I think this birth issue can only be properly resolved with 2.0.0.
>
> Can you actually raise this general issue on the dev mailing list? I'd be interested in other devs opinion. I don't might incorrect code altogether or change the Javadoc if this is a better way.
>
>> gitexe list() implemented incorrectly
>> -------------------------------------
>>
>>                 Key: SCM-869
>>                 URL: https://issues.apache.org/jira/browse/SCM-869
>>             Project: Maven SCM
>>          Issue Type: Bug
>>          Components: maven-scm-provider-gitexe
>>    Affects Versions: 1.9.5, 1.9.6
>>            Reporter: Ilya Basin
>>            Priority: Major
>>
>> Taking the Svn implementation as a model, ScmProvider.list() should be implemented as follows:
>>  * The command must directly query the remote repository for files
>>  * A local working copy is unnecessary and if it doesn't exist, the remote repository must not be checked out.
>>  * fileSet.getBasedir() indicates where to run the scm binary. The recommended value is ".".
>>  * fileSet.getFileList() indicates the files to list
>>  * repository indicates the repo URL
>> Git (among other SCMs) does not support listing remote files, so the command should just fail.
>> For listing files in a working copy, users should call the ScmProvider.status() method instead.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ScmProvider.list()

michaelo
Am 2018-03-30 um 10:47 schrieb Basin Ilya:
> Hi.
> We need your opinion on the following topic. While Svn and Cvs providers perform the list() operation remotely and don't need a checkout directory, Git and some others
> simply list the local files (generally, because their SCMs don't provide a remote list method). Arguments passed to the list() method also have different meanings for those
> providers.
>
>
> We should specify in the ScmProvider.list() javadoc that if the SCM does not support remote listing, then the method should just fail and also modify the existing
> ListCommand implementations.

My opinion on this is that the ListCommand says remote repos. If some
SCM provider implements it wrong, it either has to be dropped or
corrected. I don't like the idea to checkout or clone to list files.

The Git provider has been implemented incorrectly from the beginning. I
checked some other SCMs and they don't implement it at all.

This needs to be fixed in 2.0.0.

Michael

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ScmProvider.list()

michaelo
Am 2018-04-01 um 23:01 schrieb Stephen Connolly:

> On Sun 1 Apr 2018 at 17:21, Michael Osipov <[hidden email]> wrote:
>
>> Am 2018-04-01 um 13:54 schrieb Stephen Connolly:
>>> On Fri 30 Mar 2018 at 10:20, Michael Osipov <[hidden email]> wrote:
>>>
>>>> Am 2018-03-30 um 10:47 schrieb Basin Ilya:
>>>>> Hi.
>>>>> We need your opinion on the following topic. While Svn and Cvs
>> providers
>>>> perform the list() operation remotely and don't need a checkout
>> directory,
>>>> Git and some others
>>>>> simply list the local files (generally, because their SCMs don't
>> provide
>>>> a remote list method). Arguments passed to the list() method also have
>>>> different meanings for those
>>>>> providers.
>>>>>
>>>>>
>>>>> We should specify in the ScmProvider.list() javadoc that if the SCM
>> does
>>>> not support remote listing, then the method should just fail and also
>>>> modify the existing
>>>>> ListCommand implementations.
>>>>
>>>> My opinion on this is that the ListCommand says remote repos. If some
>>>> SCM provider implements it wrong, it either has to be dropped or
>>>> corrected. I don't like the idea to checkout or clone to list files.
>>>>
>>>> The Git provider has been implemented incorrectly from the beginning. I
>>>> checked some other SCMs and they don't implement it at all.
>>>>
>>>> This needs to be fixed in 2.0.0.
>>>
>>>
>>> It depends, git being distributed, if you have the local checkout, you
>> can
>>> list exactly without needing to go remote.
>>
>> No, the docs say remote list. Not local list.
>>
>>> But perhaps the thing here is to enhance the api to provide for both
>> cases.
>>
>> Likely, but you will end up in a split situation because not every SCM
>> will implement your usercase.
>>
>>> At the minimum, git could do an ls-remote and then list from local once
>> the
>>> revision is confirmed present (which would meet a goal of being a “read
>>> only” operation)
>>
>> ls-remote does *not* list any files remotely, it lists remote refs.
>
>
> And if we have the remote ref local then *because refs are immutable* we
> can list the files from local.
>
> I think the only issue is the lack of a call to ls-remote

I do not understand what you are trying to say. How is the output of
ls-remote/remote refs relateed to the remote file listing?
Even if you have the current branch, you still cannot list the files
because you don't know wether your local branch is uptodate with remote.

Michael


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: ScmProvider.list()

michaelo
Am 2018-04-11 um 12:23 schrieb Basin Ilya:

>
>
> On 02.04.2018 10:47, Stephen Connolly wrote:
>> On Sun 1 Apr 2018 at 22:12, Michael Osipov <[hidden email]> wrote:
>>
>>> Am 2018-04-01 um 23:01 schrieb Stephen Connolly:
>>>> On Sun 1 Apr 2018 at 17:21, Michael Osipov <[hidden email]> wrote:
>>>>
>>>>> Am 2018-04-01 um 13:54 schrieb Stephen Connolly:
>>>>>> On Fri 30 Mar 2018 at 10:20, Michael Osipov <[hidden email]>
>>> wrote:
>>>>>>> Am 2018-03-30 um 10:47 schrieb Basin Ilya:
>>>>>>>> Hi.
>>>>>>>> We need your opinion on the following topic. While Svn and Cvs
>>>>> providers
>>>>>>> perform the list() operation remotely and don't need a checkout
>>>>> directory,
>>>>>>> Git and some others
>>>>>>>> simply list the local files (generally, because their SCMs don't
>>>>> provide
>>>>>>> a remote list method). Arguments passed to the list() method also have
>>>>>>> different meanings for those
>>>>>>>> providers.
>>>>>>>>
>>>>>>>>
>>>>>>>> We should specify in the ScmProvider.list() javadoc that if the SCM
>>>>> does
>>>>>>> not support remote listing, then the method should just fail and also
>>>>>>> modify the existing
>>>>>>>> ListCommand implementations.
>>>>>>> My opinion on this is that the ListCommand says remote repos. If some
>>>>>>> SCM provider implements it wrong, it either has to be dropped or
>>>>>>> corrected. I don't like the idea to checkout or clone to list files.
>>>>>>>
>>>>>>> The Git provider has been implemented incorrectly from the beginning.
>>> I
>>>>>>> checked some other SCMs and they don't implement it at all.
>>>>>>>
>>>>>>> This needs to be fixed in 2.0.0.
>>>>>>
>>>>>> It depends, git being distributed, if you have the local checkout, you
>>>>> can
>>>>>> list exactly without needing to go remote.
>>>>> No, the docs say remote list. Not local list.
>>>>>
>>>>>> But perhaps the thing here is to enhance the api to provide for both
>>>>> cases.
>>>>>
>>>>> Likely, but you will end up in a split situation because not every SCM
>>>>> will implement your usercase.
>>>>>
>>>>>> At the minimum, git could do an ls-remote and then list from local once
>>>>> the
>>>>>> revision is confirmed present (which would meet a goal of being a “read
>>>>>> only” operation)
>>>>> ls-remote does *not* list any files remotely, it lists remote refs.
>>>>
>>>> And if we have the remote ref local then *because refs are immutable* we
>>>> can list the files from local.
>>>>
>>>> I think the only issue is the lack of a call to ls-remote
>>> I do not understand what you are trying to say. How is the output of
>>> ls-remote/remote refs relateed to the remote file listing?
>>> Even if you have the current branch, you still cannot list the files
>>> because you don't know wether your local branch is uptodate with remote.
>>
>> That is what ls-remote tells you. It tells you the revisions of every
>> branch and tag in the remote (or you can just ask for one branch)
>>
>> Then if the revision is in the .git database you can list the files
>
> Okay. Here's how I see this: the list() method should not require a local checkout directory, but ls-files requires at least a bare repo. As a compromise we could have a set of persistent bare git repositories, shared by multiple Scm clients, just like makepkg from Archlinux does.
>
> However, I have no idea how to do the same with Mercurial and other similar SCMs. This is why I'd prefer to push the efforts and responsibility to the users of ScmClient. Let them try list() and if it's not supported, then let them do checkout and list files in a directory. This would be much simpler.

+1. We can't satisfy all SCMs with the same API.


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]