[Qgis-developer] Report 2 - QGIS Symbology Sharing Tools

Wed Jun 8 15:16:54 PDT 2016

Using the raw end point sounds fine to me for now. Easy to change later if
needed.

On Thu, 9 Jun 2016 8:14 am Akbar Gumbira <akbargumbira at gmail.com> wrote:

> Hi Ale,
>
> I just tried Dulwich. There are three ways that I see to checkout a single
> file: git sparse checkout, git archive, or shallow clone (still cloning all
> files though).
>
> With git sparse checkout, Dulwich does not provide the porcelain-level
> API. The author says that it's possible though (
> https://github.com/jelmer/dulwich/issues/405#issuecomment-224579678). I
> am not sure how much work to implement it dealing with the git object,
> tree, and blob itself at the plumbing level.
>
> With git archive, unfortunately Github doesn't allow us to do this (
> https://twitter.com/GitHubHelp/status/322818593748303873). WIth shallow
> clone left, I feel that I would over-engineer it forcing to use git just to
> be able to do this. I think it's more simple if for now we use the raw
> endpoint for each hosts that we know (Github, Bitbucket, etc), and we add
> more supported hosts as users put the collection in other hosts as long as
> we know the URL to the raw metadata file.
>
> What do you think?
>
> Cheers
>
> On Wed, Jun 8, 2016 at 9:51 AM, Akbar Gumbira <akbargumbira at gmail.com>
> wrote:
>
>> Wasn't that a requisite? If we are going to use git as a storage we need
>>> a git client (pure python seems to exist, could be used as a fallback in
>>> case git is not installed in the system)
>>> https://github.com/jelmer/dulwich .
>>>
>>
>>
>>> BTW: if you do not want to deal with git in this first task, you could
>>> used the zip endpoint, and assume that the metadata will be available at a
>>> known location in the remote http repo and just wget it.
>>
>>
>> Ok. I was just trying to find solution that could work out well without
>> having git (as from the last discussion with Martin, it's better to avoid
>> using git for now). I will try pygit2 and dulwich and I'll report you back
>> what I thought using those two.
>>
>> On Wed, Jun 8, 2016 at 9:41 AM, Alessandro Pasotti <apasotti at gmail.com>
>> wrote:
>>
>>> 2016-06-08 9:21 GMT+02:00 Akbar Gumbira <akbargumbira at gmail.com>:
>>>
>>>> Yes, I have tried it. We can do it with sparse checkout, but, it
>>>> requires git on the client.
>>>>
>>>
>>>
>>> Wasn't that a requisite? If we are going to use git as a storage we need
>>> a git client (pure python seems to exist, could be used as a fallback in
>>> case git is not installed in the system)
>>> https://github.com/jelmer/dulwich .
>>>
>>> BTW: if you do not want to deal with git in this first task, you could
>>> used the zip endpoint, and assume that the metadata will be available at a
>>> known location in the remote http repo and just wget it.
>>>
>>>
>>>
>>>>
>>>> On Wed, Jun 8, 2016 at 9:18 AM, Alessandro Pasotti <apasotti at gmail.com>
>>>> wrote:
>>>>
>>>>>
>>>>> 2016-06-08 9:10 GMT+02:00 Akbar Gumbira <akbargumbira at gmail.com>:
>>>>>
>>>>>> Hi Akbar,
>>>>>>> The most flexible installation tool that I know is probably python
>>>>>>> pip.
>>>>>>> pip can install software from a zip file, from git and from other
>>>>>>> sources too.
>>>>>>> I'd suggest you to have a look to pip implementation of the install
>>>>>>> functionality, maybe there is some interesting for you.
>>>>>>
>>>>>> I just read pip code base. I think I can pick something from there
>>>>>> for downloading the resources. But the problem I have right now is to get
>>>>>> only the metadata file from the repository. Or are you suggesting that when
>>>>>> users add a repository connection, it also downloads the repository
>>>>>> directly?
>>>>>>
>>>>>
>>>>>
>>>>> If you are working on the git repos you can probably fetch just one
>>>>> file without cloning the whole repo, I did not test it but here are
>>>>> probably some pointers:
>>>>>
>>>>>
>>>>> http://stackoverflow.com/questions/2466735/how-to-checkout-only-one-file-from-git-repository
>>>>>
>>>>> http://stackoverflow.com/questions/1125476/retrieve-a-single-file-from-a-repository
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>
>>>>>> On Mon, Jun 6, 2016 at 8:39 AM, Alessandro Pasotti <
>>>>>> apasotti at gmail.com> wrote:
>>>>>>
>>>>>>> 2016-06-05 10:13 GMT+02:00 Richard Duivenvoorde <rdmailings at duif.net
>>>>>>> >:
>>>>>>>
>>>>>>>> On 05-06-16 09:02, Akbar Gumbira wrote:
>>>>>>>>
>>>>>>>> > *Are you blocked on anything?*
>>>>>>>> > ... In Github or Bitbucket they provide a direct link to
>>>>>>>> > the raw file. But I think I should look at more general approach
>>>>>>>> without
>>>>>>>> > manipulating the URL depending on the host. If you have some
>>>>>>>> input, I
>>>>>>>> > would be happy to assess it.
>>>>>>>>
>>>>>>>> Thanks Akbar,
>>>>>>>>
>>>>>>>> I did some googling:
>>>>>>>>
>>>>>>>> http://stackoverflow.com/questions/14405782/git-fetch-single-file-from-remote-repository-programatically
>>>>>>>> If you really want to keep it git, it looks like a shallow
>>>>>>>> clone/copy is
>>>>>>>> the only way? That post also talks about some undocumented feature,
>>>>>>>> but
>>>>>>>> I would not depend on that?
>>>>>>>>
>>>>>>>> Personally I would be ok when both Github and Gitlab/Gog would work
>>>>>>>> (as
>>>>>>>> both a closed source and open source member of the git-web-world)...
>>>>>>>>
>>>>>>>> Or: a script running somewhere on our server, (shallow) cloning all
>>>>>>>> registred repositories periodically, and making just the
>>>>>>>> metadata.txt
>>>>>>>> files available via http/webserver? (maybe giving us some time to
>>>>>>>> check
>>>>>>>> the repo's on structure and (malicious?) content?
>>>>>>>>
>>>>>>>> Or else: a django app for the metadata...
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Richard
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> Hi Akbar,
>>>>>>>
>>>>>>> The most flexible installation tool that I know is probably python
>>>>>>> pip.
>>>>>>>
>>>>>>> pip can install software from a zip file, from git and from other
>>>>>>> sources too.
>>>>>>>
>>>>>>> I'd suggest you to have a look to pip implementation of the install
>>>>>>> functionality, maybe there is some interesting for you.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Alessandro Pasotti
>>>>>>> w3:   www.itopen.it
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>>
>>>>>> *-------------------*
>>>>>> *Akbar Gumbira *
>>>>>> *www.akbargumbira.com <http://www.akbargumbira.com>*
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Alessandro Pasotti
>>>>> w3:   www.itopen.it
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> *-------------------*
>>>> *Akbar Gumbira *
>>>> *www.akbargumbira.com <http://www.akbargumbira.com>*
>>>>
>>>
>>>
>>>
>>> --
>>> Alessandro Pasotti
>>> w3:   www.itopen.it
>>>
>>
>>
>>
>> --
>>
>> *-------------------*
>> *Akbar Gumbira *
>> *www.akbargumbira.com <http://www.akbargumbira.com>*
>>
>
>
>
> --
>
> *-------------------*
> *Akbar Gumbira *
> *www.akbargumbira.com <http://www.akbargumbira.com>*
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20160608/f9e2fee5/attachment.html>