[SoC] [Qgis-developer] Report 2 - QGIS Symbology Sharing Tools

Wed Jun 8 23:42:44 PDT 2016

2016-06-09 0:16 GMT+02:00 Nathan Woodrow <madmanwoo at gmail.com>:

> Using the raw end point sounds fine to me for now. Easy to change later if
> needed.
>

Yes, that's the best thing to do.

>
> On Thu, 9 Jun 2016 8:14 am Akbar Gumbira <akbargumbira at gmail.com> wrote:
>
>> Hi Ale,
>>
>> I just tried Dulwich. There are three ways that I see to checkout a
>> single file: git sparse checkout, git archive, or shallow clone (still
>> cloning all files though).
>>
>> With git sparse checkout, Dulwich does not provide the porcelain-level
>> API. The author says that it's possible though (
>> https://github.com/jelmer/dulwich/issues/405#issuecomment-224579678). I
>> am not sure how much work to implement it dealing with the git object,
>> tree, and blob itself at the plumbing level.
>>
>> With git archive, unfortunately Github doesn't allow us to do this (
>> https://twitter.com/GitHubHelp/status/322818593748303873). WIth shallow
>> clone left, I feel that I would over-engineer it forcing to use git just to
>> be able to do this. I think it's more simple if for now we use the raw
>> endpoint for each hosts that we know (Github, Bitbucket, etc), and we add
>> more supported hosts as users put the collection in other hosts as long as
>> we know the URL to the raw metadata file.
>>
>> What do you think?
>>
>> Cheers
>>
>> On Wed, Jun 8, 2016 at 9:51 AM, Akbar Gumbira <akbargumbira at gmail.com>
>> wrote:
>>
>>> Wasn't that a requisite? If we are going to use git as a storage we need
>>>> a git client (pure python seems to exist, could be used as a fallback in
>>>> case git is not installed in the system)
>>>> https://github.com/jelmer/dulwich .
>>>>
>>>
>>>
>>>> BTW: if you do not want to deal with git in this first task, you could
>>>> used the zip endpoint, and assume that the metadata will be available at a
>>>> known location in the remote http repo and just wget it.
>>>
>>>
>>> Ok. I was just trying to find solution that could work out well without
>>> having git (as from the last discussion with Martin, it's better to avoid
>>> using git for now). I will try pygit2 and dulwich and I'll report you back
>>> what I thought using those two.
>>>
>>> On Wed, Jun 8, 2016 at 9:41 AM, Alessandro Pasotti <apasotti at gmail.com>
>>> wrote:
>>>
>>>> 2016-06-08 9:21 GMT+02:00 Akbar Gumbira <akbargumbira at gmail.com>:
>>>>
>>>>> Yes, I have tried it. We can do it with sparse checkout, but, it
>>>>> requires git on the client.
>>>>>
>>>>
>>>>
>>>> Wasn't that a requisite? If we are going to use git as a storage we
>>>> need a git client (pure python seems to exist, could be used as a fallback
>>>> in case git is not installed in the system)
>>>> https://github.com/jelmer/dulwich .
>>>>
>>>> BTW: if you do not want to deal with git in this first task, you could
>>>> used the zip endpoint, and assume that the metadata will be available at a
>>>> known location in the remote http repo and just wget it.
>>>>
>>>>
>>>>
>>>>>
>>>>> On Wed, Jun 8, 2016 at 9:18 AM, Alessandro Pasotti <apasotti at gmail.com
>>>>> > wrote:
>>>>>
>>>>>>
>>>>>> 2016-06-08 9:10 GMT+02:00 Akbar Gumbira <akbargumbira at gmail.com>:
>>>>>>
>>>>>>> Hi Akbar,
>>>>>>>> The most flexible installation tool that I know is probably python
>>>>>>>> pip.
>>>>>>>> pip can install software from a zip file, from git and from other
>>>>>>>> sources too.
>>>>>>>> I'd suggest you to have a look to pip implementation of the install
>>>>>>>> functionality, maybe there is some interesting for you.
>>>>>>>
>>>>>>> I just read pip code base. I think I can pick something from there
>>>>>>> for downloading the resources. But the problem I have right now is to get
>>>>>>> only the metadata file from the repository. Or are you suggesting that when
>>>>>>> users add a repository connection, it also downloads the repository
>>>>>>> directly?
>>>>>>>
>>>>>>
>>>>>>
>>>>>> If you are working on the git repos you can probably fetch just one
>>>>>> file without cloning the whole repo, I did not test it but here are
>>>>>> probably some pointers:
>>>>>>
>>>>>>
>>>>>> http://stackoverflow.com/questions/2466735/how-to-checkout-only-one-file-from-git-repository
>>>>>>
>>>>>> http://stackoverflow.com/questions/1125476/retrieve-a-single-file-from-a-repository
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Mon, Jun 6, 2016 at 8:39 AM, Alessandro Pasotti <
>>>>>>> apasotti at gmail.com> wrote:
>>>>>>>
>>>>>>>> 2016-06-05 10:13 GMT+02:00 Richard Duivenvoorde <
>>>>>>>> rdmailings at duif.net>:
>>>>>>>>
>>>>>>>>> On 05-06-16 09:02, Akbar Gumbira wrote:
>>>>>>>>>
>>>>>>>>> > *Are you blocked on anything?*
>>>>>>>>> > ... In Github or Bitbucket they provide a direct link to
>>>>>>>>> > the raw file. But I think I should look at more general approach
>>>>>>>>> without
>>>>>>>>> > manipulating the URL depending on the host. If you have some
>>>>>>>>> input, I
>>>>>>>>> > would be happy to assess it.
>>>>>>>>>
>>>>>>>>> Thanks Akbar,
>>>>>>>>>
>>>>>>>>> I did some googling:
>>>>>>>>>
>>>>>>>>> http://stackoverflow.com/questions/14405782/git-fetch-single-file-from-remote-repository-programatically
>>>>>>>>> If you really want to keep it git, it looks like a shallow
>>>>>>>>> clone/copy is
>>>>>>>>> the only way? That post also talks about some undocumented
>>>>>>>>> feature, but
>>>>>>>>> I would not depend on that?
>>>>>>>>>
>>>>>>>>> Personally I would be ok when both Github and Gitlab/Gog would
>>>>>>>>> work (as
>>>>>>>>> both a closed source and open source member of the
>>>>>>>>> git-web-world)...
>>>>>>>>>
>>>>>>>>> Or: a script running somewhere on our server, (shallow) cloning all
>>>>>>>>> registred repositories periodically, and making just the
>>>>>>>>> metadata.txt
>>>>>>>>> files available via http/webserver? (maybe giving us some time to
>>>>>>>>> check
>>>>>>>>> the repo's on structure and (malicious?) content?
>>>>>>>>>
>>>>>>>>> Or else: a django app for the metadata...
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>
>>>>>>>>> Richard
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> Hi Akbar,
>>>>>>>>
>>>>>>>> The most flexible installation tool that I know is probably python
>>>>>>>> pip.
>>>>>>>>
>>>>>>>> pip can install software from a zip file, from git and from other
>>>>>>>> sources too.
>>>>>>>>
>>>>>>>> I'd suggest you to have a look to pip implementation of the install
>>>>>>>> functionality, maybe there is some interesting for you.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Alessandro Pasotti
>>>>>>>> w3:   www.itopen.it
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *-------------------*
>>>>>>> *Akbar Gumbira *
>>>>>>> *www.akbargumbira.com <http://www.akbargumbira.com>*
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Alessandro Pasotti
>>>>>> w3:   www.itopen.it
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> *-------------------*
>>>>> *Akbar Gumbira *
>>>>> *www.akbargumbira.com <http://www.akbargumbira.com>*
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Alessandro Pasotti
>>>> w3:   www.itopen.it
>>>>
>>>
>>>
>>>
>>> --
>>>
>>> *-------------------*
>>> *Akbar Gumbira *
>>> *www.akbargumbira.com <http://www.akbargumbira.com>*
>>>
>>
>>
>>
>> --
>>
>> *-------------------*
>> *Akbar Gumbira *
>> *www.akbargumbira.com <http://www.akbargumbira.com>*
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.osgeo.org
>> List info: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>

-- 
Alessandro Pasotti
w3:   www.itopen.it
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/soc/attachments/20160609/45087cdb/attachment-0001.html>