[QGIS-Developer] mirroring qgis.org repos

[Richard Duivenvoorde]

On 05-06-18 09:54, Richard Duivenvoorde wrote:
> On 05-06-18 09:31, Jürgen E. Fischer wrote:
>> Hi Bernhard,
>>
>> On Tue, 05. Jun 2018 at 08:29:09 +0200, Bernhard Ströbl wrote:
>>> has anything changed recently? We have -again- since several days problems
>>> mirroring ubuntugis locally:
>>>
>>> debmirror --dist=xenial --section=main --arch=i386,amd64 --cleanup
>>> --host=qgis.org --root=ubuntugis-ltr --method=https --rsync-extra=none
>>> /mirrorpath
>>> fails with 403 both via http and https
>>
>> Odd.  With wget and a browser I can get
>> https://qgis.org/ubuntugis-ltr/dists/xenial/Release.
>>
>> But I can reproduce with debmirror.   I works when when I use nocache.qgis.org
>> instead of qgis.org.
>>
>> Looks like a cloudflare issue.
> 
> Using myself here (adding -v -debug).

Did some more testing, and it is definitly a cloudflare issue.
If I disable caching on qgis.org temporarily it works.

But if I use a silly (googled) perl script which mimics the perl lines
from debmirror:

# Create a user agent object
use LWP::UserAgent;
my $ua = LWP::UserAgent->new;
my $res = $ua->mirror(
'https://anonymous@qgis.org/ubuntugis-ltr/dists/xenial/Release',
'/tmp/Release' );
# Check the outcome of the response
if ($res->is_success) {
    print $res->content;
}
else {
    print $res->status_line, "\n";
}

it also(!) just works...
Tried to set the ssl_verification to false, other headers, but I could
not make debmirror work.

We cannot remove cache from qgis.org (as we generate too much web
traffic then, and our services really become slower)

You could use https://nocache.qgis.org ?

I prefer to keep ubuntu.qgis.org cached too, unless others have problems
with it too..

Sorry I cannot be of more help.

Regards,

Richard