[osgeo4w-dev] Alternative osgeo4w infrastructure

[Hugo Mercier]

Hi Jüergen,

On 27/04/2017 21:42, Jürgen E. Fischer wrote:

> 
>> - the points listed above also result in a situation where there is a mix of
>>   compiler versions used for packages. Which should be avoided
>>   (http://siomsystems.com/mixing-visual-studio-versions/)
> 
> That's certainly a fragile issue.  We don't track which compilers were used to
> produce the individual packages - so it's also unclear on which version of the
> msvcrt they depend on (msvcrt used to have several, although it was meanwhile
> split up, the dependencies were still not refined everywhere).
> 
> But I think it's unavoidable sometimes - I wouldn't like to have three GDALs
> and all their dependencies for QGIS 2 (msvc2010/py2), 3 (msvc2015/py3) and
> GRASS (MinGW) in OSGeo4W (not to mention the GRASS plugin where all three would
> meet anyway; also newer version of GDAL will not work with msvc2010 because of
> c++11).  I think it's meant to work and it does, still it should be avoided
> where possible.

Yes indeed, it seems too complex to maintain 3 different versions of
base libraries.
From what I read in the pointed article, mixing msvcrt versions could
lead to problems that is hard to spot on first sight. Everything seems
to work well ... until it does not. But that's also right that the
osgeo4w stack is a good example of mixed libraries that just work. So
maybe it is not a big deal, but it would help if we know with which
versions of msvcrt packages have been compiled.

And ... thanks for mentioning mingw that I forgot.

> 
> Others:
> * not listing of build dependencies (libraries and tools)
> * no build recipes for everything
> * those that exist don't have a form that would allow automatic invocation in a
>   uniform way
> * source packages don't contain sources nor automatically usable pointers
>   to download them.
> 
> So there's much to do on that front before all this can work.

Yes. Having a central git repository to host package sources with PR and
reviews could help to maintain the quality of building scripts.

> 
> Also there are some proprietary SDKs (esp. for use with GDAL: ECW, MrSID,
> FileGDB etc.) that I'm not sure you are allowed to make them public in such a
> setup - although you are allowed to ship the redistributable runtime parts.
> 

Ah. You mean the sources cannot be made public outside of their official
servers ? We can only get binaries and package them ?

> 
>> We actually hit a problem probably related to that
>> (https://trac.osgeo.org/osgeo4w/ticket/529) and that could mean to recompile
>> gdal and all its dependencies with msvc 2015.
> 
> I think that is just due to the fact that numpy wasn't installed, when the
> package was built.
> 
> That that problem is not already fixed, is just because I'm working on
> transitioning building gdal with msvc2015 (needed for 2.2).
> 
> Having all this in place would have certainly helped with this of course.
> 
> The nightly should already work again - 2.2 not yet.

Good to know, I'll have a look. Thanks !

> 
> 
>> So here are some propositions:
>> - find a way to add a mirroring system for the osgeo4w distribution. The idea
>> is to register a list of mirrors that would receive a copy of files
>> when they are uploaded to the main repository.
> 
> AFAIK there is also a OSUSL file server/proxy that could be set in front
> download.osgeo.org to help with this - bandwidth should be better.  Not sure if
> that extra mirror architecture would still be necessary.

I don't know exactly how that works. How are files added to
download.osgeo.org/osgeo4w ? By ftp/scp I guess ? Could this proxy be
configured so that each file copied to osgeo.org gets copied to a list
of mirrors ? Or does the mirror have to synchronize periodically with
the main site ?

How much space would be required to host an osgeo4w repository ?


>> Are there any interests for that work? Raise your hand if so, and raise
>> both hands if you could contribute (time or funds) :) we also still need
>> to find a way to fund this alternative osgeo4w / CI server.
> 
> Both hands full ;) - but consider them raised.
> 
> Although I have somewhat mixed feelings about this.  Except for Martin Landa
> who took over GRASS of course, I have been pretty much on my own on this for a
> while...

Yes you already do a lot of things ! The intention behind this is also
to make it easier for good souls to help you :)


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 213 bytes
Desc: OpenPGP digital signature
URL: <http://lists.osgeo.org/pipermail/osgeo4w-dev/attachments/20170428/b7dbea52/attachment.sig>