Fwd: Re: [gdal-dev] FWTools and GDAL 1.7.0

[Christopher Barker]

It may well be that GDAL has too many different use cases to even have a 
"standard" install, but...

On 1/5/11 1:37 PM, Tamas Szekeres wrote:
> 2011/1/5 Christopher Barker <Chris.Barker at noaa.gov
>     1) It would be nice to have binaries for the latest release front
>     and center at the main GDAL site -- having to poke around to find
>     Tamas's site is not a big deal, but not always obvious.

> With regards to the comment above, while I'm not sure about the
> objectives but I don't think the GDAL site would intend to be a hosting
> provider of various binary packages,

Well, many (most?) open source packages have "official" binaries hosted 
on its site. It's pretty common to go to a project's site and expect to 
find a way to download binaries right then and there.

>     2) A standard install location would be good. As I've messed with
>     this each time, I never know where stuff should go -- maybe
>     installers would help with that
>
> This doesn't seem to be decisive requirement to me.

It's not a strong requirement, but standard defaults do make things 
easier for everyone.

> Or some other folks may prefer installing these files
> along with their applications or keep such files in separate - project
> specific - directories.

Well, users should certainly be able to do something custom if they 
want. This is all about use-cases -- if you are building a custom app 
linked against GDAL, then you probably want to control where you put things.

However, if you are interested in the command line tools, and using GDAL 
via Python or Perl or ??, then it makes it easier to have a standard 
location.

 > Another issue of an installer may be due to a single product key
> along with the setup which would prevent from installing multiple
> versions side by side in the same environment.

Surely there are ways to accommodate that? though "dll hell" is in the 
lexicon for a reason!

>     3) If there is a standard install location, then "easy_install gdal"
>     (or setup.py build, or...) could work for the python bindings.

> I admit I don't have enough knowledge about the 'magic' tricks related
> to python-ish way installing applications.

That's the thing -- there is no magic here. If you are building a python 
extension, you need to tell the build system where its dependencies lie. 
If you are installing a pre-built python extension, then the 
dependencies need to be in a known place (or maybe on the right PATHS -- 
Windows is pretty ugly this way). Which is why a "standard" install 
location would be a good idea.

> I might also consider
> running a custom application with gdal not necessarily be the
> responsibility of a GDAL package.

well, that depends on whether you consider Python bindings a "custom 
application". In any case, I think it helps third party packages to have 
standard default install locations.

Oh for *nix -- this would be easier if we just could just put stuff in 
/usr/local/...

> You might also want to install python
> from a separate installer (either ActivePython, python.org
> <http://python.org> whatever)

True -- but it is very much a standard for third party packages to 
provide binaries for the python.org python build. Again, I'm not 
suggesting that folks should be prevented (or even discouraged) from 
doing various sorts of custom installs, just that there should be 
defaults, so that it's clear an easy for a newbie to know what to to to 
get things to "just work".

>     Another option is to have a binary installer for the python bindings
>     that includes gdal and the gdal utilities -- that would be great for
>     users like me, but I don't know how common my use case is. In that
>     case, you'd want to support a few recent pythons versions, the
>     python.org <http://python.org> binaries: 2.6, 2.7, 3.1 (maybe 2.5 too).
>
> I don't know much about this either. This may however be doable for
> those guys who know the Python packaging approach well enough.

It's not that hard (at lest once GDAL is built), but it is work.

> I don't
> think eiter of the packages referred at
> http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries would support
> this feature though.

Agreed -- that's my point!

>     One of the tricks here is which numpy to support, etc. numpy has
>     been pretty good with binary compatibility lately (except for one
>     mistake recently that was corrected)
>
>
> Not sure how this be related to a GDAL binary distribution, as far as I
> remember numpy can be installed to the Python deployment directly.

yes, but the Python bindings are built against a particular numpy. 
That's OK for version so numpy that are binary compatible, but it 
potentially fragile. Note that with the new extended buffer interface, 
it should be possible to build GDAL with full numpy support, but not 
have to compile against numpy. But I think that's only good for 2.7 and 3.*

>     However, I DON'T want gdal to give me  Python -- I use Python for
>     too many other things for that.
>
> Yes, adding more runtime environments to a simple GDAL package makes it
> more heavy weighted.

right -- I think we're talking about lightweight, GDAL only packages here.

> in many cases it's more reasonable to let the
> application (using the GDAL binaries) decide how to make a proper
> installer to run their application smoothly.

Hmm -- that's the trick -- are the Python (and Perl, and...) bindings an 
"application (using the GDAL binaries)", or are they part of GDAL? In 
many cases, the python bindings are a completely separate project from 
the library they bind, so it's a clear distinction, but not in this case.

This makes me think, though -- maybe I should think about this 
differently -- I'm trying to get the GDAL command line tools, and the 
Python bindings. Maybe I should simply consider those as separate issues 
altogether -- they don't need to share the same binaries. In that case, 
maybe the python bindings should be statically linked, or deliver the 
dlls with the bindings, and be available as an entirely stand-alone 
installer.

Indeed, having said that, I'm looking around and see that someone is 
doing that:

http://www.lfd.uci.edu/~gohlke/pythonlibs/

very nice -- I'll have to give those a test.

>     4) it might be nice if the install location for the utilities got
>     put on the user's PATH -- I don't know how hard that is in an
>     installer -- Windows really sucks in that regard.
>
> I don't think it would be beneficial in most cases. This could easily
> break other applications (using the dll-s with the same names) to fail
> unexpectedly.

I was thinking the executable utilities, not the dlls, but Windows does 
conflate those PATHs, doesn't it? (sigh)

Anyway, next time I update my Windows system (which I'll need to do 
soon), I'll think about these issues some more.

A couple notes on:

http://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries

Looking again at that page, I'm reminded why this has seemed painful. 
Under the Windows section:

"""
Minimalist windows executables are available at:

      http://download.osgeo.org/gdal/win32/1.6/gdalwin32exe160.zip
"""

these are out of date -- if they are going to exist, they really should 
be updated. They are hosted by osgeo, and thus look "official".

"""
Other plugins will be added in the same location (such as Oracle/OCI):

      http://download.osgeo.org/gdal/win32/
"""

How well maintained is this set?

"""
A more featureful set of windows binaries, including python, proj and c# 
support is available as part of the  FWTools package.
"""

no longer kept up to date, either.

"""
Windows binaries built in MinGW are available at:

      http://map.hut.fi/files/Geoinformatica/win32/

The Geoinformatica-yy-mm-dd.zip contains GDAL (usually a development 
version), Perl-GDAL, Perl, and many other things.
"""

good for MinGW users, I suppose -- I remember them not working for me, 
tough I can't recall how or why not. They also suffer from perhaps 
trying to be too much (though if it all worked, I wouldn't care, I have 
a fast network and large hard drive)


"""
Tamas Szekeres maintains a complete set of Win32 and Win64 binary 
packages (compiled with VC2003/VC2005/VC2008) available at the following 
location.

      http://vbkto.dyndns.org/sdk/
"""

These are fabulous -- maybe they should be first on the list? Though 
there is a LOT there -- you need to know what you are looking for.

"""
  OSGeo4W is a binary distribution of a broad set of open source 
geospatial software for Win32 environments (Windows XP, Vista, etc). 
OSGeo4W includes GDAL/OGR,  GRASS, MapServer?,  OpenEV,  uDig, as well 
as many other packages (about 70 as of summer 2008).
"""

I think for folks that are primarily FOSS4G folks, this is great. A bit 
of a mess if you just want GDAL though.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov