[osgeo4w-dev] Tech Review and Experiments

[Cameron Shorter]

Frank,
I'm very excited to hear about your progress so far. In answer to your 
questions, we should start with what I consider to be business drivers:

1. We want to sell OSGeo.
2. Windows users may be GIS savvy, but not tech savvy. Simplicity to the 
user is very important.
3. Disk space and bandwidth is cheap and getting cheaper.

So:
A cut down Cygwin for OSGeo sounds like a good idea - although I note 
the dated look and feel.
I'd err on course grained rather than fine grained packaging.

Frank Warmerdam wrote:
> Folks,
>
> Yesterday I spent most of the day looking into approaches to OSGeo4W.   I
> started out with a review of NSIS, especially with regard to supporting
> fine grained packages with dependencies and with installing packages 
> from the
> network.  I came to the following conclusions.
>
> 1) I couldn't find any obvious way to express detailed dependencies 
> between
> components though others tell me there is a way.  In any event, I'm 
> concerned
> it will be hard to express complex dependencies (like python-mapscript 
> depends
> on having python and mapserver packages).
>
> 2) There are mechanisms to download files from the internet and if 
> this was
> a zip file for instance, it could be unpacked as a form of 
> installing.  But
> this seems to lose the benefits of NSIS tracking the file list for 
> uninstall
> purposes.
>
> 3) I can't see any obvious way to run an NSIS based installer to add 
> additional
> packages to an existing install or to uninstall individual components.
>
> Based on the above, and the degree of frustration I've had in past doing
> anything moderately complicated with NSIS I just couldn't see how to go
> forward with it.  So, at that point I started casting around again 
> looking
> for a many-package-net-installer.
>
> I reviewed the option of producing MSI files, possibly using something 
> like
> the WiX project.  This has the benefit of being very automatically 
> installable
> in a way that system administrators for large windows installations 
> would like.
> But I couldn't figure out how to handle this as a network installation 
> if we
> were to use even moderately fine grained packaging.
>
> My understanding is that a single MSI file can have components which the
> user can individually select.  But all the components must be embedded
> in the MSI file.  If we took the approach that all of OSGeo4W was one MSI
> file it would be (eventually) huge.
>
> If we took the alternate approach that we produce many MSI files, one
> per project or component, then we could "front end" things with an 
> installer
> build using something like NSIS to select the packages, and then fetch
> and do a UI-less install of all the MSI files.  The biggest downside of
> this approach in my mind is that (as far as I know) each MSI file ends
> up being a package in the "Add or Remove Programs" area and I'm not sure
> that removing it from there would truely understand the dependencies.  At
> the very least, I think we would need to take a more coarse grained 
> approach
> if we wanted to do that.
>
> I looked around for other open source net installer packages for a while
> and really found nothing else that seemed any closer to what I wanted.
>
> I did come to the conclusion that what I really want is what I have on
> Linux with package managers.  Basically, I have a graphical front end 
> that
> makes it easy to find all available packages, and install them from 
> the net.
> The packages understand their depencencies, and installing a package 
> pulls
> in anything it depends on.  That's what I want for OSGeo4W.
>
> Looking at it that way, an obvious candidate technology is the setup 
> program
> used for Cygwin.  It is package oriented and net-install based with good
> understanding of dependencies.  *But* I didn't want to depend on the 
> cygwin
> "unix emulation" view of the world.  I want to be able to work with 
> "real"
> win32 based applications.
>
> Pursuing this I looked at whether the Cygwin "setup" could be adapted 
> to our
> needs.  I fetched the source, modified it to talk about "OSGeo4W", and 
> created
> a setup.ini file from scratch myself and a few packages (zlib, libjpeg,
> and libtiff).  This was successful.  What I like about this approach:
>
> 1) Net install works naturally.  Setup knows how to show progress though
> a full install well, taking into account the network install aspect.
>
> 2) A reasonable GUI for selecting packages from a tree list.
>
> 3) The GUI can be rebranded for OSGeo and modified as needed.
>
> 4) I can very easily see how this approach will allow us to manage many
> packages independently, and them become part of a coherent install.
>
> Things I don't like about the Cygwin Setup approach:
>
> 1) If we use fine grained packaging, the number of packages may be
> somewhat overwhelming for folks (as I find is the case with Cygwin).
> I think the obvious approach to this using a "meta package" category as
> our primary "easy install" interface.
>
> 2) The Cygwin user interface feels a bit old fashioned and clunky.
> At the very least I think I would see if some steps (such as mirror
> selection) can be streamlined out.
>
> 3) There doesn't seem to be an out of the box approach to doing batch
> installs.  You basically need to do it through the gui interface.  I'd
> be inclined to add something for this (like "setup.exe -install 
> gdal-1.5.0"
> doing a batch install of the named package and it's dependencies).
>
> 4) There is no built-in scripting language for "install actions".  For
> Cygwin you can run a bash or .bat script for post-install actions,
> but since we won't want a bash dependency we are basically down to
> running .bat files.  It seems like some complex post-install actions
> that can't be handled via .bat files would either need to be custom
> .exe files or perhaps C code built into the setup program (as is
> done for the desktop icon generation).   We had previously
> contemplated always installing a scripting language (such as PHP)
> as part of a base install.  This would be a plausible solution.
>
> So, my question to the group is what technological approach do we want to
> take.  If there are folks advocating an NSIS approach, can you prepare a
> prototype that demonstrates net installation and good dependency 
> understanding?
>
> A related question is whether we want to do "fine grained" packages 
> like you
> find in Cygwin, and the various linux packaging systems.  For instance,
> this might mean two packages for libjpeg - one for the runtime DLL, 
> and one
> for the developer files.  Or do we want to use a more coarse grained 
> approach.
> For instance we might just bundle stuff like libjpeg, and libtiff into a
> single GDAL package, and just treat developer stuff as part of the 
> standard
> packages.
>
> Benefits of course grained:
>  o Less fudzing around for users doing installs, trying to find what they
>    want out of a long list of packages.
>  o Less packages to maintain.
>
> Benefits of fine grained:
>  o Easier to upgrade individual components.  For instance, upgrading 
> libjpeg
>    doesn't require having to be able to build the whole GDAL project.
>  o Reduces the amount of unnecessary stuff getting installed.  Select 
> just
>    what you need.
>  o Keeps things factored out more naturally so we don't fall into the
>    situation of having to add a depencency on GDAL (for instance) when
>    another package only really needs libjpeg.
>
> I personally am leaning somewhat towards fine grained, but a large 
> part of
> that is the influence of the linux (and cygwin) packaging communities who
> have already gone that way.  I'm not sure I understand all the 
> implications
> of this decision, but I feel more comfortable approach it in a well 
> explored
> and successfully used manner.
>
> I will continue a bit further with my "cygwin setup" experiment, in an 
> effort
> to at least get GDAL and it's dependencies packaged as a demonstration 
> and then
> upload it for folks to see.  After that I'll hold off till others have 
> a chance
> to make a case for other approaches (and hopefully demonstrate them!).
>
> I would note that the actual low level binary packages produced for 
> Cygwin can
> be fairly easily repackaged for other purposes.  Basically for each 
> package
> I upload a .tar.bz file and a text file expressing it's dependencies 
> on other
> packages, version info and such.  It should be relatively easy to 
> construct
> MSI files, or NSIS packages from the same binaries using some 
> scripting.  So,
> if we can decide on fine-grained vs. coarse-grained we can start 
> working on
> packages and work out the detailed of the actual setup technology over 
> a bit
> more time - or at least without undue fear of wasting all our work so 
> far.
>
> Best regards,


-- 
Cameron Shorter
Geospatial Systems Architect
Tel: +61 (0)2 8570 5050
Mob: +61 (0)419 142 254

Think Globally, Fix Locally
Commercial Support for Geospatial Open Source Solutions
http://www.lisasoft.com/LISAsoft/SupportedProducts.html