[Board] Fwd: [osgeo4w-dev] Binaries Packaging - A Strategic Investment

[Frank Warmerdam]

fyi...


---------- Forwarded message ----------
From: Matt Wilkie <matt.wilkie at gov.yk.ca>
Date: Wed, Mar 21, 2012 at 1:24 PM
Subject: Re: [osgeo4w-dev] Binaries Packaging - A Strategic Investment
To: Frank Warmerdam <warmerdam at pobox.com>
Cc: OSGeo-Board <board at lists.osgeo.org>, Alan Boudreault
<aboudreault at mapgears.com>, Brian Hamlin <maplabs at light42.com>,
osgeo4w-dev <osgeo4w-dev at lists.osgeo.org>, "ubuntu at lists.osgeo.org"
<ubuntu at lists.osgeo.org>


Hello Everybody,

I can affirm the importance of the Windows packaging effort, Osgeo4w,
to both my work and leisure involvement in GIS. I think O4w's web face
greatly under represents the value and maturity of the package
ecosystem and ease of deployment. Indeed part of the reason I ebbed
away from Linux in my leisure time was because I had less trouble
putting together a working GIS toolchain on Windows. O4w "just
worked". On linux (ubuntu) I spent a lot time trying to get the right
build environment together, and was never quite successful.

Speaking specifically to Osgeo4w, here are a couple areas I think
could bear some study and improvement:

== Revision management of package construction ==

Sometimes I've had to add or remove some small piece of a package in
response to specific need or defect, like creating a new environment
variable or restructuring how current path is calculated. If it were a
code project I'd commit to svn or hg and say "closes #99, doesn't
resolve path when no trailing backslash". Retaining the why something
was changed is important, there might be only a single character
difference and I or someone else in future may not realise it's on
purpose.

We don't generally do this with packages though, because by and large
we're only adding 1kb of work to 10mb or more of upstream stuff, most
of which is binary.

If the fix is important enough, it means generating a new package for
the mirror.  For the mirror this means storing 10mb+1kb for 1kb of
value. Each time. That's a lot of wasted storage for the server, and a
lot duplicate bandwidth use for both servers and end users. (I'm
sitting on one such change right now because I'm debating waiting for
more changes in order for it be worthwhile.)

I don't have a solution, it's just something I've observed and wonder
about from time to time.


== Needless packaging ==

There are some packages and requests for creating packages that don't
really warrant the effort of creating and maintaining yet another
package of the same thing. This is certainly true for Python modules,
which have a package ecosystem via `easy_install` and `pip`, and maybe
for others too. Some work towards playing well with others might
alleviate the total package load, and thereby allow us to focus more
on things which make a bigger difference.

Speaking again to python, maybe if we actually used and hosted a pip
environment for the python modules it would make things easier for
packagers. In building something for Osgeo it automatically is ready
for World. (Disclaimer: I've never made a pip package and have no idea
how much work it is compared to what we do currently.)

Of course there is a tension between allowing for 3rd party
installation and removal, and maintaining Osgeo4w's "batteries
included" independance and portability, two features which I think
have been very important to the project's success.


== Osgeo ==

Stepping outside Osgeo4w, I wonder if the Osgeo structure on the whole
has become too interdependent and unwieldy. I've been lightly
participating in the SAC mailing list for a bit and see a lot of
discussion but not so much progress (on the things I comprehend and
are important to me). It seems, again from my limited personal
perspective, that there is now so much to think about and so much to
test that moving forward on any particular initiative is slow and
difficult.

Two small examples of this are: a request to use Osgeo4w logo on home
page and link to the o4w site from the o4w site, and a request to
close/migrate o4w tickets from the main Osgeo tracker to the o4w
tracker. Both are 6 months old and have had zero response.

Why would they, when there are things going on like the download
server for all projects failing completely, periodic and sustained CPU
loads spiking through the roof with no clear reason why, and single
threads nearing 60 messages long regarding VM upgrade problems?

best regards,

matt wilkie
--------------------------------------------
Geomatics Analyst
Information Management and Technology
Yukon Department of Environment
10 Burns Road * Whitehorse, Yukon * Y1A 4Y9
867-667-8133 Tel * 867-393-7003 Fax
http://environmentyukon.gov.yk.ca/geomatics/
--------------------------------------------



On 20-Mar-2012 11:04 PM, Frank Warmerdam wrote:
>
> Folks,
>
> I've mentioned this before, and I don't have anything surprising to add now.
> I just wanted to bump this topic.
>
> I believe that producing good quality integrated distributions of OSGeo
> binary software for a major user platforms is strategically important for
> OSGeo and would be worth an investment of moderate amounts of money to
> promote.
>
> For me two packaging efforts stick out, though I might be biased.
>
> 1) OSGeo4W - I think the Windows environment is (still?) very important and
> OSGeo4W is a credible community effort to satisfy it that could benefit
> from more involvement, polish and a broader package set.
>
> 2) Debian/Ubuntu/LiveDVD - I believe that Ubuntu is today the dominant
> desktop/server linux system and that the packaging efforts of the DebianGIS,
> UbuntuGIS and LiveDVD groups build on one another and provide high impact.
>
> If board members or community members see high impact and reasonably
> priced opportunities to extend these efforts with OSGeo money I hope
> they will come forward with them.  I'd also like to see us do more on the
> OSGeo web site, with case studies, etc to promote these package suites
> in a manner appropriate to their level of readiness.
>
> I also think the MacOS environment is very important but I'm not entirely
> clear on the best way of addressing that.  Good ideas on this aspect are
> also welcome.
>
> Best regards,


-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer