[osgeo4w-dev] Binaries Packaging - A Strategic Investment

Matt Wilkie matt.wilkie at gov.yk.ca
Wed Mar 21 16:24:08 EDT 2012


Hello Everybody,

I can affirm the importance of the Windows packaging effort, Osgeo4w, to 
both my work and leisure involvement in GIS. I think O4w's web face 
greatly under represents the value and maturity of the package ecosystem 
and ease of deployment. Indeed part of the reason I ebbed away from 
Linux in my leisure time was because I had less trouble putting together 
a working GIS toolchain on Windows. O4w "just worked". On linux (ubuntu) 
I spent a lot time trying to get the right build environment together, 
and was never quite successful.

Speaking specifically to Osgeo4w, here are a couple areas I think could 
bear some study and improvement:

== Revision management of package construction ==

Sometimes I've had to add or remove some small piece of a package in 
response to specific need or defect, like creating a new environment 
variable or restructuring how current path is calculated. If it were a 
code project I'd commit to svn or hg and say "closes #99, doesn't 
resolve path when no trailing backslash". Retaining the why something 
was changed is important, there might be only a single character 
difference and I or someone else in future may not realise it's on purpose.

We don't generally do this with packages though, because by and large 
we're only adding 1kb of work to 10mb or more of upstream stuff, most of 
which is binary.

If the fix is important enough, it means generating a new package for 
the mirror.  For the mirror this means storing 10mb+1kb for 1kb of 
value. Each time. That's a lot of wasted storage for the server, and a 
lot duplicate bandwidth use for both servers and end users. (I'm sitting 
on one such change right now because I'm debating waiting for more 
changes in order for it be worthwhile.)

I don't have a solution, it's just something I've observed and wonder 
about from time to time.


== Needless packaging ==

There are some packages and requests for creating packages that don't 
really warrant the effort of creating and maintaining yet another 
package of the same thing. This is certainly true for Python modules, 
which have a package ecosystem via `easy_install` and `pip`, and maybe 
for others too. Some work towards playing well with others might 
alleviate the total package load, and thereby allow us to focus more on 
things which make a bigger difference.

Speaking again to python, maybe if we actually used and hosted a pip 
environment for the python modules it would make things easier for 
packagers. In building something for Osgeo it automatically is ready for 
World. (Disclaimer: I've never made a pip package and have no idea how 
much work it is compared to what we do currently.)

Of course there is a tension between allowing for 3rd party installation 
and removal, and maintaining Osgeo4w's "batteries included" independance 
and portability, two features which I think have been very important to 
the project's success.


== Osgeo ==

Stepping outside Osgeo4w, I wonder if the Osgeo structure on the whole 
has become too interdependent and unwieldy. I've been lightly 
participating in the SAC mailing list for a bit and see a lot of 
discussion but not so much progress (on the things I comprehend and are 
important to me). It seems, again from my limited personal perspective, 
that there is now so much to think about and so much to test that moving 
forward on any particular initiative is slow and difficult.

Two small examples of this are: a request to use Osgeo4w logo on home 
page and link to the o4w site from the o4w site, and a request to 
close/migrate o4w tickets from the main Osgeo tracker to the o4w 
tracker. Both are 6 months old and have had zero response.

Why would they, when there are things going on like the download server 
for all projects failing completely, periodic and sustained CPU loads 
spiking through the roof with no clear reason why, and single threads 
nearing 60 messages long regarding VM upgrade problems?

best regards,

matt wilkie
--------------------------------------------
Geomatics Analyst
Information Management and Technology
Yukon Department of Environment
10 Burns Road * Whitehorse, Yukon * Y1A 4Y9
867-667-8133 Tel * 867-393-7003 Fax
http://environmentyukon.gov.yk.ca/geomatics/
--------------------------------------------


On 20-Mar-2012 11:04 PM, Frank Warmerdam wrote:
> Folks,
>
> I've mentioned this before, and I don't have anything surprising to add now.
> I just wanted to bump this topic.
>
> I believe that producing good quality integrated distributions of OSGeo
> binary software for a major user platforms is strategically important for
> OSGeo and would be worth an investment of moderate amounts of money to
> promote.
>
> For me two packaging efforts stick out, though I might be biased.
>
> 1) OSGeo4W - I think the Windows environment is (still?) very important and
> OSGeo4W is a credible community effort to satisfy it that could benefit
> from more involvement, polish and a broader package set.
>
> 2) Debian/Ubuntu/LiveDVD - I believe that Ubuntu is today the dominant
> desktop/server linux system and that the packaging efforts of the DebianGIS,
> UbuntuGIS and LiveDVD groups build on one another and provide high impact.
>
> If board members or community members see high impact and reasonably
> priced opportunities to extend these efforts with OSGeo money I hope
> they will come forward with them.  I'd also like to see us do more on the
> OSGeo web site, with case studies, etc to promote these package suites
> in a manner appropriate to their level of readiness.
>
> I also think the MacOS environment is very important but I'm not entirely
> clear on the best way of addressing that.  Good ideas on this aspect are
> also welcome.
>
> Best regards,


More information about the osgeo4w-dev mailing list