[osgeo4w-dev] Binaries Packaging - A Strategic Investment
matt.wilkie at gov.yk.ca
Wed Mar 21 16:24:08 EDT 2012
I can affirm the importance of the Windows packaging effort, Osgeo4w, to
both my work and leisure involvement in GIS. I think O4w's web face
greatly under represents the value and maturity of the package ecosystem
and ease of deployment. Indeed part of the reason I ebbed away from
Linux in my leisure time was because I had less trouble putting together
a working GIS toolchain on Windows. O4w "just worked". On linux (ubuntu)
I spent a lot time trying to get the right build environment together,
and was never quite successful.
Speaking specifically to Osgeo4w, here are a couple areas I think could
bear some study and improvement:
== Revision management of package construction ==
Sometimes I've had to add or remove some small piece of a package in
response to specific need or defect, like creating a new environment
variable or restructuring how current path is calculated. If it were a
code project I'd commit to svn or hg and say "closes #99, doesn't
resolve path when no trailing backslash". Retaining the why something
was changed is important, there might be only a single character
difference and I or someone else in future may not realise it's on purpose.
We don't generally do this with packages though, because by and large
we're only adding 1kb of work to 10mb or more of upstream stuff, most of
which is binary.
If the fix is important enough, it means generating a new package for
the mirror. For the mirror this means storing 10mb+1kb for 1kb of
value. Each time. That's a lot of wasted storage for the server, and a
lot duplicate bandwidth use for both servers and end users. (I'm sitting
on one such change right now because I'm debating waiting for more
changes in order for it be worthwhile.)
I don't have a solution, it's just something I've observed and wonder
about from time to time.
== Needless packaging ==
There are some packages and requests for creating packages that don't
really warrant the effort of creating and maintaining yet another
package of the same thing. This is certainly true for Python modules,
which have a package ecosystem via `easy_install` and `pip`, and maybe
for others too. Some work towards playing well with others might
alleviate the total package load, and thereby allow us to focus more on
things which make a bigger difference.
Speaking again to python, maybe if we actually used and hosted a pip
environment for the python modules it would make things easier for
packagers. In building something for Osgeo it automatically is ready for
World. (Disclaimer: I've never made a pip package and have no idea how
much work it is compared to what we do currently.)
Of course there is a tension between allowing for 3rd party installation
and removal, and maintaining Osgeo4w's "batteries included" independance
and portability, two features which I think have been very important to
the project's success.
== Osgeo ==
Stepping outside Osgeo4w, I wonder if the Osgeo structure on the whole
has become too interdependent and unwieldy. I've been lightly
participating in the SAC mailing list for a bit and see a lot of
discussion but not so much progress (on the things I comprehend and are
important to me). It seems, again from my limited personal perspective,
that there is now so much to think about and so much to test that moving
forward on any particular initiative is slow and difficult.
Two small examples of this are: a request to use Osgeo4w logo on home
page and link to the o4w site from the o4w site, and a request to
close/migrate o4w tickets from the main Osgeo tracker to the o4w
tracker. Both are 6 months old and have had zero response.
Why would they, when there are things going on like the download server
for all projects failing completely, periodic and sustained CPU loads
spiking through the roof with no clear reason why, and single threads
nearing 60 messages long regarding VM upgrade problems?
Information Management and Technology
Yukon Department of Environment
10 Burns Road * Whitehorse, Yukon * Y1A 4Y9
867-667-8133 Tel * 867-393-7003 Fax
On 20-Mar-2012 11:04 PM, Frank Warmerdam wrote:
> I've mentioned this before, and I don't have anything surprising to add now.
> I just wanted to bump this topic.
> I believe that producing good quality integrated distributions of OSGeo
> binary software for a major user platforms is strategically important for
> OSGeo and would be worth an investment of moderate amounts of money to
> For me two packaging efforts stick out, though I might be biased.
> 1) OSGeo4W - I think the Windows environment is (still?) very important and
> OSGeo4W is a credible community effort to satisfy it that could benefit
> from more involvement, polish and a broader package set.
> 2) Debian/Ubuntu/LiveDVD - I believe that Ubuntu is today the dominant
> desktop/server linux system and that the packaging efforts of the DebianGIS,
> UbuntuGIS and LiveDVD groups build on one another and provide high impact.
> If board members or community members see high impact and reasonably
> priced opportunities to extend these efforts with OSGeo money I hope
> they will come forward with them. I'd also like to see us do more on the
> OSGeo web site, with case studies, etc to promote these package suites
> in a manner appropriate to their level of readiness.
> I also think the MacOS environment is very important but I'm not entirely
> clear on the best way of addressing that. Good ideas on this aspect are
> also welcome.
> Best regards,
More information about the osgeo4w-dev