[Aust-NZ] RE: Finding a way forward - GeoNetwork as platformto support - unclassified [SEC=UNCLASSIFIED]
Cameron Shorter
cameron.shorter at gmail.com
Mon May 19 18:58:54 PDT 2008
Bruce,
It is great of you to raise this discussion. Often the hardest part in
solving an issue is to identify what the root cause of the problem is,
and an open discussion like this can be very useful.
To quickly pigeon hole me: I have over 15 years experience as a software
developer, much of which involved working on large military software
projects. In parallel I've also been working on Open Source and Open
Standards Geospatial projects both in my spare time and work. I'm now
Geospatial System's Architect at LISAsoft where we have just been hired
to improve the User Interface to Geonetwork. While LISAsoft has a wealth
of experience in Open Source, OGC Catalogs, and Java, we don't have in
depth knowledge of Geonetwork yet. (I'm planning to address this over
the next couple of months).
Comparing the Geonetwork project to both proprietary and open source
projects I've worked on, I've had the following thorughts:
* From what I can see, we have some good technical developers like Simon
Pigot and Byron Cochrane working on Geonework. I don't want to imply
otherwise in this email.
* Open Source is a new business model with many advantages, but we as an
industry need to learn how to maximise the benefits offered. There needs
to be changes both at the developer level and also from the sponsors in
the way contracts are written and projects are managed.
* Complex software projects like Geonetwork will always have technical
glitches. The challenge is to effectively scope and manage these issues
and upcoming working packages.
* Open Source development is typically very open is its development
process and hence the public have visibility to all project issues very
early, compared to closed shop development who hide these embarrassing
details.
* Successful Open Source projects has contributors from many sponsors
which ensures each sponsor gains more than their own effort. However to
maximise potential, each organisation needs to adjust its management
style. Hierarchical management, focusing just on project deliverables
and not helping others will often ostracize the project from the
community and reduce the project's return on investment. Basically, you
need to give a little if you want help in return.
* Government contracts usually focus on buying "Features". Core project
infrastructure is not directly supported. Proprietary developers address
this by selling multiple licenses to cover core infrastructure
development. Government contracts need to address this. It can be
justified as "Opportunity Management". Ie, If I fund core infrastructure
development now, the project I'm sponsoring will thrive, others will
contribute back to the project, and I will indirectly benefit.
* There has been discussion about the disadvantages of branching.
Branching (in Geonetwork's case) is a direct result of contract
deadlines and quality requirements not matching priorities of other
developers. This is a standard software development solution applied to
both proprietary and open source projects. Branching allows you to have
a stable application and unstable development branch. But it comes at a
hefty management cost.
* Listening to discussions about Geonetwork, it sounds like there has
been a mis-alignment between expectations for product development and
what can actually be achieved. I'm not close enough to the project to
know specifically, but do know of likely causes:
1. For the work being done, the project seems under-resourced.
2. There is a substantial overhead involved in managing multiple
branches of software and keeping them synchronised. This might not have
been acknowledged and factored into budgets.
3. It seems that project management and software development is being
done by the same person (Simon) which again would effect his performance.
4. We as an industry are still learning about the different software
development processes inherent in working with Open Source and are
learning that we need to address different risks in our project management.
5. The scope of the software development required might not have been
appreciated initially, or there might be a case of scope creep during
the project.
6. Developers have probably been involved in important community work
which might not have been included in budgets.
=================
So how can we as Open Source Geospatial developers improve things:
1. Provide an audit of the current state of the Geonetwork project(s).
(This might already have been done)
2. Identify state of the project and provide realistic estimates for
future work.
3. Include costs involved in community involvement and support in work.
4. Question our contract deadlines and determine whether they are
enforcing an excessive overhead on maintaining multiple branches.
5. Educate stakeholders about the real state of our software. It sounds
like the software is in better shape than a "beta" release implies.
=================
As mentioned by others, I'm very keen to see Geonetwork stand out as a
huge success and I believe we can make this happen!
Ben.Searle at ga.gov.au wrote:
> Classification: CLASSIFICATION REQUIRED!
>
> Byron and Simon,
>
> I agree with Byron's thoughts in relation to the potential of GeoNetwork and
> the potential for OSGeo in this area. OSDM is keen to support this community
> and has considerable interest in GeoNetwork.
>
> OSDM has been under staffed in recent years but this is beginnoing to change
> and I would like to devote some staff (and my) time to moving this forward.
> As I have mentioned previously there is a short term (12-24 months) need for
> productive and stable tools and a longer term need for a re-architecturing of
> the GeoNetwork and GeoServer applications to better address the current and
> future Web 2.0 collaborative user needs.
>
> Some of you may be aware of the NCRIS project on the Spatial Information
> Services Stack that should kick off later this year.
>
> OSGeo can play a significant role in the Spatial Data Infrastructure space
> and OSDM is keen to work with the OSGeO community with this. Any suggestions
> on how we can help?
>
> Regards
>
> Ben Searle
> General Manager
> OSDM
>
> Ben Searle
>
> This email has been sent from a BlackBerry device provided by Geoscience
> Australia.
>
> ----- Original Message -----
> From: aust-nz-bounces at lists.osgeo.org <aust-nz-bounces at lists.osgeo.org>
> To: Simon.Pigot at utas.edu.au <Simon.Pigot at utas.edu.au>
> Cc: anzlicmet-l at listserv.its.utas.edu.au
> <anzlicmet-l at listserv.its.utas.edu.au>; aust-nz at lists.osgeo.org
> <aust-nz at lists.osgeo.org>
> Sent: Tue May 20 06:17:02 2008
> Subject: RE: [Aust-NZ] RE: Finding a way forward - GeoNetwork as platformto
> support - unclassified
>
> Good Morning,
>
> First, let me just voice my appreciation for all the great work that Simon
> has done, both on GeoNetwork and the MEST version. I know Simon has
> contributed a great deal to the development of GeoNetwork we are very lucky
> to have him on our team. His work to avoid the forking issues has been
> tremendous. Kudos. If my comments came across as too negative or minimised
> your contributions, my apologies.
>
> Personally, I think this is a great discussion to have. And I agree with
> Bruce that this would be a great showcase for FOSS4G. Of course I have a
> good deal of self interest involved in seeing growth and progress in
> GeoNetwork as both a tool and a community. The main point of my last email
> was to illustrate some of the pitfalls I see in our way. While my
> programming skills are not nearly as proficient as someone like Simon ( I am
> working on that :)), I have gained over the past 15 years, some pretty good
> experience with implementations of metadata systems (with varying degrees of
> success).
>
> I see this community as a place to help us progress with both our individual
> and collective goals. My own efforts are currently focused on developing an
> automated Spatial Metadata Extraction Tool I call SMET that will integrate
> with GeoNetwork. I am also interested in, among other things, workflow and
> notification mechanisms for GeoNetwork, and developing/collecting XSL
> translators for different metadata formats. I would like to find others
> interested in these topics.
>
> Cheers,
> Byron
>
>
> -----Original Message-----
> From: Simon Pigot [mailto:sppigot at utas.edu.au]
> Sent: Monday, 19 May 2008 9:36 p.m.
> To: COCHRANE BYRON, MR
> Cc: aust-nz at lists.osgeo.org; anzlicmet-l at listserv.its.utas.edu.au
> Subject: Re: [Aust-NZ] RE: Finding a way forward - GeoNetwork as platform to
> support - unclassified
>
> Hi everyone,
>
> Bruce, to my recollection, the September '07 Geonetwork technical meeting did
> not say a production version would be ready in a few weeks, instead I recall
> that BlueNet was asked to develop a basic version 1.0 metadata entry tool
> which included the ANZLIC profile - that was delivered. Everything since then
> has been version 1.1 :-). Note: the MEST is based on Geonetwork 2.2 and has
> been since BETA4 in early March
> - so it's not behind anyone :-).
>
> Up until last month we've fed a lot of the code from the MEST back into the
> trunk but there is a need to write proposals and get them approved by the
> community before some of the later changes can be committed.
> Community approval is a good thing but it takes time to get this together and
> slot things in to the trunk - that time lag would be a problem for anyone
> (not just BlueNet) - especially when the priority for us has been features we
> need.
>
> BlueNet did also commit to coordinating development of the MEST release with
> the trunk (at the Sept '07 meeting) - which we've done - indeed there hasn't
> been much of the usual danger of 'fork'ing :-) because we've been pretty
> assiduous at keeping the MEST synced with the trunk.
>
> In the absence of a plugin architecture for profile support, then maybe the
> following would be steps forward:
>
> 1. Get rid of the word BETA (and MEST which I don't like either) - the
> current code is based on 2.2 final anyway - the BETA label is just something
> BlueNet has used because it has shown our progress toward having a production
> tool that we can use for the BlueNet researcher workshops to be held over the
> next few weeks. The next release will be the final because we've frozen for
> the workshops now. Maybe calling it GeoNetwork 2.2 ANZLIC would be less
> confusing (the current BETA5 already has an 'About' menu that tells you a bit
> about who did what and when).
>
> 2. We need to develop and submit proposals for the functions that were added
> recently to GeoNetwork 2.2 ANZLIC (or whatever we finally decide to call it)
> and that are not in the trunk - we need to keep the number of differences
> between GeoNetwork 2.2 ANZLIC and the trunk to a minimum because it makes
> syncing with the trunk easier. BTW: It's not only skins Byron - its also
> about slightly more fundamental things not yet in the
> 2.2 trunk eg. saxon for xslt2, temporal search, separating editing &
> ownership rights, remote html in Z etc
>
> 3. A set of instructions needs to be provided that will allow anyone to take
> the profiles from GeoNetwork 2.2 ANZLIC and add them to an install of the raw
> trunk. This would suit those that want to customise the trunk for themselves.
> (I think this is what Byron was suggesting?).
>
> This all needs more discussion - but in a positive way - its not about lack
> of progress - its more about the expanding community, changing/more
> sophisticated needs and moving on from the things we agreed at the Sept
> 07 meeting.
>
> Cheers,
> Simon
>
> COCHRANE BYRON, MR wrote:
>
>> Bruce, et al,
>>
>> GeoNetwork has become a subject that has involved a good deal of my
>>
> professional time and concern. So I am very interested in its development.
>
>> First, a little background. I have been using GeoNetwork since March of
>>
> last year. As Data Manager, I needed a good metadata cataloguing tool to
> manage the geospatial data holdings of the NZDF. Then, as now, I determined
> GeoNetwork to be the "best of breed" software for performing this duty. The
> fact that it was Opensource was, in my mind, a plus. Since that time I have
> moved into the role of GIS Developer, but the implementation and support of
> GeoNetwork has remained a major focus of my work. Today we have
> approximately 1500 records in our catalogue and are running GeoNetwork 2.1
> and will soon be migrating to GeoNetwork 2.2.
>
>> I came to my initial decision to use GeoNetwork without knowing that ANZLIC
>>
> was also considering moving their MEST effort to this platform. The fact
> that they latter did so made it much easier to convince managers that this
> was the right path to take - particularly after the decision by BlueNet to
> include the ANZLIC and ADO profiles was made.
>
>> To date, I have not implemented any of the BlueNet versions of GeoNetwork.
>>
> At this point, I am unlikely to do so in the near future. There are many
> reasons for this, and I would like to elaborate on a few of them. First is
> the problem that you identified, Bruce - the tool has not moved out of Beta
> and watching the trac, I am uncertain when this may happen. Another reason
> is that we have been ahead of MEST in release versions. I had already moved
> to GeoNetwork 2.1 when the MEST was at 2.0.3.
>
>> Most importantly, the MEST tool is becoming more than what we want. What I
>>
> mean is that we want a tool that includes the ANZLIC and ADO profiles and
> MEST does that, but I am not so interested in the gui changes on top of
> GeoNetwork. Not that many of them aren't useful improvements, they are, and
> I would use them if they were included in core GeoNetwork. Luckily, Simon is
> a core contributor to GeoNetwork so it is likely that much of his work will
> merge back with trunk. But I have done my own customising of GeoNetwork to
> meet our own needs and for sake of simplicity, would rather apply these
> changes to trunk GeoNetwork trunk rather than a fork when GeoNetwork updates
> come.
>
>> I am very interested in improvements to GeoNetwork that would allow the
>>
> ANZLIC and ADO profiles to be more easily plugged into my own implementation
> of GeoNetwork. GeoNetwork contributors like Simon Pigot and François
> Prunayre have been looking at this. I am willing to contribute whatever help
> I can if this is important enough to get GeoNetwork more broadly implemented
> in the region, but right now my GeoNetwork development efforts are more in
> the areas of automated metadata capture and workflow aids.
>
>> To sum it up, I am interested in the tools of MEST but not so much the
>>
> skin. I for one, would like to see the name MEST disappear. It adds
> confusion in that I find myself needing to explain that "MEST IS GeoNetwork"
> again and again. At least the release numbers could be synced. And, as I
> have indicated, it would be useful if the tools improvements in MEST,
> especially the profile schemas, could be separated from the gui improvements.
>
>> Some of these problems I believe are inherent in government trying to adapt
>>
> to the world of Opensource. I think that we as a group can and should help.
> Asking a contractor to perform work that is open source is new to most
> administrators and developers in government. At least I am finding this to
> be true as we have hired contract support for developing an automated
> metadata harvester that I am developing to run along side GeoNetwork.
>
>> Another problem I see is one that seems inherent to Opensource. This
>>
> problem is forking of the source code. I am worried that MEST may become a
> permanent fork of GeoNetwork and I do not think this a good thing. Luckily,
> since Simon is a contributor to GeoNetwork, this risk is somewhat minimised,
> but by no means removed. As we pool our efforts to develop MEST we need to
> keep very clear on what is MEST specific and what is general GeoNetwork in
> order to avoid the forking problem.
>
>> A third problem I see is one that seems inherent in metadata itself. What
>>
> I have seen through my career is what I consider an over-emphasis on getting
> the model right before implementing anything. While the proper schema is
> important, I did not think that the lack of a ANZLIC or military profile was
> sufficient reason to delay our implementing a metadata solution now. Yes, it
> does mean that we will need to revisit all we have in our system now will
> need to be revisited when we implement a different schema, we would need to
> do this anyway when the schema changes - and the schemas will change, or our
> own definitions of how the schemas be completed change. What is important is
> that it helps us do our work. GeoNetwork is good now. I see no reason to
> delay implementing it. What we need is a good community of local developers
> and users to help others with implementations and migrations.
>
>> I think GeoNetwork is an excellent opportunity to prove once again the
>>
> worth of Opensource - particularly for GIS. There is very little out there
> to compete with GeoNetwork capabilities in the metadata catalogue world. And
> as we all know, the payoffs to a well functioning distributed Metadata
> Catalogue network would be enormous. More than anything, of course, open
> source is about the community. Combining our efforts to finding solutions to
> the MEST-GeoNetwork problems is just what open source is about. I feel that
> part of the failure of MEST to progress faster is that it has fallen into the
> old model of just hire someone to do it and wait for the results rather than
> do it ourselves as hackers are wont to do by nature.
>
>> We have begun making some efforts at a GeoNetwork support community here in
>>
> New Zealand, but it is very small. If we could combine efforts across the
> Tasman it would definitely be a good thing.
>
>> Cheers,
>> Byron Cochrane
>> GIS Developer
>> NZDF Joint Geospatial Support Facility HMNZ Naval Base (Private Bag
>> 32901), Devonport, Auckland 9, New Zealand
>> Ph.: 9-446-1836
>> [397-8236 (internal)]
>> email : byron.cochrane at nzdf.mil.nz
>>
>>
>> ----------------------------------------------------------------------
>>
>>
>>
>> I'm writing this email as a member of the Australia - New Zealand Chapter
>>
> of the Open Source Geospatial Foundation (OSGeo-AustNZ). This chapter is
> currently in the final stages of formation.
>
>>
>> I'm concerned at how ANZLIC's adoption of the GeoNetwork open source
>>
> application as our spatial metadata tool is being handled and also at a
> perceived lack of progress in getting a production version of this
> application out.
>
>> 
>> Please note: This email is not about trying to apportion blame, but about
>>
> ***trying to find a way forward that benefits us all***.
>
>>
>> _Situation_
>>
>> I participated in a two day ANZLIC GeoNetwork Workshop in September last
>>
> year http://www.osdm.gov.au/Metadata/GeoNetwork/Committees/default.aspx.
>
>> I understood from this workshop that we would have a version of GeoNetwork
>>
> to use in a Production environment within a short period of time (weeks, not
> months).
>
>> The Victorian Spatial Information Infrastructure organisation subsequently
>>
> commissioned the CRC-Spatial Information to undertake testing of the tool to
> determine its suitability. I understand that this work found the tool to be
> suitable.
>
>> ANZLIC and the Australian Government subsequently adopted GeoNetwork for
>>
> spatial metadata use http://www.osdm.gov.au/Metadata/GeoNetwork/default.aspx.
>
>>
>>
>>
>> *** Eight months *** after this workshop we still **do not** have a
>> production version of GeoNetwork that supports the 'ANZLIC Profile
>> of ISO 19115/19139' (AP)
>>
>> 
>>
>> What we are doing at the moment as an ANZLIC spatial community is clearly
>>
> not working.
>
>>
>> I'm hearing a number of reports of organisations considering attempting to
>>
> implement their own independent solutions.
>
>> If this did occur, we as an Australian and New Zealand spatial community
>>
> would have lost a wonderful opportunity to pool resources to develop a
> superior resource that benefits us all.
>
>> However, regardless of the problems, we still have considerable good will
>>
> in trying to get this to work and to find a way forward.
>
>>
>> _Finding new ways forward_
>>
>>
>> I see this as a critical issue for both the (AustNZ) Open Source spatial
>>
> community and for the ANZLIC community to resolve together:
>
>> - ANZLIC have taken a calculated gamble that open source software would be
>>
> a good way to provide a free version of a tool that supports the AP and have
> invested significant time and effort into trying to make this happen.
>
>> - Significant momentum is being developed for the use of Open Source
>>
> software within the Australian and New Zealand Governments and industry
> alike.
>
>> - OSGeo-AustNZ is developing considerable momentum as the local focal point
>>
> for Open Source spatial software. OSGeo-AustNZ has been successful in
> attracting the International Conference for Free and Open Source Software for
> Geospatial to Sydney Australia in 2009 (FOSS4G-2009).
>
>>
>> We all understand that utilising Open Source software and development
>>
> models is an area that is not well understood in many organisations.
>
>> Is there a role for OSGeo-AustNZ to *help* move the AP work forward to
>> enable:
>>
>> - ANZLIC and the Australian and New Zealand spatial communities to
>> have a production version of GeoNetwork that supports the AP in a
>> reasonably short period of time;
>>
>> - OSGeo-AustNZ and ANZLIC to have a success story for FOSS4G-2009;
>>
>> - Australia and New Zealand have a vibrant open source community to support
>>
> GeoNetwork (the OS application), the GeoNetwork OS community and more
> importantly ongoing AP development work based on GeoNetwork?
>
>>
>> We have a considerable depth of experience in both spatial and a wide
>>
> variety of Open Source projects within OSGeo-AustNZ. How can this be used?
>
>>
>>
>> How can we harness the good intent of a large number of government
>>
> departments and companies to build a GeoNetwork AP community where:
>
>> - anyone is free to commission work;
>>
>> - anyone with the relevant technical background can develop the
>> necessary skills to actively participate in development effort;
>>
>> - we ensure that there is appropriate QA on design, build, test and
>> implementation to deliver a high quality product;
>>
>> - we work as a part of the parent GeoNetwork community and regularly
>> contribute development upstream;
>>
>> - we release early and often, to ensure that the ANZLIC spatial communities
>>
> have quality tools to work with?
>
>>
>>
>> I don't have the answers to these questions. However I believe that we as a
>>
> combined community (OSGeo-AustNZ and ANZLIC) do.
>
>>
>>
>> _Where to from here_
>>
>> I'd like to see this issue debated openly on the OSGeo-AustNZ list and
>>
> using tried and tested Open Source methodology to find an effective way
> forward.
>
>> If you are not on this list yet, please consider subscribing to the
>> list
>> at:
>>
>> http://lists.osgeo.org/mailman/listinfo/aust-nz
>>
>>
>>
>>
>> I look forward to reading your responses.
>>
>>
>>
>>
>> Bruce Bannerman
>>
>>
>>
>>
>>
>>
>>
>> -------------- next part -------------- A non-text attachment was
>> scrubbed...
>> Name: not available
>> Type: application/pgp-signature
>> Size: 189 bytes
>> Desc: This is a digitally signed message part Url :
>> http://lists.osgeo.org/pipermail/aust-nz/attachments/20080518/10f45d2e
>> /attachment-0001.bin
>>
>> ------------------------------
>>
>> _______________________________________________
>> Aust-NZ mailing list
>> Aust-NZ at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/aust-nz
>>
>>
>> End of Aust-NZ Digest, Vol 10, Issue 7
>> **************************************
>>
>> The information contained in this Internet Email message is intended
>> for the addressee only and may contain privileged information, but not
>> necessarily the official views or opinions of the New Zealand Defence
>>
> Force.
>
>> If you are not the intended recipient you must not use, disclose, copy
>> or distribute this message or the information in it.
>>
>> If you have received this message in error, please Email or telephone
>> the sender immediately.
>>
>> _______________________________________________
>> Aust-NZ mailing list
>> Aust-NZ at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/aust-nz
>>
>>
>
> The information contained in this Internet Email message is intended
> for the addressee only and may contain privileged information, but not
> necessarily the official views or opinions of the New Zealand Defence Force.
> If you are not the intended recipient you must not use, disclose, copy or
> distribute this message or the information in it.
>
> If you have received this message in error, please Email or telephone
> the sender immediately.
> _______________________________________________
> Aust-NZ mailing list
> Aust-NZ at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/aust-nz
>
>
> ------
> anzlicmet-l - discussion of the anzlic metadata entry tool (met)
> to find out how to subscribe/unsubscribe etc send a blank email to anzlicmet-l-request at listserv.its.utas.edu.au
> to post send your email to anzlicmet-l at listserv.its.utas.edu.au
>
--
Cameron Shorter
Geospatial Systems Architect
Tel: +61 (0)2 8570 5050
Mob: +61 (0)419 142 254
Think Globally, Fix Locally
Commercial Support for Geospatial Open Source Solutions
http://www.lisasoft.com/LISAsoft/SupportedProducts.html
More information about the Oceania
mailing list