[Aust-NZ] RE: Finding a way forward - GeoNetwork as platform to support - unclassified

[COCHRANE BYRON, MR]

Good Morning,

First, let me just voice my appreciation for all the great work that Simon has done, both on GeoNetwork and the MEST version.  I know Simon has contributed a great deal to the development of GeoNetwork we are very lucky to have him on our team. His work to avoid the forking issues has been tremendous.  Kudos.  If my comments came across as too negative or minimised your contributions, my apologies.  

Personally, I think this is a great discussion to have.  And I agree with Bruce that this would be a great showcase for FOSS4G.  Of course I have a good deal of self interest involved in seeing growth and progress in GeoNetwork as both a tool and a community.  The main point of my last email was to illustrate some of the pitfalls I see in our way.  While my programming skills are not nearly as proficient as someone like Simon ( I am working on that :)), I have gained over the past 15 years, some pretty good experience with implementations of metadata systems (with varying degrees of success).

I see this community as a place to help us progress with both our individual and collective goals.  My own efforts are currently focused on developing an automated Spatial Metadata Extraction Tool I call SMET that will integrate with GeoNetwork.  I am also interested in, among other things, workflow and notification mechanisms for GeoNetwork, and developing/collecting XSL translators for different metadata formats.  I would like to find others interested in these topics.

Cheers,
Byron


-----Original Message-----
From: Simon Pigot [mailto:sppigot at utas.edu.au] 
Sent: Monday, 19 May 2008 9:36 p.m.
To: COCHRANE BYRON, MR
Cc: aust-nz at lists.osgeo.org; anzlicmet-l at listserv.its.utas.edu.au
Subject: Re: [Aust-NZ] RE: Finding a way forward - GeoNetwork as platform to support - unclassified

Hi everyone,

Bruce, to my recollection, the September '07 Geonetwork technical meeting did not say a production version would be ready in a few weeks, instead I recall that BlueNet was asked to develop a basic version 1.0 metadata entry tool which included the ANZLIC profile - that was delivered. Everything since then has been version 1.1 :-). Note: the MEST is based on Geonetwork 2.2 and has been since BETA4 in early March
- so it's not behind anyone :-).

Up until last month we've fed a lot of the code from the MEST back into the trunk but there is a need to write proposals and get them approved by the community before some of the later changes can be committed. 
Community approval is a good thing but it takes time to get this together and slot things in to the trunk - that time lag would be a problem for anyone (not just BlueNet) - especially when the priority for us has been features we need.

BlueNet did also commit to coordinating development of the MEST release with the trunk (at the Sept '07 meeting) - which we've done - indeed there hasn't been much of the usual danger of 'fork'ing :-) because we've been pretty assiduous at keeping the MEST synced with the trunk.

In the absence of a plugin architecture for profile support, then maybe the following would be steps forward:

1. Get rid of the word BETA (and MEST which I don't like either) - the current code is based on 2.2 final anyway - the BETA label is just something BlueNet has used because it has shown our progress toward having a production tool that we can use for the BlueNet researcher workshops to be held over the next few weeks. The next release will be the final because we've frozen for the workshops now. Maybe calling it GeoNetwork 2.2 ANZLIC would be less confusing (the current BETA5 already has an 'About' menu that tells you a bit about who did what and when).

2. We need to develop and submit proposals for the functions that were added recently to GeoNetwork 2.2 ANZLIC (or whatever we finally decide to call it) and that are not in the trunk - we need to keep the number of differences between GeoNetwork 2.2 ANZLIC and the trunk to a minimum because it makes syncing with the trunk easier. BTW: It's not only skins Byron - its also about slightly more fundamental things not yet in the
2.2 trunk eg. saxon for xslt2, temporal search, separating editing & ownership rights, remote html in Z etc

3. A set of instructions needs to be provided that will allow anyone to take the profiles from GeoNetwork 2.2 ANZLIC and add them to an install of the raw trunk. This would suit those that want to customise the trunk for themselves. (I think this is what Byron was suggesting?).

This all needs more discussion - but in a positive way - its not about lack of progress - its more about the expanding community, changing/more sophisticated needs and moving on from the things we agreed at the Sept
07 meeting.

Cheers,
Simon

COCHRANE BYRON, MR wrote:
> Bruce, et al,
>
> GeoNetwork has become a subject that has involved a good deal of my professional time and concern.  So I am very interested in its development. 
>
> First, a little background.  I have been using GeoNetwork since March of last year.  As Data Manager, I needed a good metadata cataloguing tool to manage the geospatial data holdings of the NZDF.  Then, as now, I determined GeoNetwork to be the "best of breed" software for performing this duty.  The fact that it was Opensource was, in my mind, a plus.  Since that time I have moved into the role of GIS Developer, but the implementation and support of GeoNetwork has remained a major focus of my work.  Today we have approximately 1500 records in our catalogue and are running GeoNetwork 2.1 and will soon be migrating to GeoNetwork 2.2.
>
> I came to my initial decision to use GeoNetwork without knowing that ANZLIC was also considering moving their MEST effort to this platform.  The fact that they latter did so made it much easier to convince managers that this was the right path to take - particularly after the decision by BlueNet to include the ANZLIC and ADO profiles was made. 
>
> To date, I have not implemented any of the BlueNet versions of GeoNetwork.  At this point, I am unlikely to do so in the near future.  There are many reasons for this, and I would like to elaborate on a few of them.  First is the problem that you identified, Bruce - the tool has not moved out of Beta and watching the trac, I am uncertain when this may happen.  Another reason is that we have been ahead of MEST in release versions.  I had already moved to GeoNetwork 2.1 when the MEST was at 2.0.3. 
>
> Most importantly, the MEST tool is becoming more than what we want.  What I mean is that we want a tool that includes the ANZLIC and ADO profiles and MEST does that, but I am not so interested in the gui changes on top of GeoNetwork.  Not that many of them aren't useful improvements, they are, and I would use them if they were included in core GeoNetwork.  Luckily, Simon is a core contributor to GeoNetwork so it is likely that much of his work will merge back with trunk.  But I have done my own customising of GeoNetwork to meet our own needs and for sake of simplicity, would rather apply these changes to trunk GeoNetwork trunk rather than a fork when GeoNetwork updates come.
>
> I am very interested in improvements to GeoNetwork that would allow the ANZLIC and ADO profiles to be more easily plugged into my own implementation of GeoNetwork.  GeoNetwork contributors like Simon Pigot and François Prunayre have been looking at this.  I am willing to contribute whatever help I can if this is important enough to get GeoNetwork more broadly implemented in the region, but right now my GeoNetwork development efforts are more in the areas of automated metadata capture and workflow aids.
>
> To sum it up, I am interested in the tools of MEST but not so much the skin. I for one, would like to see the name MEST disappear.  It adds confusion in that I find myself needing to explain that "MEST IS GeoNetwork" again and again.  At least the release numbers could be synced.  And, as I have indicated, it would be useful if the tools improvements in MEST, especially the profile schemas, could be separated from the gui improvements.
>
> Some of these problems I believe are inherent in government trying to adapt to the world of Opensource.  I think that we as a group can and should help.  Asking a contractor to perform work that is open source is new to most administrators and developers in government.  At least I am finding this to be true as we have hired contract support for developing an automated metadata harvester that I am developing to run along side GeoNetwork.
>
> Another problem I see is one that seems inherent to Opensource.  This problem is forking of the source code.  I am worried that MEST may become a permanent fork of GeoNetwork and I do not think this a good thing.  Luckily, since Simon is a contributor to GeoNetwork, this risk is somewhat minimised, but by no means removed.  As we pool our efforts to develop MEST we need to keep very clear on what is MEST specific and what is general GeoNetwork in order to avoid the forking problem.
>
> A third problem I see is one that seems inherent in metadata itself.  What I have seen through my career is what I consider an over-emphasis on getting the model right before implementing anything.  While the proper schema is important, I did not think that the lack of a ANZLIC or military profile was sufficient reason to delay our implementing a metadata solution now.  Yes, it does mean that we will need to revisit all we have in our system now will need to be revisited when we implement a different schema, we would need to do this anyway when the schema changes - and the schemas will change, or our own definitions of how the schemas be completed change.  What is important is that it helps us do our work. GeoNetwork is good now.  I see no reason to delay implementing it.  What we need is a good community of local developers and users to help others with implementations and migrations.
>
> I think GeoNetwork is an excellent opportunity to prove once again the worth of Opensource - particularly for GIS.  There is very little out there to compete with GeoNetwork capabilities in the metadata catalogue world.  And as we all know, the payoffs to a well functioning distributed Metadata Catalogue network would be enormous.  More than anything, of course, open source is about the community.  Combining our efforts to finding solutions to the MEST-GeoNetwork problems is just what open source is about.  I feel that part of the failure of MEST to progress faster is that it has fallen into the old model of just hire someone to do it and wait for the results rather than do it ourselves as hackers are wont to do by nature.
>
> We have begun making some efforts at a GeoNetwork support community here in New Zealand, but it is very small.  If we could combine efforts across the Tasman it would definitely be a good thing.
>
> Cheers,
> Byron Cochrane
> GIS Developer
> NZDF Joint Geospatial Support Facility HMNZ Naval Base (Private Bag 
> 32901), Devonport, Auckland 9, New Zealand
> Ph.: 9-446-1836
> [397-8236 (internal)]
> email : byron.cochrane at nzdf.mil.nz
>
>
> ----------------------------------------------------------------------
>
>
>
> I'm writing this email as a member of the Australia - New Zealand Chapter of the Open Source Geospatial Foundation (OSGeo-AustNZ). This chapter is currently in the final stages of formation.
>
>
>
> I'm concerned at how ANZLIC's adoption of the GeoNetwork open source application as our spatial metadata tool is being handled and also at a perceived lack of progress in getting a production version of this application out.
>
>
> 
> Please note: This email is not about trying to apportion blame, but about ***trying to find a way forward that benefits us all***. 
>
>
>
> _Situation_
>
> I participated in a two day ANZLIC GeoNetwork Workshop in September last year http://www.osdm.gov.au/Metadata/GeoNetwork/Committees/default.aspx.
>
> I understood from this workshop that we would have a version of GeoNetwork to use in a Production environment within a short period of time (weeks, not months).
>
> The Victorian Spatial Information Infrastructure organisation subsequently commissioned the CRC-Spatial Information to undertake testing of the tool to determine its suitability. I understand that this work found the tool to be suitable.
>
> ANZLIC and the Australian Government subsequently adopted GeoNetwork for spatial metadata use http://www.osdm.gov.au/Metadata/GeoNetwork/default.aspx.
>   
>
>
> *** Eight months *** after this workshop we still **do not** have a 
> production version of GeoNetwork that supports the 'ANZLIC Profile 
> of ISO 19115/19139' (AP)
>
> 
>
> What we are doing at the moment as an ANZLIC spatial community is clearly not working.
>
>
>
> I'm hearing a number of reports of organisations considering attempting to implement their own independent solutions.
>
> If this did occur, we as an Australian and New Zealand spatial community would have lost a wonderful opportunity to pool resources to develop a superior resource that benefits us all.
>
>
> However, regardless of the problems, we still have considerable good will in trying to get this to work and to find a way forward.
>
>
>
> _Finding new ways forward_
>
>
> I see this as a critical issue for both the (AustNZ) Open Source spatial community and for the ANZLIC community to resolve together:
>
>
> - ANZLIC have taken a calculated gamble that open source software would be a good way to provide a free version of a tool that supports the AP and have invested significant time and effort into trying to make this happen.
>
> - Significant momentum is being developed for the use of Open Source software within the Australian and New Zealand Governments and industry alike.
>
> - OSGeo-AustNZ is developing considerable momentum as the local focal point for Open Source spatial software. OSGeo-AustNZ has been successful in attracting the International Conference for Free and Open Source Software for Geospatial to Sydney Australia in 2009 (FOSS4G-2009).
>
>
>
> We all understand that utilising Open Source software and development models is an area that is not well understood in many organisations.
>
>
> Is there a role for OSGeo-AustNZ to *help* move the AP work forward to
> enable:
>
> - ANZLIC and the Australian and New Zealand spatial communities to 
> have a production version of GeoNetwork that supports the AP in a 
> reasonably short period of time;
>
> - OSGeo-AustNZ and ANZLIC to have a success story for FOSS4G-2009;
>
> - Australia and New Zealand have a vibrant open source community to support GeoNetwork (the OS application), the GeoNetwork OS community and more importantly ongoing AP development work based on GeoNetwork?
>
>
>
> We have a considerable depth of experience in both spatial and a wide variety of Open Source projects within OSGeo-AustNZ. How can this be used?
>
>
>
>
> How can we harness the good intent of a large number of government departments and companies to build a GeoNetwork AP community where:
>
> - anyone is free to commission work;
>
> - anyone with the relevant technical background can develop the 
> necessary skills to actively participate in development effort;
>
> - we ensure that there is appropriate QA on design, build, test and 
> implementation to deliver a high quality product;
>
> - we work as a part of the parent GeoNetwork community and regularly 
> contribute development upstream;
>
> - we release early and often, to ensure that the ANZLIC spatial communities have quality tools to work with?
>
>
>
>
> I don't have the answers to these questions. However I believe that we as a combined community (OSGeo-AustNZ and ANZLIC) do.
>
>
>
>
> _Where to from here_
>
> I'd like to see this issue debated openly on the OSGeo-AustNZ list and using tried and tested Open Source methodology to find an effective way forward.
>
> If you are not on this list yet, please consider subscribing to the 
> list
> at:
>
> http://lists.osgeo.org/mailman/listinfo/aust-nz
>
>
>
>
> I look forward to reading your responses.
>
>
>
>
> Bruce Bannerman
>
>
>
>
>
>
>
> -------------- next part -------------- A non-text attachment was 
> scrubbed...
> Name: not available
> Type: application/pgp-signature
> Size: 189 bytes
> Desc: This is a digitally signed message part Url : 
> http://lists.osgeo.org/pipermail/aust-nz/attachments/20080518/10f45d2e
> /attachment-0001.bin
>
> ------------------------------
>
> _______________________________________________
> Aust-NZ mailing list
> Aust-NZ at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/aust-nz
>
>
> End of Aust-NZ Digest, Vol 10, Issue 7
> **************************************
>
> The information contained in this Internet Email message is intended 
> for the addressee only and may contain privileged information, but not 
> necessarily the official views or opinions of the New Zealand Defence Force.
> If you are not the intended recipient you must not use, disclose, copy 
> or distribute this message or the information in it.
>
> If you have received this message in error, please Email or telephone 
> the sender immediately.
>   
> _______________________________________________
> Aust-NZ mailing list
> Aust-NZ at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/aust-nz
>   

The information contained in this Internet Email message is intended
for the addressee only and may contain privileged information, but not
necessarily the official views or opinions of the New Zealand Defence Force.
If you are not the intended recipient you must not use, disclose, copy or 
distribute this message or the information in it.

If you have received this message in error, please Email or telephone
the sender immediately.