[gdal-dev] ogr2ogr problem converting UK Ordnance Survey MasterMap

[Frank Warmerdam]

On Thu, Jul 1, 2010 at 4:55 PM, Peter J Halls <P.Halls at york.ac.uk> wrote:
> Jez, Even,
>
>   there are actually several issues relating to using GDAL/OGR to read
> Ordnance Survey of Great Britain (OSGB) GML files distributed as their
> Mastermap
> product.  One of these I reported as bug #1604 - I now find that this was
> against GDAL-1.4.0 - which concerns handling 'duplicate' tokens: OGR ignores
> Namespace and so treats <osgb:point> as the starter and fouls on the
> following
> <gml:point> which contains the geometry.  There is a similar problem with
> polygon objects.

Peter,

Ouch, the namespace stripping issue is unfortunate.  I'm not sure
of a cheap fix.

>   The data Jez describes below is simpler than much of the data in the file

I did implement some degree of "complex structure flattening" when
I worked on the custom NAS (german GML profile) reader.  I thought
perhaps it had made it into the mainline GML reader, but perhaps not.
If so, I think it could be ported.

If someone files a ticket specifically on this issue I can try to address
this or perhaps more likely have Chaitanya do it as he is now getting
quite familiar with the GML driver.

> perhaps this next point is not an issue for him.  Several of the tokens are
> described in the schema snippet as 'unbounded': this means that there can be
> several instances
>
>        <osgb:changeHistory>
>                <osgb:changeDate>2004-12-19</osgb:changeDate>
>                <osgb:reasonForChange>Revised</osgb:reasonForChange>
>                <osgb:changeDate>2002-09-07</osgb:changeDate>
>                <osgb:reasonForChange>Revised</osgb:reasonForChange>
>                <osgb:changeDate>2001-03-12</osgb:changeDate>
>                <osgb:reasonForChange>New</osgb:reasonForChange>
>        </osgb:changeHistory>

Hmm, that is also somewhat ugly.  OGR has the concept of a
string list field type, so in theory this could be reduced to two
string list fields:

changeHistory_changeData: 2004-12-19, 2002-09-07,...
changeHistory_reasonForChange: Revised, Revised

I also thought I had done something like this for the NAS driver,
but perhaps it did not make it back into the mainstream GML
driver.

Likewise, if a focused ticket is filed, I'll turn this over to
Chaitanya.

>  I do not know whether the
> GDAL/OGR
> GML driver was designed primarily for writing gml rather than for reading:
> maybe.
>
>   Where does this leave us?  As I mentioned, there are also problems with
> most
> other gml readers: this is not solely an issue with GDAL/OGR.  I have an
> immediate need for the ITN data and have written my own parser to extract
> the
> information from the gml source: so far, so good.  However, as I mentioned,
> there is now the problem of how to store these data: shapefiles use the
> dBaseIV
> format and have no structure for handling these multiple attributes.  In a
> sample dataset, I have a record with 49 changeHistory records, for example;
> some
> other multiple constructs have several thousand entries.  I happen to have
> access to Oracle, although to use GDAL/OGR to write to it requires that I do
> some significant work on the oci driver: I've been trying to understand the
> code
> of that to assess what I can reasonably do.  Alternatively, I could use oci
> directly and bypass GDAL/OGR entirely.  All this, however, is non-trivial
> and
> holding me back from doing what I am supposed to be doing ... but does seem
> to
> be the only way forward, having exhausted FME, etc.

Even if we do the stringlist and related stuff you are quite
right that there aren't many output formats that will support
the esoteric arrangement well.  OGR was really intended
to read *simplistic* GML files that match existing GIS
type conventions (flat, non-repeating).  I am interested in
extending it somewhat to read important GML profiles
reasonably well, but there are limits to how much of this
can be done without a fundamental rewrite.

I really do try to discourage GML generators from using
some of these more esoteric practices.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent