[gdal-dev] Open source vector geoprocessing libraries?

[Jan Hartmann]

Personally, I find GML far too complex to be of practical use (for my 
own purposes, mind). The GML 3.1 specification is 595 pages of small 
print, almost none of which are part of existing datasets. I just want 
to store existing datasets in a system-independent way, and preferably, 
to be able to read them directly into my applications. Perhaps GML is 
the future for new datasets in large governmental or international 
organisations, although I have my doubts about that, but for the job of 
archiving what already exists it is certainly overkill. I prefer a 
small, conceptually clear standard like OGR, that already can process 
everything that exists under the sun, and can be handled by individuals 
or small companies. I really dislike standards that require massive 
bureaucracies to get implemented. We've got enough of those here in 
Europe. Again IMHO.

Oh, and OGR already includes a subset of GML 2.0. Is this comparable 
with WCS/GDAL for raster maps?

http://www.gdal.org/ogr/drv_gml.html

Jan

On 15-1-2010 12:25, Peter J Halls wrote:
> Jan,
>
>    for a vector archival format, surely GML is the nearest equivalent 
> to Geotiff?  That should preserve all information; be vendor neutral; 
> and it be possible to retrieve all information in the future.
>
> Peter
>
> Jan Hartmann wrote:
>> I was thinking along the same lines, but more in the direction of OGR 
>> as an archival standard.  I have been working with archives and 
>> stored maps all my life and am now busy with lots of digital 
>> historical maps. For raster maps the best storing format is Geotiff, 
>> from which all other formats can be derived if you want something 
>> small or quick for high performance purposes. For vector maps there 
>> is no such standard. Most vector maps are exchanged as shapefiles, 
>> but this is certainly not optimal. Some sort of file-based OGR format 
>> could perhaps fill this gap; conceptually, it certainly is the best 
>> model there is. For archival purposes however, there should be three 
>> additional options:
>>
>> 1) If a map is converted to OGR and converted back, there should be 
>> an option of getting it back byte-identical. Law #1 of archiving: 
>> always make sure that you can get back the original
>> 2) For long term archival purposes, there should be an ascii format 
>> (XML or GeoJSON) associated with OGR. I have had to converted binary 
>> files from a 60-bits Cyber long ago, so I know what I am talking 
>> about. God knows what a computer word will look like in the age of 
>> quark-computers.
>> 3) There should be some sort of lossless compression scheme 
>> associated with OGR
>>
>> As to additional functionality, it wouldn't take first place for me, 
>> but it would be nice to have it, perhaps not baked into the format 
>> but as additional libraries and an API to be linked in:
>>
>> 1) Most of the time I do not need optimization for speed. I find it 
>> more important to create a work-flow with as few copying steps as 
>> possible (always a gigantic source of errors). Only at the last 
>> moment, e.g. for setting up a high performance server, I create the 
>> necessary production files, optimized for speed or memory space.
>> 2) IMHO large vector maps are useless without indices. It would be 
>> nice to have an indexing scheme for these OGR maps, perhaps as 
>> standalone files, comparable to the OVR raster files used by gdaladdo.
>> 3) Topology would be nice too. What to think about the way ArcGIS 
>> does it nowadays? It uses some sort of shapefiles as base maps, but 
>> computes the topology (you can choose different criteria) and puts it 
>> in separate files. There is work going on for topology in the 
>> PostGIS, I believe, but it is a horribly difficult subject, of course.
>> 4) Regular GIS functions are already available via the GEOS library.
>> 5) I am not a big fan of Metadata. Most maps are from governmental 
>> organisations, and my experience with Metadata is that those 
>> bureaucratic offices want to put the complete structure of their 
>> specific organisation into the definition of the map. It is 
>> impossible to get all these definitions into one overarching metadata 
>> system. The nice thing about maps is that every map can be combined 
>> with every other. The problem with (governmental) organisations is 
>> that they create their own small universums, which aren't compatible 
>> with other universums, and even don't know about each other's 
>> existence. It's like combining Euclidean and non-Euclidean 
>> universums, don't try it! There should be documentation associated 
>> with a map, of course, but that is different from the basic 
>> definition of a map in terms of points, lines polygons and projections.
>>
>> Again and again, I am not asking for this functionality or even 
>> commenting on the ongoing work on OGR; I don't know enough about its 
>> internals or the way people are working on it to be in any way 
>> qualified for that. These are just the thoughts of a long-term (and 
>> very happy) GDAL/OGR user from an historical/archival point of view.
>>
>> Jan
>>
>> On 15-1-2010 9:10, Peter J Halls wrote:
>>> Dear All,
>>>
>>> Mateusz Loskot wrote:
>>> %< Snip
>>>>
>>>> Yes. Also, most applications I've seen using OGR do define their own
>>>> data models and translate OGRFeature to features of their own types.
>>>> Perhaps it would be interesting to know why they don't use OGRFeature
>>>> as a part of their data model, what's missing...
>>>
>>> Thinking about this in terms of my own programs, I think that it is 
>>> not necessarily that OGRFeature is missing anything but rather that 
>>> my program data structures (objects) are designed for some specific 
>>> task.  So, for example, I have a program that reads from a GPS and 
>>> creates OGRFeatures for storage somewhere using OGR; another uses 
>>> OGR to read data of some format and then builds a Green & Sibson 
>>> neighbourhood structure from the OGRFeatures in order to measure 
>>> neighbourhood characteristics and writes or updates attribute 
>>> tables; and so on.  These simple OGC-type features are, for me, 
>>> ideal input into or output from what are primarily research models.  
>>> Indeed, were the data structure more complex, I should probably have 
>>> to unpick it into a more simple structure, like OGRFeature, in order 
>>> to build appropriate data structures for whatever it was I was doing.
>>>
>>> Note: I am not using OGR as a component of a GIS, rather my programs 
>>> are either extensions to GIS methodology (eg neighbours and cluster 
>>> detection) or are designed to model the behaviour of some 
>>> phenomenon.  I use OGR for format independent spatial object IO 
>>> because it is easy to map OGR objects to my objects.  This means 
>>> that I use my own object methods for tasks like intersection 
>>> detection, etc, when needed.
>>>
>>> Having said that, do I want more?  There are times when geometric 
>>> topology is, or could be, very useful.  Currently, if I really want 
>>> that, I create an ESRI 'coverage' dataset and use that as input via 
>>> infolib: its not necessarily ideal and I have no idea how long that 
>>> format will persist, but it serves my needs well.  I do not think I 
>>> would expect OGR to offer topology functions, though: I think I 
>>> would expect to use a separate but related library to build topology 
>>> from OGRFeatures.
>>>
>>>>
>>>>> There is of course some non-trivial overhead converting underlying
>>>>> features into OGRFeatures, and as was noted there is some performance
>>>>> impedance between OGR and GEOS due to the need to translate
>>>>> geometries frequently.
>>>>
>>>> There usually is yet another step (cost), it is translation from
>>>> OGRFeature to feature of application's data model.
>>>
>>> This is very true and is probably inevitable, unless one is 
>>> inventing the wheel yet again.  Of course, the overhead can be 
>>> minimised by the use of appropriate structures and avoiding repetition.
>>>
>>> Dunno how useful that is to anyone else, but if it is, then great.
>>>
>>> Best wishes,
>>>
>>> Peter
>>>
>>> -------------------------------------------------------------------------------- 
>>>
>>> Peter J Halls, GIS Advisor, University of York
>>> Telephone: 01904 433806     Fax: 01904 433740
>>> Snail mail: Computing Service, University of York, Heslington, York 
>>> YO10 5DD
>>> This message has the status of a private and personal communication
>>> -------------------------------------------------------------------------------- 
>>>
>>> _______________________________________________
>>> gdal-dev mailing list
>>> gdal-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>