[gdal-dev] Open source vector geoprocessing libraries?

[Peter J Halls]

Jan,

    for a vector archival format, surely GML is the nearest equivalent to 
Geotiff?  That should preserve all information; be vendor neutral; and it be 
possible to retrieve all information in the future.

Peter

Jan Hartmann wrote:
> I was thinking along the same lines, but more in the direction of OGR as 
> an archival standard.  I have been working with archives and stored maps 
> all my life and am now busy with lots of digital historical maps. For 
> raster maps the best storing format is Geotiff, from which all other 
> formats can be derived if you want something small or quick for high 
> performance purposes. For vector maps there is no such standard. Most 
> vector maps are exchanged as shapefiles, but this is certainly not 
> optimal. Some sort of file-based OGR format could perhaps fill this gap; 
> conceptually, it certainly is the best model there is. For archival 
> purposes however, there should be three additional options:
> 
> 1) If a map is converted to OGR and converted back, there should be an 
> option of getting it back byte-identical. Law #1 of archiving: always 
> make sure that you can get back the original
> 2) For long term archival purposes, there should be an ascii format (XML 
> or GeoJSON) associated with OGR. I have had to converted binary files 
> from a 60-bits Cyber long ago, so I know what I am talking about. God 
> knows what a computer word will look like in the age of quark-computers.
> 3) There should be some sort of lossless compression scheme associated 
> with OGR
> 
> As to additional functionality, it wouldn't take first place for me, but 
> it would be nice to have it, perhaps not baked into the format but as 
> additional libraries and an API to be linked in:
> 
> 1) Most of the time I do not need optimization for speed. I find it more 
> important to create a work-flow with as few copying steps as possible 
> (always a gigantic source of errors). Only at the last moment, e.g. for 
> setting up a high performance server, I create the necessary production 
> files, optimized for speed or memory space.
> 2) IMHO large vector maps are useless without indices. It would be nice 
> to have an indexing scheme for these OGR maps, perhaps as standalone 
> files, comparable to the OVR raster files used by gdaladdo.
> 3) Topology would be nice too. What to think about the way ArcGIS does 
> it nowadays? It uses some sort of shapefiles as base maps, but computes 
> the topology (you can choose different criteria) and puts it in separate 
> files. There is work going on for topology in the PostGIS, I believe, 
> but it is a horribly difficult subject, of course.
> 4) Regular GIS functions are already available via the GEOS library.
> 5) I am not a big fan of Metadata. Most maps are from governmental 
> organisations, and my experience with Metadata is that those 
> bureaucratic offices want to put the complete structure of their 
> specific organisation into the definition of the map. It is impossible 
> to get all these definitions into one overarching metadata system. The 
> nice thing about maps is that every map can be combined with every 
> other. The problem with (governmental) organisations is that they create 
> their own small universums, which aren't compatible with other 
> universums, and even don't know about each other's existence. It's like 
> combining Euclidean and non-Euclidean universums, don't try it! There 
> should be documentation associated with a map, of course, but that is 
> different from the basic definition of a map in terms of points, lines 
> polygons and projections.
> 
> Again and again, I am not asking for this functionality or even 
> commenting on the ongoing work on OGR; I don't know enough about its 
> internals or the way people are working on it to be in any way qualified 
> for that. These are just the thoughts of a long-term (and very happy) 
> GDAL/OGR user from an historical/archival point of view.
> 
> Jan
> 
> On 15-1-2010 9:10, Peter J Halls wrote:
>> Dear All,
>>
>> Mateusz Loskot wrote:
>> %< Snip
>>>
>>> Yes. Also, most applications I've seen using OGR do define their own
>>> data models and translate OGRFeature to features of their own types.
>>> Perhaps it would be interesting to know why they don't use OGRFeature
>>> as a part of their data model, what's missing...
>>
>> Thinking about this in terms of my own programs, I think that it is 
>> not necessarily that OGRFeature is missing anything but rather that my 
>> program data structures (objects) are designed for some specific 
>> task.  So, for example, I have a program that reads from a GPS and 
>> creates OGRFeatures for storage somewhere using OGR; another uses OGR 
>> to read data of some format and then builds a Green & Sibson 
>> neighbourhood structure from the OGRFeatures in order to measure 
>> neighbourhood characteristics and writes or updates attribute tables; 
>> and so on.  These simple OGC-type features are, for me, ideal input 
>> into or output from what are primarily research models.  Indeed, were 
>> the data structure more complex, I should probably have to unpick it 
>> into a more simple structure, like OGRFeature, in order to build 
>> appropriate data structures for whatever it was I was doing.
>>
>> Note: I am not using OGR as a component of a GIS, rather my programs 
>> are either extensions to GIS methodology (eg neighbours and cluster 
>> detection) or are designed to model the behaviour of some phenomenon.  
>> I use OGR for format independent spatial object IO because it is easy 
>> to map OGR objects to my objects.  This means that I use my own object 
>> methods for tasks like intersection detection, etc, when needed.
>>
>> Having said that, do I want more?  There are times when geometric 
>> topology is, or could be, very useful.  Currently, if I really want 
>> that, I create an ESRI 'coverage' dataset and use that as input via 
>> infolib: its not necessarily ideal and I have no idea how long that 
>> format will persist, but it serves my needs well.  I do not think I 
>> would expect OGR to offer topology functions, though: I think I would 
>> expect to use a separate but related library to build topology from 
>> OGRFeatures.
>>
>>>
>>>> There is of course some non-trivial overhead converting underlying
>>>> features into OGRFeatures, and as was noted there is some performance
>>>> impedance between OGR and GEOS due to the need to translate
>>>> geometries frequently.
>>>
>>> There usually is yet another step (cost), it is translation from
>>> OGRFeature to feature of application's data model.
>>
>> This is very true and is probably inevitable, unless one is inventing 
>> the wheel yet again.  Of course, the overhead can be minimised by the 
>> use of appropriate structures and avoiding repetition.
>>
>> Dunno how useful that is to anyone else, but if it is, then great.
>>
>> Best wishes,
>>
>> Peter
>>
>> -------------------------------------------------------------------------------- 
>>
>> Peter J Halls, GIS Advisor, University of York
>> Telephone: 01904 433806     Fax: 01904 433740
>> Snail mail: Computing Service, University of York, Heslington, York 
>> YO10 5DD
>> This message has the status of a private and personal communication
>> -------------------------------------------------------------------------------- 
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev

-- 
--------------------------------------------------------------------------------
Peter J Halls, GIS Advisor, University of York
Telephone: 01904 433806     Fax: 01904 433740
Snail mail: Computing Service, University of York, Heslington, York YO10 5DD
This message has the status of a private and personal communication
--------------------------------------------------------------------------------