[gdal-dev] Open source vector geoprocessing libraries?

[Jan Hartmann]

I was thinking along the same lines, but more in the direction of OGR as 
an archival standard.  I have been working with archives and stored maps 
all my life and am now busy with lots of digital historical maps. For 
raster maps the best storing format is Geotiff, from which all other 
formats can be derived if you want something small or quick for high 
performance purposes. For vector maps there is no such standard. Most 
vector maps are exchanged as shapefiles, but this is certainly not 
optimal. Some sort of file-based OGR format could perhaps fill this gap; 
conceptually, it certainly is the best model there is. For archival 
purposes however, there should be three additional options:

1) If a map is converted to OGR and converted back, there should be an 
option of getting it back byte-identical. Law #1 of archiving: always 
make sure that you can get back the original
2) For long term archival purposes, there should be an ascii format (XML 
or GeoJSON) associated with OGR. I have had to converted binary files 
from a 60-bits Cyber long ago, so I know what I am talking about. God 
knows what a computer word will look like in the age of quark-computers.
3) There should be some sort of lossless compression scheme associated 
with OGR

As to additional functionality, it wouldn't take first place for me, but 
it would be nice to have it, perhaps not baked into the format but as 
additional libraries and an API to be linked in:

1) Most of the time I do not need optimization for speed. I find it more 
important to create a work-flow with as few copying steps as possible 
(always a gigantic source of errors). Only at the last moment, e.g. for 
setting up a high performance server, I create the necessary production 
files, optimized for speed or memory space.
2) IMHO large vector maps are useless without indices. It would be nice 
to have an indexing scheme for these OGR maps, perhaps as standalone 
files, comparable to the OVR raster files used by gdaladdo.
3) Topology would be nice too. What to think about the way ArcGIS does 
it nowadays? It uses some sort of shapefiles as base maps, but computes 
the topology (you can choose different criteria) and puts it in separate 
files. There is work going on for topology in the PostGIS, I believe, 
but it is a horribly difficult subject, of course.
4) Regular GIS functions are already available via the GEOS library.
5) I am not a big fan of Metadata. Most maps are from governmental 
organisations, and my experience with Metadata is that those 
bureaucratic offices want to put the complete structure of their 
specific organisation into the definition of the map. It is impossible 
to get all these definitions into one overarching metadata system. The 
nice thing about maps is that every map can be combined with every 
other. The problem with (governmental) organisations is that they create 
their own small universums, which aren't compatible with other 
universums, and even don't know about each other's existence. It's like 
combining Euclidean and non-Euclidean universums, don't try it! There 
should be documentation associated with a map, of course, but that is 
different from the basic definition of a map in terms of points, lines 
polygons and projections.

Again and again, I am not asking for this functionality or even 
commenting on the ongoing work on OGR; I don't know enough about its 
internals or the way people are working on it to be in any way qualified 
for that. These are just the thoughts of a long-term (and very happy) 
GDAL/OGR user from an historical/archival point of view.

Jan

On 15-1-2010 9:10, Peter J Halls wrote:
> Dear All,
>
> Mateusz Loskot wrote:
> %< Snip
>>
>> Yes. Also, most applications I've seen using OGR do define their own
>> data models and translate OGRFeature to features of their own types.
>> Perhaps it would be interesting to know why they don't use OGRFeature
>> as a part of their data model, what's missing...
>
> Thinking about this in terms of my own programs, I think that it is 
> not necessarily that OGRFeature is missing anything but rather that my 
> program data structures (objects) are designed for some specific 
> task.  So, for example, I have a program that reads from a GPS and 
> creates OGRFeatures for storage somewhere using OGR; another uses OGR 
> to read data of some format and then builds a Green & Sibson 
> neighbourhood structure from the OGRFeatures in order to measure 
> neighbourhood characteristics and writes or updates attribute tables; 
> and so on.  These simple OGC-type features are, for me, ideal input 
> into or output from what are primarily research models.  Indeed, were 
> the data structure more complex, I should probably have to unpick it 
> into a more simple structure, like OGRFeature, in order to build 
> appropriate data structures for whatever it was I was doing.
>
> Note: I am not using OGR as a component of a GIS, rather my programs 
> are either extensions to GIS methodology (eg neighbours and cluster 
> detection) or are designed to model the behaviour of some phenomenon.  
> I use OGR for format independent spatial object IO because it is easy 
> to map OGR objects to my objects.  This means that I use my own object 
> methods for tasks like intersection detection, etc, when needed.
>
> Having said that, do I want more?  There are times when geometric 
> topology is, or could be, very useful.  Currently, if I really want 
> that, I create an ESRI 'coverage' dataset and use that as input via 
> infolib: its not necessarily ideal and I have no idea how long that 
> format will persist, but it serves my needs well.  I do not think I 
> would expect OGR to offer topology functions, though: I think I would 
> expect to use a separate but related library to build topology from 
> OGRFeatures.
>
>>
>>> There is of course some non-trivial overhead converting underlying
>>> features into OGRFeatures, and as was noted there is some performance
>>> impedance between OGR and GEOS due to the need to translate
>>> geometries frequently.
>>
>> There usually is yet another step (cost), it is translation from
>> OGRFeature to feature of application's data model.
>
> This is very true and is probably inevitable, unless one is inventing 
> the wheel yet again.  Of course, the overhead can be minimised by the 
> use of appropriate structures and avoiding repetition.
>
> Dunno how useful that is to anyone else, but if it is, then great.
>
> Best wishes,
>
> Peter
>
> -------------------------------------------------------------------------------- 
>
> Peter J Halls, GIS Advisor, University of York
> Telephone: 01904 433806     Fax: 01904 433740
> Snail mail: Computing Service, University of York, Heslington, York 
> YO10 5DD
> This message has the status of a private and personal communication
> -------------------------------------------------------------------------------- 
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev