[gdal-dev] Fastest vector format for combining shapefiles

[Simon Greener]

The need for a new vector file format has been discussed many times with no action initiated by the open source community on what to do.

> ESRI has said they will do so, but it's been several
> years since they first announced it and when it is finally is released
> there is no guarantee it will be under license terms open source
> projects can use. This isn't to say it will be unusable either, we
> just don't know.

Ahhh, isn't it wonderful waiting for the "crumbs that will fall from the master's table"!

And, "to wait" is part and parcel of being an ESRI-centric customer or user: strange that open source people are willing to do the same.
(Well, at least you aren't paying for the privilege of using the stuff.)

Why won't ESRI release an FDO (an Open Source open access API) provider for FGDBs rather than their own API? (I can find no reference to ESRI offering to do so for any of its formats.) Sounds like API lock-in is a design goal for the FGDB API!

Don't forget that a FGDB is full of ESRI concepts (not OGC or SQL/MM or those promoted by any other standards body) - more lock-in if it becomes the much hoped for replacement for shapefiles. And, what's more, we know nothing about what will be in the API. Where is the community engagement? Will we end up with an API via which we cannot (four examples will suffice):

1. Properly design (cf CASE tool) an FGDB (cf ESRI $$ extensions to Visio);
2. Create an FGDB from scratch;
3. Write data or create important objects (ie versions);
4. Create FGDB spatial and attribute indexes or even use them via the API (cf shapefile indexing);

These are points which have grounding in past ESRI practices. All done deliberately so you have to have a copy of ArcGIS to construct, design and get the best out of a fully specified FGDB?

And then, when there are serious bugs, you have to wait for 18 months for a fix while in the Open Source community you could get one in a matter of days or weeks?

Seriously, though, isn't open source about taking control of one's destiny and being a part of a truly open, collaborative, process and not waiting for the bully in the playground to tell you what you can and can't do, or who really isn't interested in your deadlines and real needs? Many times in my long GIS career I've had conversations with the 'true believers' over in Redlands. One was like this: "When will you support an Oracle Sdo_Geometry circle object in ArcSDE?". Reply: "Circles in GIS? We don't think you need them....".

The fixated concentration of the GIS community on physical file formats feels very much like a 1960s form of data management and computing. Logical separation from physical implementation, normalise for edit/denormalise for performance, logical separation from physical implementation, normalise for edit/denormalise for performance, logical..... oops the record is broken ....

The only reason I ever hear for physical file formats is "we need to do this for performance reasons"..... but this usually masks a lot of other reasons and assumptions (like it is "quicker and easier" that soon morphs into a management nightmare).....

cynically
Simon
-- 
SpatialDB Advice and Design, Solutions Architecture and Programming,
Oracle Database 10g Administrator Certified Associate; Oracle Database 10g SQL Certified Professional
Oracle Spatial, SQL Server, PostGIS, MySQL, ArcSDE, Manifold GIS, FME, Radius Topology and Studio Specialist.
39 Cliff View Drive, Allens Rivulet, 7150, Tasmania, Australia.
Website: www.spatialdbadvisor.com
    Email: simon at spatialdbadvisor.com
    Voice: +61 362 396397
Mobile: +61 418 396391
Skype: sggreener
Longitude: 147.20515 (147° 12' 18" E)
Latitude: -43.01530 (43° 00' 55" S)
GeoHash: r22em9r98wg
NAC:W80CK 7SWP3