[gdal-dev] Fastest vector format for combining shapefiles

[Simon Greener]

Matt,

(Sorry for the duplicate email, I initially replied only to you and not the list.)

> In my experience the open source community rarely, if ever, defines a
> thing before creating it. What usually happens is that somebody (whether
> an individual or small group) builds somethings that works, or almost
> works, and to the extent it responds to business or functional needs a
> community coalesces around it. The define first approach seems to
> inevitably tend towards "standards wanking" as someone in the thread you
> posted said. And thanks for the pouinter. It made interesting reading
> and likely largely covers the ground this thread might, should it continue.

That's pretty damning don't you think?

OK. We need to replace shapefiles with a physical file format that does not have a lot
of the restrictions of shapefiles, is open source, flexible, standards compliant, etc etc.

In my view I think SQLite/Spatialite has all the running for the replacement for a shapefile.
The code is very stable; it is fast; it allows for very large tables to be created and populated; it is
acceptable to and by the IT community; its type system, while interesting and flexible, allows for
all that is stored in a DBase file (with, frankly, better quality control through constraints) including
dates:

DROP   TABLE test;

CREATE TABLE test (
   the_date text,
   mydate   date,
   CONSTRAINT CK_mydate CHECK ( mydate > date('2009-10-01') )
);

insert into test (the_date,mydate) values('2009-09-17',date('2009-09-01'));

Generates "SQL error: constraint failed".

insert into test (the_date,mydate) values('2009-10-17',date('now'));

select date(the_date) as the_date, mydate from test;

the_date	mydate
2009-10-17	2009-10-17

The idea that you can ship data with constrained columns is a massive step in the direction of exchanging quality data.
Why would you avoid doing so?

It has already been extended to support spatial data including RTree indexing using code from the existing open source library.

If you don't want to use ODBC/JDBC etc to access the data then you can use the C source code
and access the data (and RTree index) directly from the file.

What else is there to do for its adoption as the new shapefile for the open source community? Perhaps the spatialite code needs
to be incubated at OSGeo?

regards
Simon
--
SpatialDB Advice and Design, Solutions Architecture and Programming,
Oracle Database 10g Administrator Certified Associate; Oracle Database 10g SQL Certified Professional
Oracle Spatial, SQL Server, PostGIS, MySQL, ArcSDE, Manifold GIS, FME, Radius Topology and Studio Specialist.
39 Cliff View Drive, Allens Rivulet, 7150, Tasmania, Australia.
Website: www.spatialdbadvisor.com
   Email: simon at spatialdbadvisor.com
   Voice: +61 362 396397
Mobile: +61 418 396391
Skype: sggreener
Longitude: 147.20515 (147° 12' 18" E)
Latitude: -43.01530 (43° 00' 55" S)
GeoHash: r22em9r98wg
NAC:W80CK 7SWP3