[OSGeo-Discuss] idea for an OSGeo project -- a new, open data format
woodbri at swoodbridge.com
Tue Nov 13 08:12:19 PST 2007
P Kishor wrote:
> So, I am thinking, Shapefile is the de facto data standard for GIS
> data. That it is open (albeit not Free) along with the deep and wide
> presence of ESRI's products from the beginning of the epoch, it has
> been widely adopted. Existence of shapelib, various language bindings,
> and ready use by products such as MapServer has continued to cement
> Shapefile as the format to use. All this is in spite of Shapefile's
> inherent drawbacks, particularly in the area of attribute data
> What if we came up with a new and improved data format -- call it
> "Open Shapefile" (extension .osh) -- that would be completely Free,
> single-file based (instead of the multiple .shp, .dbf, .shx, etc.),
> and based on SQLite, giving the .osh format complete relational data
> handling capabilities. We would require a new version of Shapelib,
> improved language bindings, make it the default and preferred format
> for MapServer, and provide seamless and painless import of regular
> .shp data into .osh for native rendering. Its adoption would be quick
> in the open source community. The non-opensource community would
> either not give a rat's behind for it, but it wouldn't affect them...
> they would still work with their preferred .shp until they learned
> better. By having a completely open and Free single-file based, built
> on SQLite, fully relational dbms capable spatial data format, it would
> be positioned for continued improvement and development.
> Is this too crazy?
Well, yes and no.
I think this idea of an open free format has some merit.
I'm not sure the your proposed solution has had enough analysis and
design or requirements backing it to have merit, but it is a stake in
I think you have to first look at the plus and minus of the existing
formats. What are your design requirements? One size fits all will not
fit anyone well.
For me shapefiles work well because they are fast! This is my number one
requirement and I can live with most other issues. For North America, I
have in the ballpark of 27M street segments, and 20-30GB of shapefile
data with qix indexes and tileindexes. I have to serve 5M map draws a
week (~440k/day) in under 1.5sec per draw. So fine if you say continue
to use shapefiles, but not so much if you tell most people that because
you get very little adoption.
There needs to be REAL value in a new data format. Having a new format
for the sake of being able to say it is free, has zero value in my book
if there are not other strong compelling reasons for people to move to
it. If people do not move to it, it will not become a defacto standard
or a standard of any use. Data vendors will not support it for data
distribution, etc. It would be better to invest everyone's time
elsewhere if there is not real concrete value to be had by this new format.
My 2 cents,
More information about the Discuss