[gdal-dev] Re: Fastest vector format for combining shapefiles

[Frank Warmerdam]

Rahkonen Jukka wrote:
> I can tell now that shapefile format is not suitable at all. The shp
> part can 
> obviously not go over 2 GB limit because after that ogr2ogr throws 
> these error messages:
> ERROR 1: Error in psSHP->sHooks.FSeek() or fwrite() writing object to
> .shp file.  

Jukka,

Hmm, not the most appropriate error report.  If you file a ticket, I
or Chaitanya can try and improve this.

> Dbf seems not to have such size limit because it grew up till 32
> gigabytes.
> I will try MapInfo tab before I believe that it is just best to keep the
> 1000
> shapefiles or upload them all to PostGIS.

The .shp format explicitly uses 32bit offsets and so cannot
practically go over 8GB (it actually uses 32bit offsets to
16bit words) and I would not trust compatability over 4GB.

The dbf format does not have any offsets in it, so it can be
arbitrarily large, and recently shapelib and GDAL/OGR were updated
to handle this.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent