[postgis-users] shp2pgsql problem with very large .shp-files

[Jeff Hoffmann]

>> On Wed, 2007-08-15 at 09:51 +0200, Stefan Zweig wrote:
>>> hi list,
>>>
>>> i have a problem with the shp2pgsql program:
>>>
>>> this command:
>>> /usr/local/pgsql/bin/shp2pgsql -s 4326 -c -I -D -k /var/gisapp/postgreexport/_G342.shp public._g342 > /var/gisapp/postgreexport/_g342.sql
>>> works alright, no problems at all
>>>
>>> whereas this command:
>>> /usr/local/pgsql/bin/shp2pgsql -s 4326 -c -I -D -k /var/gisapp/postgreexport/_G2681.shp public._g2681 > /var/gisapp/postgreexport/_g2681.sql
>>> causes shp2pgsql to answer:
>>>
>>> "/var/gisapp/postgreexport/_G2681.shp: shape (.shp) or index files (.shx) can not be opened."
>>>
>>> ls -la in /var/gisapp/postgreexport/ says the following:
>>>
>>> -rwxrwxrwx  1 root     root    171848556 2007-08-14 18:38 _G2681.dbf
>>> -rwxrwxrwx  1 root     root                143 2007-08-14 18:38 _G2681.prj
>>> -rwxrwxrwx  1 root     root  4565523268 2007-08-14 18:38 _G2681.shp
>>> -rwxrwxrwx  1 root     root    124980820 2007-08-14 18:38 _G2681.shx
>>> -rwxrwxrwx  1 root     root                308 2007-08-15 09:35 _G342.dbf
>>> -rwxrwxrwx  1 root     root                143 2007-08-15 09:35 _G342.prj
>>> -rwxrwxrwx  1 root     root                716 2007-08-15 09:35 _G342.shp
>>> -rwxrwxrwx  1 root     root                276 2007-08-15 09:35 _G342.shx
>>>
>>>
>>> so the only difference between shp-file _g2681 and shp-file _g342 is that _g2681 is an extremely big one. (and actually there is another difference: _G342.shp is a point shape, _g2681.shp is a line shape)
>>>
>>> any ideas?

This is just a shot in the dark, I've never used a shapefile that large, 
but I've always been under the impression that there's a 4GB file size 
limit on shapefiles.  The header in the shapefile only has 4 bytes for 
file size, in 2 byte words, which, if you're using a signed int, is 4GB. 
  Unsigned, it's 8GB theoretically, but I think shp2pgsql reads it as a 
signed int.  Even if the program that created the shapefile wrote it as 
an unsigned length, shp2pgsql would would read the length as negative.

Again, I don't know for certain if any of this is true, but it could be 
something to look in to.

Jeff