[mapserver-users] Is there still a 2GB size limit on shapefiles?

[Frank Warmerdam]

Folks,

The changes I made were mostly focused on supporting large DBF files which
don't have the 32bit offsets issue.

The offsets for shapefiles are actually to 16bit words, so in theory if the
offsets were treated as unsigned 32bit values they could address up to 8GB
.shp files.  I don't recall however, if I ever got that working.  Even if
it was done it is not clear to me these files would work smoothly with
other applications.

I'd been hoping someone else might have authoritative comment in this
thread. :-)

Best regards,
Frank


On Wed, May 8, 2013 at 9:49 PM, Stephen Woodbridge
<woodbri at swoodbridge.com>wrote:

> In general the shapefile spec is owned by ERSI and unless they change the
> spec you will have a problem. Last I knew, there are pointers/offsets
> inside the shapefile that are only 32bits wide per the spec. it is because
> of these that you are limited to 2GB.
>
> At some point I thought Frank W or someone was working on a version of
> shapelib that could support greater than 2GB, but only programs that used
> that version would be able to read/write these larger files. I have not
> heard anything on that front in a while so I don't know if that was just
> talk, or code ever materialized.
>
> -Steve W
>
>
> On 5/9/2013 12:32 AM, Rahkonen Jukka wrote:
>
>> Hi,
>>
>> I am not sure either if reading >2GB shapefiles works with the native
>> shp reader on all platforms, but as the ticket says, you should be
>> able to use OGR connection in the mapfile even with your current
>> version. If you use OGR you can also split the shapefile into pieces
>> and read them through the ogr tileindex system which may be faster
>> sometimes.
>>
>> -Jukka Rahkonen-
>>
>> ______________________________**__ Andrea Peri wrote:
>>
>>  If it don't work you should evalute the option to use spatialite.
>>> It work well with dataset greater than 2GB . I use it with
>>> mapserver 6.3dev and gdal 1.10.
>>>
>>
>>
>>
>> 2013/5/9 Roger André <randre at gmail.com<mailto:randr**e at gmail.com<randre at gmail.com>
>> >>
>> Pretty sure I just hit
>> this<http://trac.osgeo.org/**mapserver/ticket/3514<http://trac.osgeo.org/mapserver/ticket/3514>>
>> problem in my 6.0.3
>> local stack.  I haven't tested it in the current release, but does
>> anyone know for sure that a shapefile with any component larger than
>> 2GB works in 6.2?
>>
>> Roger
>>
>> ______________________________**_________________ mapserver-users
>> mailing list
>> mapserver-users at lists.osgeo.**org <mapserver-users at lists.osgeo.org>
>> <mailto:mapserver-users@**lists.osgeo.org<mapserver-users at lists.osgeo.org>
>> >
>>
>>
>>  http://lists.osgeo.org/**mailman/listinfo/mapserver-**users<http://lists.osgeo.org/mailman/listinfo/mapserver-users>
>
>>
>>
>>
>>
>> -- ----------------- Andrea Peri . . . . . . . . . qwerty àèìòù
>> ----------------- ______________________________**_________________
>> mapserver-users mailing list mapserver-users at lists.osgeo.**org<mapserver-users at lists.osgeo.org>
>> http://lists.osgeo.org/**mailman/listinfo/mapserver-**users<http://lists.osgeo.org/mailman/listinfo/mapserver-users>
>>
>>
> ______________________________**_________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.**org <mapserver-users at lists.osgeo.org>
> http://lists.osgeo.org/**mailman/listinfo/mapserver-**users<http://lists.osgeo.org/mailman/listinfo/mapserver-users>
>



-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20130509/24b69259/attachment-0001.html>