[mapserver-users] Layer with over 6000 shapefiles

[Rahkonen Jukka (Tike)]

Hi,

I am almost sure that the 260 GB shapefile was not valid. The shp part has an absolute limit at 8 GB but it can also be limited to 4 GB or 2 GB http://www.gdal.org/ogr/drv_shapefile.html.
The dbf part has no limits so I believe that you had all the attributes in a huge dbf file but only the first 2/4/8 GB of geometries in shp.

I have made qix index with shptree for each original shapefile, tied them together with ogrtileindex, created qix index for the index shapefile and pointed Mapserver to use that. Our users have not complained about the speed.  They do complain about countours being polylines with straight segments instead of nice curves, and for reason, actually.

-Jukka Rahkonen-


Stephen Woodbridge wrote:

On 1/29/2014 11:37 AM, Jeff McKenna wrote:
> On 1/29/2014, 12:32 PM, EVANS, JAMES R GS-13 USAF ACC 84 RADES/SCZE wrote:
>> Hi,
>> Is there a way to add a layer that points to 6000 some odd shapefiles?  I've created a set of contour files for the whole world, and when I had the info in a single shapefile it was about 260GBs.  Mapserver would serve it up, but performance was horrible.  I now have the set broken up into about 6000 separate shapefiles, but don't know how to address that in a single layer.  I tried using gdaltindex, to create a single index file that points to all the other shapefiles, but I get an error saying the shapefiles are an unsupported type.  Is there some other way to create an index that Mapserver can use to point to this many shapefiles?
>> Thanks,
>> James
>>
>
> You can use the MapServer "tile4ms" utility
> (http://mapserver.org/utilities/tile4ms.html).  There are some good
> examples on that page for you to follow.

Check the results from tile4ms with shpdump and make sure that you dont
have all null shapes. I have run into this on Ubuntu. I believe in both
the packages and compiling for source. I haven't had time to run down
what the issues are.

-Steve

_______________________________________________
mapserver-users mailing list
mapserver-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapserver-users