[mapserver-users] large shapefile

[Stephen Woodbridge]

Steve.Toutant at inspq.qc.ca wrote:
> 
> Hi,
> I read a lot on how to optimized mapserver for large shapefile but I get 
> confused. I read on shp2tile, ogrtindex, shptree, ...
> What is the link between those tools? What is the recipe?
> thanks,
> Steve

shp2tile takes a shapefile and splits in into a lot of smaller 
shapefiles each with only a fragment of the original.

ogrtindex or tile4ms then creates a tileindex of all the fragments so 
you have a virtual reconstruction of the original shapefile.

shptree build a spatial index for a shapefile so mapserver can access it 
faster.

1) build a spatial index for you large file and retest your performance.
$ shptree mybigfile.shp

if the performance is still bad you should probably turn on debug output 
in the mapfile which requires adding the following to the MAP object in 
the mapfile:

DEBUG 5
CONFIG MS_ERRORFILE "stderr"

Then render an image and look in in the apache error log file. It will 
tell how long it took to display each layer so you can fix only the slow 
layers.

2) to use shp2tile I would create a directory and put the output of 
shp2tile in that directory:

mybigshape.shp
mybigshape/

$ shp2tile --quadtree 8000 mybigshape/mybigshape
$ find mybigshape/ -name \*.shp -print > mybigshape-tidx.in
$ tile4ms mybigshape-tidx.in mybigshape-tidx
$ shptree mybigshape-tidx
$ find mybigshape/ -name \*.shp -exec shptree {} \;

And change you LAYER in the mapfile:

From:
    DATA "mybigshape"

To:
    TILEINDEX "mybigshape-tidx"

And rerun you timing test to see how the numbers changed.

-Steve W.