[mapserver-users] HOWTO Split Large SHP Files

[Stephen Woodbridge]

Luca,

There are two things that you can try before you start breaking up
shapefiles:

1) create shptree indexes for all you big shape files this will speed up
mapserver because it does not have to look at every record in the shape
to see if it needs to be drawn.

2) create a tileindex which has each country in it and a shptree for the
tileindex. You will have to reference the tileindex in your layer
definitions.

Both the tileindex and the shptree will make thins MUCH faster as you
zoom in and have less and less of the overall data being display on the
map.

-Steve

Luca Pescatore wrote:
> 
> >Luca,
> >I don't think there is a general tool to do that, but if you know C it
> >is pretty easy to use shplib to write utility to read the shape file and
> >dump specific records to different files. You can also do this using
> >Perl and the XBase and shplib addins.
> Argh! my perl Knowledge is very bad! I can try to do something in PHP, but
> i don't know
> very well PHP Class for MapServer, and i don't know how to split lines (my
> shapefile uses) in
> small pieces. If someone may help me i can do that and put this piece of
> code to GPL.
> 
> >Why do you want to split the file?
> 
> You can test at this address : 195.219.123.15 (in few week will be
> protected by login/pwd)
> 
> At maximum zoom get a lot of time for displaying roads.
> 
> My Machine is :
> AMD Athlon 1 Ghz
> 512 Mb RAM
> 100 Gb IDE
> 
> At this moment i have tried to get some info for testing performances, nxt
> Week i
> will upgrade to 1 Gb of RAM, after i will re-test with same values for
> undestanding how many
> %s may i increase performances with the same machine.
> 
> Every country have a set of files for displaying data, Roads are like 300
> Mb for every set of file.
> 
> All Shapes get like 40 Gb of space.
> 
> >performance? have you tried building a shptree index?
> Yes, it works also with a single file ? My Roads file (e.g. for northern
> italy is 400 Mb), with
> shptree may i get better performances ?
> 
> Best Regards,
>          Luca