[mapserver-users] Displaying very large shapefiles

[Puneet Kishor]

Armin,

> I tried shptree with parameters from 4 to 10, the effect was 
> always the
> same. The response was quite slow. The index file for a 
> 600000 polygon file
> with 180 MB I use has only 2 MB, so I think the index size 
> for the huge file
> is in a normal range.


I definitely don't think so. While I have no scientific reason to back this,
I think a 2 Mb index file is wayyyyyyy too small for a 180 Mb, 600K shapes
shapefile. There is something not right with your index.

Have you tried experimenting with different depths?

pk/



 
> 
> I'm only wondering why an increase in file size and polygon 
> number of about
> 4 times increases the processing time up to 50 times or more. 
> The difference
> between displaying a 600000 and 50000 polygon shapefile however (with
> appropriate zoom level) is neglibile. I don't know if the 
> indexing reaches
> its limits when the amount of shapes is too high.
> 
> Thanks
> 
> Armin
> 
> 
> 
> ------ Original Message ------
>  From Puneet Kishor  <pkishor at GeoAnalytics.com>
>  Sent: 16/10/01, 10:23:06
>  Subject: RE: [mapserver-users] Displaying very large shapefiles
> 
> > Armin,
> 
> > Just a quick observation...
> 
> >> -----Original Message-----
> >> From: Armin Burger [mailto:armin.burger at territoriumonline.com]
> >> Sent: Tuesday, October 16, 2001 5:47 AM
> >> To: mapserver-users at lists.gis.umn.edu
> >> Subject: [mapserver-users] Displaying very large shapefiles
> >>
> >>
> >> Hi everybody,
> >>
> >> I tried to display a really huge shapefile with 2.5 million
> >> polygons (the
> >> .shp file has about 450 MB). I used the 'shptree' to
> >> calculate the spatial
> >> index. But also with the index file (.qix) it takes 
> several minutes to
> 
> > [snip]
> 
> >> handle shapefiles with too many features? The index file
> >> itself has 14 MB.
> >>
> 
> > a 14 Mb index file for a 450 Mb shapefile seems to be too 
> small an index
> > file. For example, I have a 156 Mb shapefile (state and 
> county roads). It
> > was slow as a dog. Then I indexed it and shptree created a 
> 38 Mb index
> file.
> > It is now blazingly fast.
> 
> > Try reindexing with different parameters... maybe the 
> indexing is not
> > optimal at all.
> 
> > You are right. Nothing should take "minutes" in today's web 
> age. If it
> does,
> > you have to either improve it or do it some other way.
> 
> > pk/
> 
>