[mapserver-users] Layer with over 6000 shapefiles

[Even Rouault]

Le mercredi 29 janvier 2014 19:16:40, EVANS, JAMES R GS-13 USAF ACC 84 
RADES/SCZE a écrit :
> I was off on the size of the shape file.  It's 265MB, not GB.  Doh! 
> Anyway, the performance when processing that files was still surprisingly
> bad.  Quad core server and all 4 core's were maxed out for about 5 minutes
> after just loading that layer on a client.  I am running Server 2003, and
> only have 4GB ram on that server.  We are looking at replacing it. James

Did you create a .qix spatial index ?

> 
> 
> -----Original Message-----
> From: mapserver-users-bounces at lists.osgeo.org
> [mailto:mapserver-users-bounces at lists.osgeo.org] On Behalf Of Even Rouault
> Sent: Wednesday, January 29, 2014 11:09 AM
> To: mapserver-users at lists.osgeo.org; Brent Wood
> Subject: Re: [mapserver-users] Layer with over 6000 shapefiles
> 
> Le mercredi 29 janvier 2014 18:40:49, Brent Wood a écrit :
> > This is less a mapserver issue than a data management issue - and
> > shapefiles are not a particularly good way to manage data.
> > 
> > I suggest you look at a spatially enabled database, with attribute &
> > spatial indexes, such as Postgis, instead of shapefiles as a data source.
> > These are pretty much designed for such use cases.
> > 
> > 
> > That said, if you cannot use a spatial database in this case, you can
> > do things with mapserver to improve things - like have multiple layers
> > (though 6000 seems a bit excessive) in a single group - so addressable
> > as a single layer but each with its own shapefile spatial index file.
> 
> A spatialite DB with spatial index could also perhaps do it. Not sure if it
> scales well up to 260 GB however.
> 
> > Brent Wood
> > 
> > 
> > 
> > ________________________________
> > 
> >  From: "EVANS, JAMES R GS-13 USAF ACC 84 RADES/SCZE"
> > 
> > <james.evans.4 at us.af.mil> To: "mapserver-users at lists.osgeo.org"
> > <mapserver-users at lists.osgeo.org> Sent: Thursday, January 30, 2014
> > 5:32 AM
> > Subject: [mapserver-users] Layer with over 6000 shapefiles
> > 
> > 
> > Hi,
> > Is there a way to add a layer that points to 6000 some odd shapefiles?
> > I've created a set of contour files for the whole world, and when I
> > had the info in a single shapefile it was about 260GBs.  Mapserver
> > would serve it up, but performance was horrible.  I now have the set
> > broken up into about 6000 separate shapefiles, but don't know how to
> > address that in a single layer.  I tried using gdaltindex, to create a
> > single index file that points to all the other shapefiles, but I get
> > an error saying the shapefiles are an unsupported type.  Is there some
> > other way to create an index that Mapserver can use to point to this
> > many shapefiles? Thanks, James
> > 
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> 
> --
> Geospatial professional services
> http://even.rouault.free.fr/services.html
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users

-- 
Geospatial professional services
http://even.rouault.free.fr/services.html