shptree not faster than no index
ed at TOPOZONE.COM
Fri Sep 24 12:26:11 EDT 2004
No, that is a different situation. Your original scenario described 3,000 files, each of which had one polygon; having one file per object is clearly not a good idea. Having one file per 3,000 - 30,000 objects is just fine. It is also good to think about updating your data set; if you're going to need to do updates, you might want to minimize the amount of data conversion you need to do each time you update the data set.
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA 01863
ed at topozone.com
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of John Bolster
Sent: Friday, September 24, 2004 8:42 AM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no index
Thanks for the advice. I'll try it this way on the counties. However, when I start working with the other files, like roads, water and places, I'm dealing with files that are currently broken down by county and each one may have anywhere from 3,000 - 30,000 items per file. So that multiplied by 3,000 or so counties comes to 9 - 90 million items if I combined each record type into one big file each. Would you still suggest combining them in these cases, or do you think that gets too large?
> -----Original Message-----
> From: ANDY CANFIELD [mailto:andy_canfield at hotmail.com]
> Sent: Thursday, September 23, 2004 6:51 PM
> To: jb at NUCOMP.COM; MAPSERVER-USERS at LISTS.UMN.EDU
> Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no index
> I'd go with the one file of all counties. It's just easier to maintain
> one file than 3,000. I have at least 50 layers in one of my maps that
> each layer can have 300,000 to 1,000,000 plus objects in it. Using the
> shptree to create a .qix file makes these layers incredibly fast. I
> mean fast as in, you click on the map to zoom or pan, and as fast as
> your browser can render the map has been updated to reflect that pan
> or zoom. I think my smallest layer has 1,400 hundred polygons in it.
> Using the shptree and having one file is way easier than maintaining a
> single file for each boundary. I honestly don't think the map I'm
> using could be any faster, it is the browser/desktop rendering speed
> for me that limits how fast the image changes, so that's really,
> really fast. Plus if you have 3,000 files and are zoomed to an extent
> where you can see half of them or more that's 1,500 or more open and
> read operations that Mapserver has to execute vice one if you put them
> all in the same file. My two cents anyway.
> >From: John Bolster <jb at NUCOMP.COM>
> >Reply-To: John Bolster <jb at NUCOMP.COM>
> >To: MAPSERVER-USERS at LISTS.UMN.EDU
> >Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no index
> >Date: Thu, 23 Sep 2004 18:18:46 -0400
> >Thank you for your quick response.
> >In terms of the county outlines, do you think that one county
> per file, and
> >selecting which files to use with only a tileindex would be
> faster than all
> >counties being in one huge file of over 3000 counties and selecting
> >which counties to draw using a qix index?
> >Also, does the qix index on the actual tileindex do anything to speed
> >up the tileindex?
> > > -----Original Message-----
> > > From: Frank Warmerdam [mailto:warmerdam at pobox.com]
> > > Sent: Thursday, September 23, 2004 6:13 PM
> > > To: John Bolster
> > > Cc: MAPSERVER-USERS at LISTS.UMN.EDU
> > > Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no
> > > index
> > >
> > >
> > > John Bolster wrote:
> > > > Hi all,
> > > >
> > > > I'm noticing there doesn't seem to be any difference in speed
> > > between having
> > > > *.qix files vs. not having them. I'm wondering if I'm doing this
> > > >
> > > > My data is from tiger and is in shapefiles. I'm only dealing
> > > > with
> > > > outlines right now. They are each in a separate folder, each
> > > > state's counties are in one state folder. There's a tileindex on
> them all at
> > > > root of the tree.
> > >
> > > John,
> > >
> > > The impression I get is that you have exactly one county polygon
> > > per shapefile.
> > > Is that right? A .qix won't help because it is used to more
> > > quickly select the subset of shapes in a shapefile based on the
> > > extents. If you only have one shape per file then it can't help.
> > >
> > > Of course, the tileindex should allow you to quickly select the
> > > county files you want. But a more efficient approach would likely
> be to have
> > > shapefile with a .qix with all your county outlines in it.
> > >
> > > Best regards,
> > > --
> > > ---------------------------------------+--------------------------
> > > ------------
> > > I set the clouds in motion - turn up | Frank Warmerdam,
> > > warmerdam at pobox.com
> > > light and sound - activate the windows | http://pobox.com/~warmerdam
> > > and watch the world go round - Rush | Geospatial
> Programmer for Rent
> > >
> > >
> FREE pop-up blocking with the new MSN Toolbar – get it now!
More information about the mapserver-users