shptree not faster than no index

[Ed McNierney]

John -

No, that is a different situation.  Your original scenario described 3,000 files, each of which had one polygon; having one file per object is clearly not a good idea.  Having one file per 3,000 - 30,000 objects is just fine.  It is also good to think about updating your data set; if you're going to need to do updates, you might want to minimize the amount of data conversion you need to do each time you update the data set.

        - Ed

Ed McNierney
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
ed at topozone.com
(978) 251-4242

-----Original Message-----
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of John Bolster
Sent: Friday, September 24, 2004 8:42 AM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no index

Thanks for the advice. I'll try it this way on the counties. However, when I start working with the other files, like roads, water and places, I'm dealing with files that are currently broken down by county and each one may have anywhere from 3,000 - 30,000 items per file. So that multiplied by 3,000 or so counties comes to 9 - 90 million items if I combined each record type into one big file each. Would you still suggest combining them in these cases, or do you think that gets too large?

John

> -----Original Message-----
> From: ANDY CANFIELD [mailto:andy_canfield at hotmail.com]
> Sent: Thursday, September 23, 2004 6:51 PM
> To: jb at NUCOMP.COM; MAPSERVER-USERS at LISTS.UMN.EDU
> Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no index
>
>
> I'd go with the one file of all counties. It's just easier to maintain
> one file than 3,000. I have at least 50 layers in one of my maps that
> each layer can have 300,000 to 1,000,000 plus objects in it. Using the
> shptree to create a .qix file makes these layers incredibly fast. I
> mean fast as in, you click on the map to zoom or pan, and as fast as
> your browser can render the map has been updated to reflect that pan
> or zoom. I think my smallest layer has 1,400 hundred polygons in it.
> Using the shptree and having one file is way easier than maintaining a
> single file for each boundary. I honestly don't think the map I'm
> using could be any faster, it is the browser/desktop rendering speed
> for me that limits how fast the image changes, so that's really,
> really fast. Plus if you have 3,000 files and are zoomed to an extent
> where you can see half of them or more that's 1,500 or more open and
> read operations that Mapserver has to execute vice one if you put them
> all in the same file. My two cents anyway.
>
>
> >From: John Bolster <jb at NUCOMP.COM>
> >Reply-To: John Bolster <jb at NUCOMP.COM>
> >To: MAPSERVER-USERS at LISTS.UMN.EDU
> >Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no index
> >Date: Thu, 23 Sep 2004 18:18:46 -0400
> >
> >Thank you for your quick response.
> >
> >In terms of the county outlines, do you think that one county
> per file, and
> >selecting which files to use with only a tileindex would be
> faster than all
> >counties being in one huge file of over 3000 counties and selecting
> >which counties to draw using a qix index?
> >
> >Also, does the qix index on the actual tileindex do anything to speed
> >up the tileindex?
> >
> >Thanks,
> >John
> >
> > > -----Original Message-----
> > > From: Frank Warmerdam [mailto:warmerdam at pobox.com]
> > > Sent: Thursday, September 23, 2004 6:13 PM
> > > To: John Bolster
> > > Cc: MAPSERVER-USERS at LISTS.UMN.EDU
> > > Subject: Re: [UMN_MAPSERVER-USERS] shptree not faster than no
> > > index
> > >
> > >
> > > John Bolster wrote:
> > > > Hi all,
> > > >
> > > > I'm noticing there doesn't seem to be any difference in speed
> > > between having
> > > > *.qix files vs. not having them. I'm wondering if I'm doing this
> >right.
> > > >
> > > > My data is from tiger and is in shapefiles. I'm only dealing
> > > > with
> >county
> > > > outlines right now. They are each in a separate folder, each
> > > > state's counties are in one state folder. There's a tileindex on
> them all at
> >the
> > > > root of the tree.
> > >
> > > John,
> > >
> > > The impression I get is that you have exactly one county polygon
> > > per shapefile.
> > > Is that right?  A .qix won't help because it is used to more
> > > quickly select the subset of shapes in a shapefile based on the
> > > extents.  If you only have one shape per file then it can't help.
> > >
> > > Of course, the tileindex should allow you to quickly select the
> > > county files you want.  But a more efficient approach would likely
> be to have
> >one
> > > shapefile with a .qix with all your county outlines in it.
> > >
> > > Best regards,
> > > --
> > > ---------------------------------------+--------------------------
> > > ------------
> > > I set the clouds in motion - turn up   | Frank Warmerdam,
> > > warmerdam at pobox.com
> > > light and sound - activate the windows | http://pobox.com/~warmerdam
> > > and watch the world go round - Rush    | Geospatial
> Programmer for Rent
> > >
> > >
>
> _________________________________________________________________
> FREE pop-up blocking with the new MSN Toolbar – get it now!
> http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/
>
>