Attributes in tile indexes?

[Ed McNierney]

Jeff -

In addition to Steve's suggestion, please keep in mind that what you are
trying to do is reduce the amount of data MapServer needs to read in order
to retrieve the data you need.  Think about your data carefully and see
whether you're achieving that goal.

First, splitting the shapefile into multiple files and indexing them with a
TILEINDEX helps MapServer select a subset of that data to be used for a
given map draw.  If MapServer has to open most or all of those shapefiles
anyway, then the TILEINDEX isn't going to do a thing!

Since your method of producing shapefile tiles will minimize overlap between
files, you should also check the relationship of your map's spatial extent
with those tiles.  Ideally, each tile should be (roughly) somewhat larger
than the extent of the map view.  With rectangularly-tiled data, you cannot
avoid situations where the map view is centered on a corner and you need to
open four different input shapefiles.  If each rectangular tile is, however,
slightly larger than the map image size, then you will never get a situation
that is WORSE than four input files.  If you render a zoomed-out view
covering dozens of input files, your TILEINDEX isn't going to help much.

Think about how your TILEINDEX and split shapefiles are behaving.  Have you
substantially eliminated most of your input data from consideration in a
map-drawing request?  If not, then you need to go back and think about how
to restructure your tiling and indexing, since that's the whole goal of
those techniques.

     - Ed

-- 
Ed McNierney
President and Chief Mapmaker
Maps a la carte, Inc. / TopoZone.com
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
Phone: (978) 251-4242
Fax: (978) 251-1396
ed at topozone.com

> From: Stephen Woodbridge <woodbri at SWOODBRIDGE.COM>
> Reply-To: Stephen Woodbridge <woodbri at SWOODBRIDGE.COM>
> Date: Mon, 23 Apr 2007 11:53:56 -0400
> To: <MAPSERVER-USERS at LISTS.UMN.EDU>
> Subject: Re: [UMN_MAPSERVER-USERS] Attributes in tile indexes?
> 
> Jeff,
> 
> Make SURE all your tiles have a spatial index *.qix and the tileindex
> also has a spatial index. These can be built with
> 
>    shptree file.shp
> 
> -Steve W.
> 
> Jeff Dege wrote:
>> OK - I knew I had to be doming something stupid.
>> 
>> To tell a layer to draw using a tileindex shapefile, you include the
>> commands:
>> 
>>    TILEINDEX "tileindex"
>>    TILEITEM "LOCATION"
>> 
>> Using:
>> 
>>    DATA "tileindex"
>> 
>> won't work.
>> 
>> Now, after all of that, drawing with the split shapefiles is slower than
>> with the original.  So there's still some playing to do.
>>  
>> 
>>> -----Original Message-----
>>> From: UMN MapServer Users List
>>> [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of Jeff Dege
>>> Sent: Monday, April 23, 2007 9:53 AM
>>> To: MAPSERVER-USERS at LISTS.UMN.EDU
>>> Subject: Re: [UMN_MAPSERVER-USERS] Attributes in tile indexes?
>>> 
>>> Well, I've been looking at this in more detail, and finding that the
>>> tileindex layer isn't drawing at all.  With all other layers
>>> turned off,
>>> when I use shp2img with an extent that causes the tile layer
>>> to draw, I
>>> get a blank image.  When I change the layer to use the original
>>> shapefile, I get the data displayed.  (And the layer takes 40
>>> seconds to
>>> draw, even after I've run shptree on the shapefile, which is why I've
>>> been playing around with tiling.)
>>> 
>>> I identified the individual tile shapefile that contains the area I'm
>>> trying to draw and used it directly, and it works fine.  So the
>>> splitting apart seems to be working.  But the tileindex
>>> shapefile seems
>>> to be broken.
>>> 
>>>> -----Original Message-----
>>>> From: UMN MapServer Users List
>>>> [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of Stephen
>>> Woodbridge
>>>> Sent: Saturday, April 21, 2007 7:15 AM
>>>> To: MAPSERVER-USERS at LISTS.UMN.EDU
>>>> Subject: Re: [UMN_MAPSERVER-USERS] Attributes in tile indexes?
>>>> 
>>>> Jeff,
>>>> 
>>>> This should just work. There is nothing special you should
>>>> have to do. I 
>>>> do it all the time with my mapfiles. I use shp2tile to break
>>>> mine up but 
>>>> that should not make a difference.
>>>> 
>>>> First I would try changing your layer to use just one of
>>> the smaller 
>>>> shape files without the tileindex and see if that works and
>>>> that you can 
>>>> display the labels.
>>>> 
>>>> You might also want to post info about what OS, version of
>>> mapserver, 
>>>> and the LAYER block in question.
>>>> 
>>>> -Steve W
>>>> 
>>>> Jeff Dege wrote:
>>>>> I have a shape file that's too big to draw maps with -
>>> takes far too
>>>>> long.
>>>>> 
>>>>> In an attempt to speed things up, I split it into 16
>>>> smaller shapefiles,
>>>>> using:
>>>>> 
>>>>>    ogr2ogr -f "ESRI Shapefile" -spat xmin ymin xmax ymax newXX.shp
>>>>> old.shp
>>>>> 
>>>>> Then did a shptree on each of these new files, then a
>>> shptinidex to
>>>>> create a single tile index shapefile on them, and tried to
>>>> build a map.
>>>>> It works fine, so long as I don't try to display labels.
>>>> But if I do, I
>>>>> get errors:
>>>>> 
>>>>>    msDBFGetItemIndex(): Item 'NAME' not found.
>>>>> 
>>>>> Where "NAME" is the attribute I'm trying to use for the
>>>> label.  It looks
>>>>> like the tileindex shapefile doesn't expose the attributes of the
>>>>> underlying shape tiles.  I've done an ogrinfo on the
>>>> shapefiles that I'd
>>>>> generated, and the attributes are there.
>>>>> 
>>>>> Can I expose the attributes through the tileindex?
>>>>> 
>>>>> How?