shptree & .qix files

[Antti Roppola]

Hi Ben,

If your map extent takes in the whole shapefile, the index will return all
the  geometries in the shapefile. In such a case, the index would in fact
be an overhead as you end up reading every polygon from map.shp anyway.

You can test this by zooming into a small portion of the layer. If the
index is working, the performance increase should be noticeable as the
index will tell Mapserver to skip reading sections of the shapefile that
are not in the viewed extent.

This is all of course quite separate from any database attribute joins
being done. If you have a complex query against a database that then does
an atrribute based filtering, that'll add time as well. If your geometries
and attribute query were in the same database, the database query planner
would be abe to pick a balanced strategy for you (or at the least tell you
which part of the query was causing the problem).

Antti

-----Original Message-----
From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] On Behalf Of Ben Madin
Sent: Tuesday, 23 May 2006 11:26 AM
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: [UMN_MAPSERVER-USERS] shptree & .qix files

G'day all,

I have a very large shapefile, which with various queries to produce a multilayered map depending on data in a MSSQL database is taking
145 seconds to render a map (using php/mapscript). A simpler (but still 50MB) shapefile is producing a map in about 2 secs. The time is all in the $map->draw() statement, so looking around for ways to speed it up, i realised that it wasn't indexed.

After running shptree on it, and getting a .qix file, it was taking just as long... aha you say, and yes, you were right, I was still calling "map.shp"!

So I changed that to just "map", and now it takes 194 seconds?

Needless to say, I changed it back to .shp, back to 145 seconds.

My understanding (from the mapserver site) is that shapefiles are the default format for mapserver, and that creating an index should speed them up, not slow them down. have I misinterpreted this, or am I way over my head. The queries in MSSQL are taking negligible time, the php script is taking negligible time, this is over a wireless LAN, so it must be the mapserver engine.

I appreciated Stuart's suggestions for speed, but we have already given up reprojection, and none of these files have any projection associated with them as far as I can tell... how can I tell?

cheers

Ben



-- 

Ben Madin
Remote Information Pty Ltd

t/f: 08 9192 5535
m: 0448 887 220
ben at remoteinformation.com.au

---------------------------------------------------------------------- 
IMPORTANT - This message has been issued by The Department of Agriculture, Fisheries and Forestry (DAFF).  The information transmitted is for the use of the intended recipient only and may contain confidential and/or legally privileged material.  It is your responsibility to check any attachments for viruses and defects before opening or sending them on.  
Any reproduction, publication, communication, re-transmission, disclosure, dissemination or other use of the information contained in this e-mail by persons or entities other than the intended recipient is prohibited.  The taking of any action in reliance upon this information by persons or entities other than the intended recipient is prohibited.  If you have received this e-mail in error please notify the sender and delete all copies of this transmission together with any attachments.  If you have received this e-mail as part of a valid mailing list and no longer want to receive a message such as this one advise the sender by return e-mail accordingly.  Only e-mail correspondence which includes this footer, has been authorised by DAFF 
----------------------------------------------------------------------