"best" data format for attribute searches

[James Perrins]

Frank,

Would you mind elaborating on shape files supporting indexed attribute
queries in OGR ?

I was under the impression from all that I had read that Mapserver
didn't use dbf indexes for shape files - (I know it can use spatial
indexes) - or is the difference here that it does if you go via OGR - or
am I missing the point !

I would be very grateful if you could point me at an example /
documentation of how to use indexed attribute queries in mapserver
without an RDBMS.

What should you use to create the dbf indexes ?
Will indexes still work if accessed via an indexed layer created using
tile4ms.exe ?

It would be great if I can use indexed attribute queries as I'm starting
to hit performance issues with large data volumes and am trying for a
variety of reasons to delay the move to something like PostGIS

Many Thanks
James

-----Original Message-----
From: Frank Warmerdam [mailto:fwarmerdam at GMAIL.COM] 
Sent: 18 July 2005 15:15
To: MAPSERVER-USERS at LISTS.UMN.EDU
Subject: Re: [UMN_MAPSERVER-USERS] "best" data format for attribute
searches

On 7/18/05, Bart van den Eijnden <BEN at syncera-itsolutions.nl> wrote:
> Hi list,
> 
> AFAIK Mapserver does not use the Mapinfo attribute indexes.
> 
> I am trying to perform attribute searches on a 25 Mb Mapinfo file and
am worried about performance, since the indexes won't be used which will
mean scanning the whole data file for every attribute search.
> 
> What would be a wise format to shift to for this type of operation? Or
doesn't it make any sense to shift to eg PostGIS because of the way
Mapserver does queries to RDBMS's?
> 
> Thanks in advance for any insight on this.

Bart, 

The only non-RDBMS driver in OGR that supports indexed attribute
queries is Shapefiles, and there only exact lookups are indexable.
If you don't want the hassle of setting up a complex RDBMS like
PostGIS you might consider a lightweight equivelent like SQLite but
I do think Postgresql/PostGIS would be the ideal option.

PS. I do have a vague intention of implementing indexed attribute
query support for mapinfo tab files at some point, and the underlying
MITAB does support the index files so it shouldn't be too hard.  But
no timeline on such work. 

Best regards,

-- 
---------------------------------------+--------------------------------
------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent