[fdo-users] FDO SHP 3.3 and Filters

[Benoit Begin]

So I have noticed that the SHP Provider has a very, very slow response time
whenever you apply alphanumeric filters ( NAME='Benoit' ), yet when you
apply spatial filters, it has very quick response times. I'm guessing this
has to do with the fact that the SHP Provider builds an R-Tree index file
for the geometries, which allows it to process these filters much more
quickly.

As a test, I did a query on 400,000 records in my SHP file. With a filter on
a string field that returns 540 records, it takes 22 seconds to iterate
through these records (IFeatureReader.ReadNext()), without doing any
processing within those iteration, it's just the time it takes for the loop
to run through. If I apply no filters at all and within the loop, I add in
an IF statement that checks the value of the field I want to filter, it
takes 21 seconds. Without a filter at all or any statements within the loop,
it takes 3 seconds to iterate through all the records.

It seems to me as if adding some form of indexing to the DBF portion of the
SHP file, just as is done with the geometries, would be greatly beneficial
and would bring a lot of strength to this provider. While I realize it might
not be the best format to run such large queries, a lot of people use these
types of files and do not wish to convert them to a sturdier, quicker
database format. A good example would be TeleAtlas and NAVTEQ data, that is
usually provided in a SHP format.

I looked around and it looks as if the xBase format already has
specifications for indexing. Did I miss something and is this already
supported by the provider? Is there something special that has to be done
for the provider to leverage / create indexes for the data?

Thanks!
-- 
View this message in context: http://n2.nabble.com/FDO-SHP-3.3-and-Filters-tp2134208p2134208.html
Sent from the FDO Users mailing list archive at Nabble.com.