Martin,<br><br>These changes look quite promising. Please add two tickets at <a href="http://trac.osgeo.org/gdal/newticket">http://trac.osgeo.org/gdal/newticket</a> , one for each topic, and attach the patch.<br><br><div class="gmail_quote">
On Mon, Jul 19, 2010 at 5:38 PM, Martin Dobias <span dir="ltr"><<a href="http://wonder.sk">wonder.sk</a>@<a href="http://gmail.com">gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Hi,<br>
<br>
in order to speed up rendering in QGIS as a part of my GSoC project,<br>
I've took some time to profile reading of shapefiles in OGR. From the<br>
results I'd like to suggest some changes that significantly contribute<br>
to the speed of data retrieval. On a test shapefile of a road network<br>
(about 100 thousand polylines), I have seen 3-4 times faster retrieval<br>
when I've implemented the following changes:<br>
<br>
1. allow users of OGR library set which fields they really need. Most<br>
of time is wasted by fetching all the attributes, but typically none<br>
or just one attribute is necessary when rendering. For that, I've<br>
added the following call:<br>
OGRLayer::SetDesiredFields(int numFields, int* fields);<br>
The user passes an array of ints, each item tells whether the field<br>
should be fetched (1) or not (0). The numFields tells the size of the<br>
array. If numFields < 0 then the layer will return all fields (default<br>
behavior). The driver implementation then just before fetching a field<br>
checks whether to fetch the field or not. This optimization could be<br>
easily used in any driver, I've implemented it only for shapefiles.<br>
The speedup will vary depending on the size of the attribute table and<br>
number of desired fields. On my test shapefile containing 16 fields,<br>
the data has been fetched up to 3x faster when no fields were set as<br>
desired.<br>
<br>
2. reuse allocated memory. When a new shape is going to be read within<br>
shapelib, new OGRShape object and its coordinate arrays are allocated.<br>
By reusing one such temporary OGRShape object within a layer together<br>
with the coordinate arrays (only allowing them to grow - to<br>
accommodate larger shapes), I have obtained further speedup of about<br>
30%.<br>
<br>
I'm attaching patch for both cases. I'd like to hear from you if there<br>
is interest in making the OGR library faster using the suggested<br>
strategies. I don't expect the patch to be applied as-is since it is<br>
kind of a quick hack, though I hope it can serve well for a<br>
discussion.<br>
<br>
If there is interest, I think of some further optimizations by reusing<br>
OGRFeature instances and possibly also geometries - I expect further<br>
performance improvement of 10-20% in read access to features for all<br>
drivers.<br>
<br>
Regards<br>
<font color="#888888">Martin<br>
</font><br>_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">http://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br></blockquote></div><br><br clear="all"><br>-- <br>Best regards,<br>Chaitanya kumar CH.<br>
/tʃaɪθənjə/ /kʊmɑr/ <br>+91-9494447584<br>17.2416N 80.1426E<br>