[gdal-dev] Fast Pixel Access

[Jukka Rahkonen]

Luke Roth <roth.luke <at> gmail.com> writes:

> 
> Another thing that might speed up access is setting the config
option GDAL_DISABLE_READDIR_ON_OPEN = TRUE, either as an environment
variable or on the command line.  That should help with GDAL reading the
directory each time it opens a dataset.  I have an application which reads
one value from each of a large number of datasets and setting this option
made it run about 3 times faster.


Hi,

You are right. This config option makes GDAL to skip the reading of the
remote directory and saves a lot of bandwidth:

VRT case: 
Bytes Received:  4 244 509 (of which the vrt file: 4 192 577)
Sequence (clock) duration:	00:00:09.9996000
Was:
Bytes Received:  6 459 443
Sequence (clock) duration:	00:00:37.8130000

BigTIFF case: 
Bytes Received:  2 158 917
Sequence (clock) duration:	00:00:04.4368000
Was:
Bytes Received:  4 374 137
Sequence (clock) duration:	00:00:30.9192000


Conclusion:
Both options are unsuitable for serious use while amusing to play with.
Reading the BigTIFF tile offset index (or whatever it is) seems to mean
about 2 MB of compultory payload traffic. Reading the VRT file means in this
example 4 MB of payload. If this sort of net access to a large directory of
raster files should be important for someone there should be a way to find
the right raster file and righ data range in that file with minimum amount
of bytes. Perhaps some kind of rtree indexed vrt file? First aid might be to
keep the vrt file on the client side.

-Jukka Rahkonen-