[gdal-dev] OGR slow with network files on Windows

[Even Rouault]

> Do you prefer VSI cache for buffering to implementation of another
> buffering mechanism as long term solution?

>From my understanding of the code (FrankW is the author of VSI Cache), VSI cache
is currently both a cache and buffering mechanism. There's perhaps a small
penalty of having the caching part when you only need buffering, because, per
file, there's an array of pointers whose size is file_size / CHUNK_SIZE When a
chunk is cached, it points to a structure that holds CHUNK_SIZE bytes. There's
also a linked list of cached chunks to implement a Least Recently Used strategy
and evict old chunks when the cache size is reached.

Personnaly I'd avoid introducing another VSI mechanism for the sake of
maintanibility.

> > I have not a particular informed opinion on this. Did you try experimenting
> > increasing CHUNK_SIZE to see if this improves performance ?
>
> No, CHUNK_SIZE cannot be set as environment variable and I am using
> precompiled osgeo4w. Maybe it is not that important. However, it could
> be useful to change it be read from environment variable so that it
> would be easy to experiment or to force higher value if necessary.

Time to compile your own GDAL then ;-)

I'm also wondering if setting unconditionnaly VSI_CACHE might not have some
theoretical penalty for fast access storage when there is a lot of seeking and
reading by small amounts (I believe this is the typical I/O pattern when you
display a JPEG2000). It might be relevant to have an option to enable
caching/buffering for files only located on SMB folders.

Even