[gdal-dev] [Qgis-developer] QGIS Multi-threaded Rendering

[Even Rouault]

Hi Martin,

(CC'ing gdal-dev : original email is
http://lists.osgeo.org/pipermail/qgis-developer/2013-December/029716.html)

just commenting on the interesting raster topics.

> Finally, some further thoughts/questions:
>
> - rasters - currently we do not have API to cancel requests for raster
> blocks. This means that currently we need to wait until the raster
> block is fully read even when we cancel the rendering job. GDAL has
> some support for asynchronous requests - anyone has some experience
> with it?

Indeed there's an API in GDAL for asynchronous requests :
http://trac.osgeo.org/gdal/wiki/rfc24_progressive_data_support
Note that it has a "real" implementation only in the JPIPKAK driver
(driver that implement the JPIP protocol - JPEG2000 trough the network -
with the Kakadu library). For other drivers that don't have a dedicated
implementation, a default implementation will just turn the async requests
as a sync request (more precisely GetNextUpdatedRegion() will just do
a RasterIO(), other methods mentionned in the RFC are no-op).
Real cancellation / async request is generally hard to implement since the
low level libraries used by drivers must support it. The analysis must be
done on a per-driver basis (determine which drivers are important for you,
which ones are slow, see if it can be remedied in the general synchronous
case, otherwise imagine how async request could be implemented, ...)
We could imagine that the default GetNextUpdatedRegion(), instead of
issuing a single RasterIO() could split it up into several RasterIO()
if the request window crosses several tiles (in the case of a tiled
dataset) to allow cancellation between the calls to GetNextUpdatedRegion().
But that's no always a beneficial strategy. Drivers can use the fact that
a RasterIO() is done on a "big" window to optimize things a bit, w.r.t
issuing several RasterIO() on smaller windows. In GDAL 1.10, I've for example
implemented multi-threaded block decoding in the OpenJPEG driver : if the
window of a RasterIO() call spans over several tiles and you have several
cores, then each tile will be decoded in parallel.
Regarding the current API of asynchronous requests, I feel there's perhaps
a lack of a way of signaling that data is ready. GetNextUpdatedRegion()
works on a pulling startegy. Perhaps a callback mechanism could be usefull.
That topic needs some thinking...

>
> - rasters (again) - there are no intermediate updates of the raster
> layer when rendering. What that means is that until the raster layer
> is fully rendered, the preview is completely blank. There is a way to
> constrain the raster block requests to smaller tiles, but what would
> be the performance consequences? I am not that familiar with the way
> how raster drivers are implemented in GDAL... anyone to bring some
> wisdom?

You can issue a GDALRasterIO() call with a buffer size that is smaller than
the source window. If the raster has overviews (.ovr file, computed internal
overviews, or "implicit" overviews in formats like JPEG2000 or ECW), then they
will be used. (Of course, you can also directly issue GDALRasterIO() on a
overview band, but that will be equivalent performance-wise).
Otherwise, full resolution data will be fetched and then sub-sampled, which can
take some time. No magic solution here.

Even

-- 
Geospatial professional services
http://even.rouault.free.fr/services.html