[gdal-dev] [Qgis-developer] QGIS Multi-threaded Rendering

[Dmitriy Baryshnikov]

Hi,

Why not reading small chunks (i.e. 256 x 256) with RasterIO 
<http://www.gdal.org/classGDALDataset.html#ae077c53268d2272eebed10b891a05743> 
from big raster? We'll get less memory usage.
The program can count the percentage using the size of GDALDataset and 
size of chunks and cancel on any chunk.

By the way, can GDALDataset RasterIO different chunks from separate threads?

Best regards,
     Dmitry

12.12.2013 19:28, Even Rouault ?????:
> Selon Martin Dobias <wonder.sk at gmail.com>:
>
>> Hi Even
>>
>> thanks for your thoughts.
>>
>> On Thu, Dec 12, 2013 at 8:30 PM, Even Rouault
>> <even.rouault at mines-paris.org> wrote:
>>>> - rasters - currently we do not have API to cancel requests for raster
>>>> blocks. This means that currently we need to wait until the raster
>>>> block is fully read even when we cancel the rendering job. GDAL has
>>>> some support for asynchronous requests - anyone has some experience
>>>> with it?
>>> Indeed there's an API in GDAL for asynchronous requests :
>>> http://trac.osgeo.org/gdal/wiki/rfc24_progressive_data_support
>>> Note that it has a "real" implementation only in the JPIPKAK driver
>>> (driver that implement the JPIP protocol - JPEG2000 trough the network -
>>> with the Kakadu library). For other drivers that don't have a dedicated
>>> implementation, a default implementation will just turn the async requests
>>> as a sync request (more precisely GetNextUpdatedRegion() will just do
>>> a RasterIO(), other methods mentionned in the RFC are no-op).
>>> Real cancellation / async request is generally hard to implement since the
>>> low level libraries used by drivers must support it. The analysis must be
>>> done on a per-driver basis (determine which drivers are important for you,
>>> which ones are slow, see if it can be remedied in the general synchronous
>>> case, otherwise imagine how async request could be implemented, ...)
>>> We could imagine that the default GetNextUpdatedRegion(), instead of
>>> issuing a single RasterIO() could split it up into several RasterIO()
>>> if the request window crosses several tiles (in the case of a tiled
>>> dataset) to allow cancellation between the calls to GetNextUpdatedRegion().
>>> But that's no always a beneficial strategy. Drivers can use the fact that
>>> a RasterIO() is done on a "big" window to optimize things a bit, w.r.t
>>> issuing several RasterIO() on smaller windows. In GDAL 1.10, I've for
>> example
>>> implemented multi-threaded block decoding in the OpenJPEG driver : if the
>>> window of a RasterIO() call spans over several tiles and you have several
>>> cores, then each tile will be decoded in parallel.
>>> Regarding the current API of asynchronous requests, I feel there's perhaps
>>> a lack of a way of signaling that data is ready. GetNextUpdatedRegion()
>>> works on a pulling startegy. Perhaps a callback mechanism could be usefull.
>>> That topic needs some thinking...
>> So obviously adding async API support for other common formats would
>> take a lot of time... I was secretly hoping that the remark in rfc24
>> mentioning only jpipkak driver was just a temporary state of affairs
>> :-)
>>
>> However I realize that actually I do not really need to have truly
>> asynchronous requests. The requests from QGIS are being made from
>> within a worker thread, so it is not a problem if GDAL block that
>> thread. The only thing I need is to be able to tell GDAL to stop
>> reading data as soon as possible once the rendering gets cancelled.
>> API-wise, I would like to have a method in GDALDataset, using which I
>> could register my custom function returning cancellation status of the
>> request (just true/false). The drivers would be responsible for
>> calling that function whenever appropriate during RasterIO to check
>> whether the request has been cancelled or not. It would be probably
>> much less hassle to implement this functionality in drivers - although
>> I understand that still many drivers using custom libraries won't be
>> able to provide that functionality.
> Yes, that's a reasonable possibility. The callback could also take
> as parameter a percentage of completion. That could actually be a
> GDALProgressFunc (
> http://trac.osgeo.org/gdal/browser/trunk/gdal/port/cpl_progress.h )
>
>
>> Regards
>> Martin
>>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20131213/adad0afd/attachment.html>