[gdal-dev] New driver implementation with larger block size

[Ari Jolma]

07.06.2016, 10:54, Jukka Rahkonen kirjoitti:
> Javier Calzado <francisco.javier.calzado <at> ericsson.com> writes:
>
>> Thanks for your answer Even,
>>
>> I knew about GDAL_ONE_BIG_READ, but given that our tool will access
>> several raster simultaneously and will perform intensive operations with
>> them, I like the idea of allowing GDAL caching blocks, as long as cache
>> hits rate is high, this will increase performance I guess. GDAL cache is
>> pretty smart and well done, this way we avoid re-implementing our own
>> custom cache system. Besides that, we will do intensive access to raster
>> on a pixel basis, and it seems that performance, according to some tests I
>> did, is highly improved using direct block pointers (GetLockedBlockRef)
>> over RasterIO pixel access. So, unless I am missing something with
>> RasterIO, I think we will go for the cached block access method.
>>
>> So, I guess the only way to accomplish larger block sizes in my new driver
>> is to implement a custom rasterband derived from GDALRasterBand and
>> implement IReadBlock/IWriteBlock methods, right?
>>
>> My concern in that case is ... what would happen with blocks that cannot
>> be completely filled with raster data? I mean, given a 150x150px raster
>> for example, if a define block size as 150x100, the second block will have
>> the last 50 scanlines empty, does GDAL internally handle this situation
>> properly?
> Hi,
>
> Unfortunately I do not really understand what you are talking about but when
> I saw words "blocks" and "caching" I started to think that the same words
> appear in Ari Jolma's raster algebra
> https://trac.osgeo.org/gdal/wiki/rfc62_raster_algebra
>
> Could there be some overlaps?

Yes. But mainly for me to learn :)

I actually did not know / realize about the GetLockedBlockRef / 
GDALRasterBlock class and it seems that it will speed up the raster 
algebra methods since I've been copying the blocks to an intermediary cache.

Also, Javier is concerned about how to make the GDAL raster block cache 
system work for one driver and I'm concerned how to make the cache 
system work for application code - we're so to speak on opposite sides 
of the cache system.

To use this space for some comments on the raster algebra code. I've 
done some research and profiling and I'm going to test using non-native 
blocks as an option. That's because I realized that when more than one 
band is involved in a raster algebra method at the same time it is very 
beneficial to have the same block size in all bands. The QGIS raster 
calculator uses that approach.

Ari

>
> -Jukka Rahkonen-
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev