[gdal-dev] Map algebra

[Ari Jolma]

02.04.2016, 16:08, Even Rouault kirjoitti:
>>> - Are the methods handling 2 bands able to deal with different block
>>> sizes ?
>> Yes, the band size needs to be the same
> Not sure to understand your answer: my question was rather about block sizes.


I forgot a "however": Yes, the block sizes can differ as they easily do 
even with same driver and same band size but different datatype. 
Currently the band sizes have to be the same.


>
>>
>>> - And with bands of different data types ?
>> Yes. It creates pretty huge switch tables however. They are mostly
>> hidden in macros but are a problem. I don't see how I can get the
>> datatype into the templates from the bands and I don't like the idea of
>> leaving that to the users.
> Perhaps we don't need to instanciate all template possibilities: just the
> <same_type,same_type> ones, and in case of different types promote to double ?


Promoting may be costly if the raster is huge. I believe it is quite 
common to have to operate on both integer rasters and floating point 
rasters at the same time. Determining which templates are needed is not 
easy beforehand.


>
>>> - Why the need to reinvent a hashset mechanism ? Isn't std::set good
>>> enough ? - Same for gma_array vs std::vector ?
>> Simply ignorance of C++ I think. I do see that for example C++11 and
>> boost have lots of useful things but did not (yet) go that way. Earlier
>> C++ standards seem to have many tools too. I learnt C++ >20 years ago...
>> I believe I also need some small classes like "mapper", which
>> reclassifies data from old values to new values and are not readily
>> available.
> std::set, std::map, std::vector are available in C++98 and already used in
> GDAL. We should use them when practical (admitedly I developed a CPLHashSet a
> long time ago, but in retrospect that didn't make sense)

Yes, I agree that C++98 tools should be used.


> Isn't there some duplication with GDAL own's block cache ? Perhaps I'm just
> confusion by the proximity of the naming. I didn't study your code closely.

I'll look into that.

Ari


>
>>> - there is some overlap/connection with the pixel functions of VRT ( see
>>> "Using Derived Bands" of http://gdal.org/gdal_vrttut.html).
>> ok
>>
>> Thanks for the comments!
>>
>> Ari
>>
>>>> The C code is of course not this nice but it is not very far from this.
>>>> All the basic four method types I mention below are implemented and
>>>> adding new methods is not very difficult. The code makes heavy use of
>>>> templates and macros in an attempt for versatility (GDAL rasters can
>>>> have many datatypes) and easiness to write methods in a few lines of
>>>> code. The whole set of code is now ~2500 lines but it creates rather
>>>> large executables due to templates. The code is C++ but the API is C
>>>> like, I've so far done no changes to existing GDAL code because of this.
>>>>
>>>> As written below, the code works on two levels of x,y loops: the blocks
>>>> within a band, which is generic and cells within a block, which is
>>>> separate for each method. The methods work on line by line (no recursion
>>>> etc.) and thus for example the fill_depressions is not optimal but
>>>> instead very simple. Also, very large rasters can be processed.
>>>>
>>>> I think this may be somewhat similar thing in relation to GDAL as the
>>>> network support, so it could be a thing to add to the library if wished.
>>>>
>>>> I've added RFC 62 "raster algebra" to spark discussion.
>>>>
>>>> Ari
>>>>
>>>> 20.03.2016, 10:21, Ari Jolma kirjoitti:
>>>>> Folks,
>>>>>
>>>>> I played around a bit with the map algebra idea and came up with
>>>>> something that could work nicely as a plugin. The initial code is at
>>>>> https://github.com/ajolma/gdal/tree/trunk/gdal/map_algebra
>>>>>
>>>>> The idea is based on processing raster bands block by block and
>>>>> calling a given function on each block. Using macros and templates the
>>>>> code should be rather nice and manageable even when supporting
>>>>> multiple data types for raster cells.
>>>>>
>>>>> Further, the idea is to support three kinds of map algebra methods,
>>>>> mostly working on a raster band in-place.
>>>>>
>>>>> 1) Simple ones, which do not take any arguments beyond the raster band
>>>>> itself, examples are functions like log, sin, rand.
>>>>>
>>>>> 2) Those which take one non-spatial argument. The argument can be a
>>>>> scalar like a number, or more complex like a reclassifier.
>>>>>
>>>>> 3) Those which operate on two rasters. Examples are summation of
>>>>> rasters and computing flow directions on a DEM. The latter is a bit
>>>>> more complex since it is a focal method and requires a 3 x 3 matrix of
>>>>> blocks from the other operand raster.
>>>>>
>>>>> Maybe a fourth kind of methods are those which compute something from
>>>>> a raster.
>>>>>
>>>>> This would lead to four functions in the C API and raster band methods
>>>>> in the bindings.
>>>>>
>>>>> Comments are welcome, the initial code base contains a small demo
>>>>> program. Eventually, if this works, I'll make a RFC from this.
>>>>>
>>>>> Ari
>>>> _______________________________________________
>>>> gdal-dev mailing list
>>>> gdal-dev at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev