<p dir="ltr">Hi Ari, <br>
I began some work to clarify my ideas here:<a href="http://www.github.com/JamesRamm/GeoAlg" target="_top" rel="nofollow" link="external"> www.github.com/JamesRamm/GeoAlg</a> perhaps there is potential for merging with your project? For neighbourhoods I provide 2 iterators - one simple block based and one 'buffered'. The block can be the natural block or user defined. I also planned to provide a 'mosaic' iterator to support the planned 'reduce' function. For callbacks I initially planned a simple function, but then realised that many algorithms may wish to save state, so I changed to a simple interface class where there is a single function to override. My final plan was to expose opencv and other functionality (eg maybe taudem) as a library of ready made callbacks. </p>
<p dir="ltr">If you think it worthwhile and possible, I would be open to porting relevant parts to your code. Features I see as absolutely essential are: <br>
- 'map' and 'reduce' functions for raster datasets, which handle large data and user defined callbacks<br>
- ability to save state in callbacks. This is essential for more complex algorithms.<br>
- A rich set of block iterators</p>
<p dir="ltr">A ready made library of a number of callbacks (preferably by exposing functionality from existing well known libraries) would also be useful and help build interest I think.</p>
<p dir="ltr">I will be in Bonn at foss4g from the Tuesday evening, it would be good to meet!</p>
<div class="gmail_extra"><br><div class="gmail_quote">On 8 Aug 2016 1:56 p.m., "Ari Jolma-2 [via OSGeo.org]" <<a href="/user/SendEmail.jtp?type=node&node=5279934&i=0" target="_top" rel="nofollow" link="external">[hidden email]</a>> wrote:<br type="attribution"><blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<p>Hi,</p>
<p>Sorry for missing this last month due to holidays.</p>
<p>In fact in my approach I do not want to limit to single pixel
operations but support neighborhood operations too. That is
necessary for watershed analysis for example. Making sure that the
neighborhood is readily available for computations is probably
what is causing the biggest problem for the code. I also have a
callback support. See for example <a href="https://github.com/ajolma/gdal/blob/trunk/gdal/map_algebra/test.cpp" rel="nofollow" link="external" target="_blank">https://github.com/ajolma/<wbr>gdal/blob/trunk/gdal/map_<wbr>algebra/test.cpp</a>
where on the lines 5 to 12 is a callback function.</p>
<p>When I left the code for my extended holidays I was doing
profiling and intended to test another approach for caching. Now
the caching is based on the native GDAL blocks, which is efficient
for going through a single dataset. Since the block may be
different for different datasets this approach may not be the best
when there are more than one dataset. For example the raster
algebra in QGIS uses an approach where the block is the same for
all datasets. That simplifies the code and may be very efficient
as a whole.</p>
<p>I'll be at the code sprint in FOSS4G 2016 and that would be a
good place to discuss the RFC too.<br>
</p>
<p>Best,<br>
</p>
<p>Ari</p>
<br>
<div>13.07.2016, 10:09, James Ramm
kirjoitti:<br>
</div>
<blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style="border-left:2px solid #cccccc;padding:0 1em" type="cite">
<div dir="ltr">Peter,
<div>I think 'Grid Algebra' would be what Ari Jolma is proposing
here: <a href="https://trac.osgeo.org/gdal/wiki/rfc62_raster_algebra" rel="nofollow" style="font-size:12.8px" link="external" target="_blank">https://trac.osgeo.org/<wbr>gdal/wiki/rfc62_raster_algebra</a></div>
<div><br>
</div>
<div>As Even pointed out, there is some overlap, though my
proposal is technically very different. </div>
<div>The key differences I see are:</div>
<div><br>
</div>
<div>- Users can submit functions which operate on each sub
window of the raster, rather than an algebraic expression.
This potentially allows for much more complicated algorithms
to be used (e.g. I dont think it would be possible to run a
watershed segmentation with a raster algebra implementation,
or to have algorithms which behave differently depending on
the 'location' within the raster or the values of surrounding
pixels etc etc).</div>
<div><br>
</div>
<div>- Functions can be chained together for a complex
processing toolchain. Some overlap with VRT here, although
again this introduces a little more flexibility. </div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 12 July 2016 at 15:47, Peter Halls <span dir="ltr"><<a href="http:///user/SendEmail.jtp?type=node&node=5279906&i=0" rel="nofollow" link="external" target="_blank">[hidden email]</a>></span>
wrote:<br>
<blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style="border-left:2px solid #cccccc;padding:0 1em" class="gmail_quote">
<div dir="ltr">
<div>
<div>James,<br>
<br>
</div>
in reality, are you not requesting an
implementation of Tomlin's 'Grid Algebra' in GDAL? That
defines the whole range of functions from whole raster
to pixel and has the distinct advantage of being both
published and extremely well known because of other
implementations ... which also provide ready-made
reference bases for the GDAL implementors ...<br>
<br>
</div>
Best wishes,<br>
Peter<br>
</div>
<div class="gmail_extra">
<div>
<div><br>
<div class="gmail_quote">On 12 July 2016 at 15:39,
James Ramm <span dir="ltr"><<a href="http:///user/SendEmail.jtp?type=node&node=5279906&i=1" rel="nofollow" link="external" target="_blank">[hidden email]</a>></span>
wrote:<br>
<blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style="border-left:2px solid #cccccc;padding:0 1em" class="gmail_quote">
<div dir="ltr">Hi Even,
<div><br>
</div>
<div>The difference I see with pixel functions
is that, as far as I understand, the pixel
function is applied per pixel, so there is no
possibility of e.g. the pixel buffer when have
the function apply to 'blocks'. </div>
<div>I may be way off, but many of the
algorithms we deal with require some kind of
neighbourhood search - a polygonise algorithm
or flow direction algorithm being good
examples. </div>
<div>I dont think VRT pixel functions allow
this? </div>
<div><br>
</div>
<div>So in that sense I'd see a VRT being 'just'
another potential input data source. </div>
<div><br>
</div>
<div>Perhaps VRT pixel functions could be
expanded to also allow 'window' functions?</div>
<div><br>
</div>
<div>A downside is it requires creating a VRT
even when you only want to apply a such a
function to a single dataset. Small effort,
but still a bit more than throwing in any
GDALDataset to be processed. </div>
<div><br>
</div>
<div>I see the overlap with raster algebra,
although yes technically very different. </div>
<div><br>
</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On 12 July 2016 at
14:55, Even Rouault <span dir="ltr"><<a href="http:///user/SendEmail.jtp?type=node&node=5279906&i=2" rel="nofollow" link="external" target="_blank">[hidden email]</a>></span>
wrote:<br>
<blockquote style='border-left:2px solid #CCCCCC;padding:0 1em' style="border-left:2px solid #cccccc;padding:0 1em" class="gmail_quote">James,<br>
<br>
There's some intersection with Ari's
proposal :<br>
<a href="https://trac.osgeo.org/gdal/wiki/rfc62_raster_algebra" rel="nofollow" link="external" target="_blank">https://trac.osgeo.org/gdal/<wbr>wiki/rfc62_raster_algebra</a>
. At least regarding the<br>
overall purposes, since technically this is
quite different.<br>
<br>
Actually what you propose is very close to
the existing VRT pixel functions of<br>
derived bands. See "Using Derived Bands" in<br>
<a href="http://www.gdal.org/gdal_vrttut.html" rel="nofollow" link="external" target="_blank">http://www.gdal.org/gdal_<wbr>vrttut.html</a>
. In the last days, we've merged<br>
Antonio's work regarding a predefined set of
pixel functions.<br>
Perhaps some extension to allow passing user
parameters to the pixel func<br>
could be useful. It is possible to use pixel
functions from Python as shown in<br>
<a href="https://github.com/OSGeo/gdal/blob/trunk/autotest/gcore/testnonboundtoswig.py#L303" rel="nofollow" link="external" target="_blank">https://github.com/OSGeo/gdal/<wbr>blob/trunk/autotest/gcore/<wbr>testnonboundtoswig.py#L303</a><br>
although this is a bit ugly as it uses
ctypes and not SWIG. But should be<br>
possible through SWIG by introducing proper
types similarly to what is done<br>
for the progress functions or error handler
functions.<br>
<br>
Even<br>
<div>
<div><br>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
<br>______________________________<wbr>_________________
<br>gdal-dev mailing list
<br><a href="http:///user/SendEmail.jtp?type=node&node=5279906&i=3" rel="nofollow" link="external" target="_blank">[hidden email]</a>
<br><a href="http://lists.osgeo.org/mailman/listinfo/gdal-dev" rel="nofollow" link="external" target="_blank">http://lists.osgeo.org/<wbr>mailman/listinfo/gdal-dev</a>
<br>
<br>
<hr noshade size="1" color="#cccccc">
<div style="color:#444;font:12px tahoma,geneva,helvetica,arial,sans-serif">
<div style="font-weight:bold">If you reply to this email, your message will be added to the discussion below:</div>
<a href="http://osgeo-org.1560.x6.nabble.com/GDAL-raster-processing-library-tp5275948p5279906.html" target="_blank" rel="nofollow" link="external">http://osgeo-org.1560.x6.<wbr>nabble.com/GDAL-raster-<wbr>processing-library-<wbr>tp5275948p5279906.html</a>
</div>
<div style="color:#666;font:11px tahoma,geneva,helvetica,arial,sans-serif;margin-top:.4em;line-height:1.5em">
To unsubscribe from GDAL raster processing library?, <a href="" target="_blank" rel="nofollow" link="external">click here</a>.<br>
<a href="http://osgeo-org.1560.x6.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml" rel="nofollow" style="font:9px serif" target="_blank" link="external">NAML</a>
</div></blockquote></div></div>
<br/><hr align="left" width="300" />
View this message in context: <a href="http://osgeo-org.1560.x6.nabble.com/GDAL-raster-processing-library-tp5275948p5279934.html">Re: GDAL raster processing library?</a><br/>
Sent from the <a href="http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html">GDAL - Dev mailing list archive</a> at Nabble.com.<br/>