[GRASS-dev] Compute mahalanobis distance using Scipy

[Javier Martínez-López]

That would be great, I would like to learn and help doing that!
El 16/02/2015 12:07, "Paulo van Breugel" <p.vanbreugel at gmail.com> escribió:

> Hi Javier,
>
> Thanks. I need to test whether the mahalanobis function you provided is
> faster than the one I used (which Glynn wrote, based on the numpy function
> I believe); I think I can use both together with the function to run it in
> parallel, so it would be a matter of selecting the fastest one (or the one
> using less memory, if there is such a difference).
>
> I am not sure if I understand the parallel part, but it seems the data is
> 'sliced' in subsets to compute the distance. If so, that should indeed
> solve the problem with large files? Anyway, I'll try how it works for me.
>
> For now I do the partitioning of the data in subsets in GRASS (create a
> dummy layer, split that in tiles using r.tile, set the region to match tile
> 1, import the input layers in python (which will be restricted to the
> region defined by tile 1), run the function and import the result back in
> GRASS, set region to match tile 2, impor the data... etc). Much slower I
> guess (not really tested yet), but should be able to handle any data size
> that GRASS can handle.
>
> To create a grass plugin to compute the mahalanobis distance based on that
> code would be trivial I think, I can give that a try if you want. As for
> the rest of the code, I would be interested to help, but my grass/python
> programming are very limited, so hopefully there are others interested as
> well.
>
> Cheers,
>
> Paulo
>
>
>
> On Mon, Feb 16, 2015 at 11:35 AM, Javier Martínez-López <
> javi.martinez.lopez at gmail.com> wrote:
>
>> Hi Paulo,
>>
>> to use it with python just copy the mahalanobis function definitions,
>> import all necessary libraries and use it as in line 597 (the
>> covariance matrix and the mean are computed just before this line). I
>> am sorry that the code is not documented yet. I am not sure if this
>> will solve the large raster layers problem but you can try it. As you
>> can see in the script, I did not write the parallel mahalanobis
>> functions (Sturla Molden in CC did it) and it works perfectly for me,
>> so I agree that it would be great to implement it also in GRASS (maybe
>> as an add on?). However, I am not so experienced with that. The
>> ehab.py script also does a lot data preprocessing to allow batch runs
>> using heterogeneous data sources and finally performs some patch
>> analysis and metrics out of the resulting similarity maps using the
>> ndimage library (scipy). Any help on how to implement it in GRASS
>> would be more than welcome! Cheers, Javier
>>
>> On Sun, Feb 15, 2015 at 10:21 PM, Paulo van Breugel
>> <p.vanbreugel at gmail.com> wrote:
>> > Hi Javier,
>> >
>> > This looks really useful, thanks! My python skills are very limited, so
>> I
>> > will have to look at it better to see what I can understand. But to
>> start,
>> > the parallel function, does that also solve the handling of potentially
>> very
>> > large raster layers?
>> >
>> > Btw, this does look like a very useful set of functions, any plan to
>> develop
>> > this into a GRASS addon or function?
>> >
>> > Paulo
>> >
>> >
>> > On Sun, Feb 15, 2015 at 9:22 PM, Javier Martínez-López
>> > <javi.martinez.lopez at gmail.com> wrote:
>> >>
>> >> Hi Paulo,
>> >>
>> >> you can see an implementation of the mahalanobis distance computed in
>> >> parallel (using all computer processors) for a single image here:
>> >>
>> >> https://github.com/javimarlop/eHabpy/blob/master/ehab.py
>> >>
>> >> see lines 32-86 and 597. I hope it helps! Cheers, Javier
>> >>
>> >> On Sat, Feb 14, 2015 at 7:29 PM, Vaclav Petras <wenzeslaus at gmail.com>
>> >> wrote:
>> >> >
>> >> > On Sat, Feb 14, 2015 at 11:47 AM, Paulo van Breugel
>> >> > <p.vanbreugel at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >>
>> >> >> For a quick solution, what about using r.tile to split the input
>> data
>> >> >> in
>> >> >> tiles and compute the mahalanobis distance per tile.
>> >> >
>> >> >
>> >> > See PyGRASS GridModule class which will do tiling (and a lot of other
>> >> > things) for you.
>> >> >
>> >> >
>> >> >
>> http://grass.osgeo.org/grass70/manuals/libpython/pygrass.modules.grid.html
>> >> >
>> >> > _______________________________________________
>> >> > grass-dev mailing list
>> >> > grass-dev at lists.osgeo.org
>> >> > http://lists.osgeo.org/mailman/listinfo/grass-dev
>> >
>> >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20150216/8b5a48d3/attachment-0001.html>