[GRASS-dev] i.superpixels.slic ( was: #3142 Implementing SLIC image segmentation)

[Rashad Kanavath]

On Mon, Dec 19, 2016 at 4:05 PM, Moritz Lennert <
mlennert at club.worldonline.be> wrote:

> On 19/12/16 09:38, Rashad Kanavath wrote:
>
>>
>>
>> On Sun, Dec 18, 2016 at 2:53 PM, Moritz Lennert
>> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>>
>> wrote:
>>
>>     On 18/12/16 14:01, Rashad Kanavath wrote:
>>
>>         Hello,
>>
>>         As promised, I had pushed code to G7 addons repo
>>
>>         https://trac.osgeo.org/grass/browser/grass-addons/grass7/ima
>> gery/i.superpixels.slic
>>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/im
>> agery/i.superpixels.slic>
>>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/im
>> agery/i.superpixels.slic
>>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/im
>> agery/i.superpixels.slic>>
>>
>>
>>     Great job, thanks a lot !
>>
>>
>>         I had tested with some small datasets and is working.
>>
>>         More testing welcome :)
>>
>>
>>     I get the following warning during installation:
>>
>>     main.c: In function ‘main’:
>>     main.c:514:13: warning: implicit declaration of function ‘min’
>>     [-Wimplicit-function-declaration]
>>          seedx = min(g_width-1,seedx);
>>                  ^~~
>>
>>     I think you have to explicitely define min() through a macro, or
>>     AFAIK you can use fmin() as elsewhere in the code.
>>
>>
>> I will push a fix for that.
>>
>
> Thanks, but your fix creates another error:
>
> main.c: In function ‘main’:
> main.c:514:18: error: expected expression before ‘double’
>      seedx = fmin(double (g_width-1), seedx);
>                   ^~~~~~
> main.c:514:13: error: too few arguments to function ‘fmin’
>      seedx = fmin(double (g_width-1), seedx);
>              ^~~~
> In file included from /usr/include/features.h:364:0,
>                  from /usr/include/stdio.h:27,
>                  from main.c:21:
> /usr/include/x86_64-linux-gnu/bits/mathcalls.h:360:1: note: declared here
>  __MATHCALLX (fmin,, (_Mdouble_ __x, _Mdouble_ __y), (__const__));
>  ^
> make: *** [OBJ.x86_64-pc-linux-gnu/main.o] Erreur 1
>
> as 'double' is not a function, so double(g_width-1)) doesn't make sense.
>
> If you want to cast g_width-1 to double before submitting it to fmin(),
> then AFAIK this would have to be:
>
> seedx = fmin ((double) (g_width-1), seedx)
>
> However, again AFAIK, this type conversion is done implicitely anyhow, so
> you can just write:
>
> seedx = fmin (g_width-1, seedx)
>
>
>
>>
>>     Also:
>>
>>     - The output as it is now is not very useful. What we would need is
>>     a map with each pixel containing the label of the superpixel it
>>     belongs to. In the original code, there are both outputs: i) an
>>     image of the superpixel limits overlayed over the original image,
>>     ii) the labeled pixels
>>
>>
>> you need contour segments as seperate output and also the current one.
>> right?
>>
>
> I don't know what you understand by 'contour segments'. What we would need
> would be an output similar to that of i.segment, i.e. each pixel having the
> id of the superpixel it belongs to (and there would be no specific boundary
> pixels).
>
> The current output is actually not useful at all for us, I think. It is
> used for visualisation purposes in the original software, but in GRASS we
> can just vectorize the superpixels if we want to display their contours on
> top of the original image.
>
> This means you can just get rid of the entire boundary detection part
> (lines 730-817 IIUC), and just output the klabels array to a map. Something
> like this seems to do the job (cf superpixel_id_output.png):
>
>         int r, z;
>         CELL *ubuff[nrows];
>         for(  r = 0; r < nrows; r++ )
>         {
>                 ubuff[r] = Rast_allocate_c_buf();
>         }
>
>         z = 0;
>         for (y = 0; y < nrows; y++)
>         {
>                 for(x = 0; x < ncols; x++)
>                 {
>                         ubuff[y][x] = klabels[z]+1; /* +1 to avoid
> category value 0*/
>                         z++;
>                 }
>         }
>
>     outfd = Rast_open_new(result, CELL_TYPE);
>
>         for (z = 0; z < nrows; z++)
>         {
>         Rast_put_row(outfd, ubuff[z], CELL_TYPE);
>         }
>
>         for (z = 0; z < nrows; z++)
>         {
>                 G_free(ubuff[z]);
>         }
>
>
>
>>
>>     - Linked to the above: currently the superpixel boundaries are two
>>     pixels wide. A one-pixel boundary would be enough.
>>
>>
>> you can adjust no of superpixels with k. default now is 200
>>
>
> Yes, but this is not what I am talking about.
>
> Here's an example with the NC demo dataset:
>
> g.region rast=lsat7_2002_10 -p
> i.superpixels.slic red=lsat7_2002_30 green=lsat7_2002_20
> blue=lsat7_2002_10 iter=500 k=500 output=superpixels
> d.rgb red=lsat7_2002_30 green=lsat7_2002_20 blue=lsat7_2002_10
> d.rast map=superpixels values=1
>
> Then zoom in close, and look at the boundaries (see
> superpixels_large_boudaries.png attached). You can see that the boundary
> is at least 2 pixels wide. I would expect a one-pixel boundary to be enough.
>
> But this is actually not very important since, as mentioned above, the
> output with the boundaries is not really useful IMHO.
>
>
>
>>
>>     - I don't know the details of the algorithm well enough, but would
>>     it be possible to extend it beyond the use of r,g,b maps as input.
>>     Ideally, the use should be able to provide a group as input and all
>>     maps in the group are used in the definition of the superpixels.
>>     This should also include the case when you only have one band (e.g.
>>     black and white orthophoto).
>>
>>
>> I need help in understanding this part. because in code, it operates on
>> lab color space. first it takes RBG and convert to LAB. This is reason,
>> I had to put red,green,blue parameters.
>>
>
> I think that the authors come more from pattern recognition in images, and
> link that pattern recognition to human recognizable color space. And LAB
> color space is considered closer to human perception.
>
> After perfunctory reading of their technical paper and a glance over the
> code, however, I don't really see any reason why going through LAB color
> space would be necessary.
>
> I think you can just skip the whole translation to LAB space part and use
> as many bands as you wish. The only difference would be that you would use
> kseedsl, kseedsa, kseedsb, but as many kseedsN as you have bands (possibly
> an array of arrays with N arrays of seed arrays where N is the number
> bands), and spectral distance would be calculated not by
>
> dist =  (L[i] - kseedsl[n])*(L[i] - kseedsl[n]) +
>         (A[i] - kseedsa[n])*(A[i] - kseedsa[n]) +
>         (B[i] - kseedsb[n])*(B[i] - kseedsb[n]);
>
> but through the equivalent for whatever number of bands you have.
>
> So, for me, the next steps would be:
>
> - implementation of id output as above
> - implementation of the algorithm using any number of bands (without going
> through the LAB color space)
> - implementation of user defined compactness, and, better yet, SLICO, i.e.
> automatic detection of compactness. More info and the source code of that
> are at the bottom of http://ivrl.epfl.ch/research/superpixels.
>
>
Thanks for your input on this. It is really helpful to me. I will continue
to work on this and let you know when I push next version on addons.

Hopefully next weekend.

>
> Moritz
>
>
>
>
>
>
>
>


-- 
Regards,
   Rashad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20161219/b3207f9b/attachment-0001.html>