[GRASS-dev] i.superpixels.slic ( was: #3142 Implementing SLIC image segmentation)

[Moritz Lennert]

On 19/12/16 09:38, Rashad Kanavath wrote:
>
>
> On Sun, Dec 18, 2016 at 2:53 PM, Moritz Lennert
> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>> wrote:
>
>     On 18/12/16 14:01, Rashad Kanavath wrote:
>
>         Hello,
>
>         As promised, I had pushed code to G7 addons repo
>
>         https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic>
>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic>>
>
>
>     Great job, thanks a lot !
>
>
>         I had tested with some small datasets and is working.
>
>         More testing welcome :)
>
>
>     I get the following warning during installation:
>
>     main.c: In function ‘main’:
>     main.c:514:13: warning: implicit declaration of function ‘min’
>     [-Wimplicit-function-declaration]
>          seedx = min(g_width-1,seedx);
>                  ^~~
>
>     I think you have to explicitely define min() through a macro, or
>     AFAIK you can use fmin() as elsewhere in the code.
>
>
> I will push a fix for that.

Thanks, but your fix creates another error:

main.c: In function ‘main’:
main.c:514:18: error: expected expression before ‘double’
      seedx = fmin(double (g_width-1), seedx);
                   ^~~~~~
main.c:514:13: error: too few arguments to function ‘fmin’
      seedx = fmin(double (g_width-1), seedx);
              ^~~~
In file included from /usr/include/features.h:364:0,
                  from /usr/include/stdio.h:27,
                  from main.c:21:
/usr/include/x86_64-linux-gnu/bits/mathcalls.h:360:1: note: declared here
  __MATHCALLX (fmin,, (_Mdouble_ __x, _Mdouble_ __y), (__const__));
  ^
make: *** [OBJ.x86_64-pc-linux-gnu/main.o] Erreur 1

as 'double' is not a function, so double(g_width-1)) doesn't make sense.

If you want to cast g_width-1 to double before submitting it to fmin(), 
then AFAIK this would have to be:

seedx = fmin ((double) (g_width-1), seedx)

However, again AFAIK, this type conversion is done implicitely anyhow, 
so you can just write:

seedx = fmin (g_width-1, seedx)


>
>
>     Also:
>
>     - The output as it is now is not very useful. What we would need is
>     a map with each pixel containing the label of the superpixel it
>     belongs to. In the original code, there are both outputs: i) an
>     image of the superpixel limits overlayed over the original image,
>     ii) the labeled pixels
>
>
> you need contour segments as seperate output and also the current one.
> right?

I don't know what you understand by 'contour segments'. What we would 
need would be an output similar to that of i.segment, i.e. each pixel 
having the id of the superpixel it belongs to (and there would be no 
specific boundary pixels).

The current output is actually not useful at all for us, I think. It is 
used for visualisation purposes in the original software, but in GRASS 
we can just vectorize the superpixels if we want to display their 
contours on top of the original image.

This means you can just get rid of the entire boundary detection part 
(lines 730-817 IIUC), and just output the klabels array to a map. 
Something like this seems to do the job (cf superpixel_id_output.png):

         int r, z;
         CELL *ubuff[nrows];
         for(  r = 0; r < nrows; r++ )
         {
                 ubuff[r] = Rast_allocate_c_buf();
         }

         z = 0;
         for (y = 0; y < nrows; y++)
         {
                 for(x = 0; x < ncols; x++)
                 {
                         ubuff[y][x] = klabels[z]+1; /* +1 to avoid 
category value 0*/
                         z++;
                 }
         }

     outfd = Rast_open_new(result, CELL_TYPE);

         for (z = 0; z < nrows; z++)
         {
         Rast_put_row(outfd, ubuff[z], CELL_TYPE);
         }

         for (z = 0; z < nrows; z++)
         {
                 G_free(ubuff[z]);
         }


>
>
>     - Linked to the above: currently the superpixel boundaries are two
>     pixels wide. A one-pixel boundary would be enough.
>
>
> you can adjust no of superpixels with k. default now is 200

Yes, but this is not what I am talking about.

Here's an example with the NC demo dataset:

g.region rast=lsat7_2002_10 -p
i.superpixels.slic red=lsat7_2002_30 green=lsat7_2002_20 
blue=lsat7_2002_10 iter=500 k=500 output=superpixels
d.rgb red=lsat7_2002_30 green=lsat7_2002_20 blue=lsat7_2002_10
d.rast map=superpixels values=1

Then zoom in close, and look at the boundaries (see 
superpixels_large_boudaries.png attached). You can see that the boundary 
is at least 2 pixels wide. I would expect a one-pixel boundary to be enough.

But this is actually not very important since, as mentioned above, the 
output with the boundaries is not really useful IMHO.


>
>
>     - I don't know the details of the algorithm well enough, but would
>     it be possible to extend it beyond the use of r,g,b maps as input.
>     Ideally, the use should be able to provide a group as input and all
>     maps in the group are used in the definition of the superpixels.
>     This should also include the case when you only have one band (e.g.
>     black and white orthophoto).
>
>
> I need help in understanding this part. because in code, it operates on
> lab color space. first it takes RBG and convert to LAB. This is reason,
> I had to put red,green,blue parameters.

I think that the authors come more from pattern recognition in images, 
and link that pattern recognition to human recognizable color space. And 
LAB color space is considered closer to human perception.

After perfunctory reading of their technical paper and a glance over the 
code, however, I don't really see any reason why going through LAB color 
space would be necessary.

I think you can just skip the whole translation to LAB space part and 
use as many bands as you wish. The only difference would be that you 
would use kseedsl, kseedsa, kseedsb, but as many kseedsN as you have 
bands (possibly an array of arrays with N arrays of seed arrays where N 
is the number bands), and spectral distance would be calculated not by

dist =  (L[i] - kseedsl[n])*(L[i] - kseedsl[n]) +
	(A[i] - kseedsa[n])*(A[i] - kseedsa[n]) +
	(B[i] - kseedsb[n])*(B[i] - kseedsb[n]);

but through the equivalent for whatever number of bands you have.

So, for me, the next steps would be:

- implementation of id output as above
- implementation of the algorithm using any number of bands (without 
going through the LAB color space)
- implementation of user defined compactness, and, better yet, SLICO, 
i.e. automatic detection of compactness. More info and the source code 
of that are at the bottom of http://ivrl.epfl.ch/research/superpixels.


Moritz







-------------- next part --------------
A non-text attachment was scrubbed...
Name: superpixels_large_boundaries.png
Type: image/png
Size: 9155 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20161219/9e1810be/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: superpixel_id_output.png
Type: image/png
Size: 59259 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20161219/9e1810be/attachment-0003.png>