[GRASS-dev] i.superpixels.slic ( was: #3142 Implementing SLIC image segmentation)

Mon Dec 19 08:30:13 PST 2016

Hi Rashad,

Another reflection: IIUC, in the current implementation all pixel values 
from all bands are read into memory. This can quickly become a limiting 
factor when working with very large images.

For now we can work this way until the general algorithm is clear, but 
in the long run, I imagine that the segment library should be used to 
only load parts of the data at a time.

Maybe MarkusM help us with that ?

Moritz

On 19/12/16 16:05, Moritz Lennert wrote:
> On 19/12/16 09:38, Rashad Kanavath wrote:
>>
>>
>> On Sun, Dec 18, 2016 at 2:53 PM, Moritz Lennert
>> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>> wrote:
>>
>>     On 18/12/16 14:01, Rashad Kanavath wrote:
>>
>>         Hello,
>>
>>         As promised, I had pushed code to G7 addons repo
>>
>>         https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic>
>>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>>         <https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic>>
>>
>>
>>     Great job, thanks a lot !
>>
>>
>>         I had tested with some small datasets and is working.
>>
>>         More testing welcome :)
>>
>>
>>     I get the following warning during installation:
>>
>>     main.c: In function ‘main’:
>>     main.c:514:13: warning: implicit declaration of function ‘min’
>>     [-Wimplicit-function-declaration]
>>          seedx = min(g_width-1,seedx);
>>                  ^~~
>>
>>     I think you have to explicitely define min() through a macro, or
>>     AFAIK you can use fmin() as elsewhere in the code.
>>
>>
>> I will push a fix for that.
>
> Thanks, but your fix creates another error:
>
> main.c: In function ‘main’:
> main.c:514:18: error: expected expression before ‘double’
>       seedx = fmin(double (g_width-1), seedx);
>                    ^~~~~~
> main.c:514:13: error: too few arguments to function ‘fmin’
>       seedx = fmin(double (g_width-1), seedx);
>               ^~~~
> In file included from /usr/include/features.h:364:0,
>                   from /usr/include/stdio.h:27,
>                   from main.c:21:
> /usr/include/x86_64-linux-gnu/bits/mathcalls.h:360:1: note: declared here
>   __MATHCALLX (fmin,, (_Mdouble_ __x, _Mdouble_ __y), (__const__));
>   ^
> make: *** [OBJ.x86_64-pc-linux-gnu/main.o] Erreur 1
>
> as 'double' is not a function, so double(g_width-1)) doesn't make sense.
>
> If you want to cast g_width-1 to double before submitting it to fmin(),
> then AFAIK this would have to be:
>
> seedx = fmin ((double) (g_width-1), seedx)
>
> However, again AFAIK, this type conversion is done implicitely anyhow,
> so you can just write:
>
> seedx = fmin (g_width-1, seedx)
>
>
>>
>>
>>     Also:
>>
>>     - The output as it is now is not very useful. What we would need is
>>     a map with each pixel containing the label of the superpixel it
>>     belongs to. In the original code, there are both outputs: i) an
>>     image of the superpixel limits overlayed over the original image,
>>     ii) the labeled pixels
>>
>>
>> you need contour segments as seperate output and also the current one.
>> right?
>
> I don't know what you understand by 'contour segments'. What we would
> need would be an output similar to that of i.segment, i.e. each pixel
> having the id of the superpixel it belongs to (and there would be no
> specific boundary pixels).
>
> The current output is actually not useful at all for us, I think. It is
> used for visualisation purposes in the original software, but in GRASS
> we can just vectorize the superpixels if we want to display their
> contours on top of the original image.
>
> This means you can just get rid of the entire boundary detection part
> (lines 730-817 IIUC), and just output the klabels array to a map.
> Something like this seems to do the job (cf superpixel_id_output.png):
>
>          int r, z;
>          CELL *ubuff[nrows];
>          for(  r = 0; r < nrows; r++ )
>          {
>                  ubuff[r] = Rast_allocate_c_buf();
>          }
>
>          z = 0;
>          for (y = 0; y < nrows; y++)
>          {
>                  for(x = 0; x < ncols; x++)
>                  {
>                          ubuff[y][x] = klabels[z]+1; /* +1 to avoid
> category value 0*/
>                          z++;
>                  }
>          }
>
>      outfd = Rast_open_new(result, CELL_TYPE);
>
>          for (z = 0; z < nrows; z++)
>          {
>          Rast_put_row(outfd, ubuff[z], CELL_TYPE);
>          }
>
>          for (z = 0; z < nrows; z++)
>          {
>                  G_free(ubuff[z]);
>          }
>
>
>>
>>
>>     - Linked to the above: currently the superpixel boundaries are two
>>     pixels wide. A one-pixel boundary would be enough.
>>
>>
>> you can adjust no of superpixels with k. default now is 200
>
> Yes, but this is not what I am talking about.
>
> Here's an example with the NC demo dataset:
>
> g.region rast=lsat7_2002_10 -p
> i.superpixels.slic red=lsat7_2002_30 green=lsat7_2002_20
> blue=lsat7_2002_10 iter=500 k=500 output=superpixels
> d.rgb red=lsat7_2002_30 green=lsat7_2002_20 blue=lsat7_2002_10
> d.rast map=superpixels values=1
>
> Then zoom in close, and look at the boundaries (see
> superpixels_large_boudaries.png attached). You can see that the boundary
> is at least 2 pixels wide. I would expect a one-pixel boundary to be enough.
>
> But this is actually not very important since, as mentioned above, the
> output with the boundaries is not really useful IMHO.
>
>
>>
>>
>>     - I don't know the details of the algorithm well enough, but would
>>     it be possible to extend it beyond the use of r,g,b maps as input.
>>     Ideally, the use should be able to provide a group as input and all
>>     maps in the group are used in the definition of the superpixels.
>>     This should also include the case when you only have one band (e.g.
>>     black and white orthophoto).
>>
>>
>> I need help in understanding this part. because in code, it operates on
>> lab color space. first it takes RBG and convert to LAB. This is reason,
>> I had to put red,green,blue parameters.
>
> I think that the authors come more from pattern recognition in images,
> and link that pattern recognition to human recognizable color space. And
> LAB color space is considered closer to human perception.
>
> After perfunctory reading of their technical paper and a glance over the
> code, however, I don't really see any reason why going through LAB color
> space would be necessary.
>
> I think you can just skip the whole translation to LAB space part and
> use as many bands as you wish. The only difference would be that you
> would use kseedsl, kseedsa, kseedsb, but as many kseedsN as you have
> bands (possibly an array of arrays with N arrays of seed arrays where N
> is the number bands), and spectral distance would be calculated not by
>
> dist =  (L[i] - kseedsl[n])*(L[i] - kseedsl[n]) +
> 	(A[i] - kseedsa[n])*(A[i] - kseedsa[n]) +
> 	(B[i] - kseedsb[n])*(B[i] - kseedsb[n]);
>
> but through the equivalent for whatever number of bands you have.
>
> So, for me, the next steps would be:
>
> - implementation of id output as above
> - implementation of the algorithm using any number of bands (without
> going through the LAB color space)
> - implementation of user defined compactness, and, better yet, SLICO,
> i.e. automatic detection of compactness. More info and the source code
> of that are at the bottom of http://ivrl.epfl.ch/research/superpixels.
>
>
> Moritz
>
>
>
>
>
>
>
>
>
> _______________________________________________
> grass-dev mailing list
> grass-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-dev
>