[GRASS-dev] i.superpixels.slic ( was: #3142 Implementing SLIC image segmentation)
Markus Metz
markus.metz.giswork at gmail.com
Tue Dec 20 01:04:28 PST 2016
On Mon, Dec 19, 2016 at 6:02 PM, Rashad Kanavath <mohammedrashadkm at gmail.com>
wrote:
>
> Hello
>
> On Mon, Dec 19, 2016 at 5:30 PM, Moritz Lennert <
mlennert at club.worldonline.be> wrote:
>>
>> Hi Rashad,
>>
>> Another reflection: IIUC, in the current implementation all pixel values
from all bands are read into memory. This can quickly become a limiting
factor when working with very large images.
>>
>> For now we can work this way until the general algorithm is clear, but
in the long run, I imagine that the segment library should be used to only
load parts of the data at a time.
>
> I had asked this question myself :-).
> But then I don't know how to deal with large datasets in grass. I
Couldn't find any method to allow streaming/tiled processing. That would be
awesome!
>
> I look forward reply from MarkusM for this.
We can use some methods from i.segment to handle large datasets, i.e. using
the segment library, adding a memory option etc. Also keeping track of
superpixel characteristics can be adopted from keeping track of region
properties in the region growing algorithm.
About RGB or LAB, I agree with Moritz that this does not matter, instead it
is more important that any number of bands can be used as input, and any
kind of data, e.g. mixing vegetation indices with spectral bands.
Thanks a lot for the implementation! Since this is work in progress, the
module could be moved from grass-addons to sandbox until the name and
options are fixed?
Markus M
>
> Thanks again.
>
>>
>> Maybe MarkusM help us with that ?
>>
>> Moritz
>>
>>
>> On 19/12/16 16:05, Moritz Lennert wrote:
>>>
>>> On 19/12/16 09:38, Rashad Kanavath wrote:
>>>>
>>>>
>>>>
>>>> On Sun, Dec 18, 2016 at 2:53 PM, Moritz Lennert
>>>> <mlennert at club.worldonline.be <mailto:mlennert at club.worldonline.be>>
wrote:
>>>>
>>>> On 18/12/16 14:01, Rashad Kanavath wrote:
>>>>
>>>> Hello,
>>>>
>>>> As promised, I had pushed code to G7 addons repo
>>>>
>>>>
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>>>> <
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>
>>>> <
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>>>> <
https://trac.osgeo.org/grass/browser/grass-addons/grass7/imagery/i.superpixels.slic
>>
>>>>
>>>>
>>>> Great job, thanks a lot !
>>>>
>>>>
>>>> I had tested with some small datasets and is working.
>>>>
>>>> More testing welcome :)
>>>>
>>>>
>>>> I get the following warning during installation:
>>>>
>>>> main.c: In function ‘main’:
>>>> main.c:514:13: warning: implicit declaration of function ‘min’
>>>> [-Wimplicit-function-declaration]
>>>> seedx = min(g_width-1,seedx);
>>>> ^~~
>>>>
>>>> I think you have to explicitely define min() through a macro, or
>>>> AFAIK you can use fmin() as elsewhere in the code.
>>>>
>>>>
>>>> I will push a fix for that.
>>>
>>>
>>> Thanks, but your fix creates another error:
>>>
>>> main.c: In function ‘main’:
>>> main.c:514:18: error: expected expression before ‘double’
>>> seedx = fmin(double (g_width-1), seedx);
>>> ^~~~~~
>>> main.c:514:13: error: too few arguments to function ‘fmin’
>>> seedx = fmin(double (g_width-1), seedx);
>>> ^~~~
>>> In file included from /usr/include/features.h:364:0,
>>> from /usr/include/stdio.h:27,
>>> from main.c:21:
>>> /usr/include/x86_64-linux-gnu/bits/mathcalls.h:360:1: note: declared
here
>>> __MATHCALLX (fmin,, (_Mdouble_ __x, _Mdouble_ __y), (__const__));
>>> ^
>>> make: *** [OBJ.x86_64-pc-linux-gnu/main.o] Erreur 1
>>>
>>> as 'double' is not a function, so double(g_width-1)) doesn't make sense.
>>>
>>> If you want to cast g_width-1 to double before submitting it to fmin(),
>>> then AFAIK this would have to be:
>>>
>>> seedx = fmin ((double) (g_width-1), seedx)
>>>
>>> However, again AFAIK, this type conversion is done implicitely anyhow,
>>> so you can just write:
>>>
>>> seedx = fmin (g_width-1, seedx)
>>>
>>>
>>>>
>>>>
>>>> Also:
>>>>
>>>> - The output as it is now is not very useful. What we would need is
>>>> a map with each pixel containing the label of the superpixel it
>>>> belongs to. In the original code, there are both outputs: i) an
>>>> image of the superpixel limits overlayed over the original image,
>>>> ii) the labeled pixels
>>>>
>>>>
>>>> you need contour segments as seperate output and also the current one.
>>>> right?
>>>
>>>
>>> I don't know what you understand by 'contour segments'. What we would
>>> need would be an output similar to that of i.segment, i.e. each pixel
>>> having the id of the superpixel it belongs to (and there would be no
>>> specific boundary pixels).
>>>
>>> The current output is actually not useful at all for us, I think. It is
>>> used for visualisation purposes in the original software, but in GRASS
>>> we can just vectorize the superpixels if we want to display their
>>> contours on top of the original image.
>>>
>>> This means you can just get rid of the entire boundary detection part
>>> (lines 730-817 IIUC), and just output the klabels array to a map.
>>> Something like this seems to do the job (cf superpixel_id_output.png):
>>>
>>> int r, z;
>>> CELL *ubuff[nrows];
>>> for( r = 0; r < nrows; r++ )
>>> {
>>> ubuff[r] = Rast_allocate_c_buf();
>>> }
>>>
>>> z = 0;
>>> for (y = 0; y < nrows; y++)
>>> {
>>> for(x = 0; x < ncols; x++)
>>> {
>>> ubuff[y][x] = klabels[z]+1; /* +1 to avoid
>>> category value 0*/
>>> z++;
>>> }
>>> }
>>>
>>> outfd = Rast_open_new(result, CELL_TYPE);
>>>
>>> for (z = 0; z < nrows; z++)
>>> {
>>> Rast_put_row(outfd, ubuff[z], CELL_TYPE);
>>> }
>>>
>>> for (z = 0; z < nrows; z++)
>>> {
>>> G_free(ubuff[z]);
>>> }
>>>
>>>
>>>>
>>>>
>>>> - Linked to the above: currently the superpixel boundaries are two
>>>> pixels wide. A one-pixel boundary would be enough.
>>>>
>>>>
>>>> you can adjust no of superpixels with k. default now is 200
>>>
>>>
>>> Yes, but this is not what I am talking about.
>>>
>>> Here's an example with the NC demo dataset:
>>>
>>> g.region rast=lsat7_2002_10 -p
>>> i.superpixels.slic red=lsat7_2002_30 green=lsat7_2002_20
>>> blue=lsat7_2002_10 iter=500 k=500 output=superpixels
>>> d.rgb red=lsat7_2002_30 green=lsat7_2002_20 blue=lsat7_2002_10
>>> d.rast map=superpixels values=1
>>>
>>> Then zoom in close, and look at the boundaries (see
>>> superpixels_large_boudaries.png attached). You can see that the boundary
>>> is at least 2 pixels wide. I would expect a one-pixel boundary to be
enough.
>>>
>>> But this is actually not very important since, as mentioned above, the
>>> output with the boundaries is not really useful IMHO.
>>>
>>>
>>>>
>>>>
>>>> - I don't know the details of the algorithm well enough, but would
>>>> it be possible to extend it beyond the use of r,g,b maps as input.
>>>> Ideally, the use should be able to provide a group as input and all
>>>> maps in the group are used in the definition of the superpixels.
>>>> This should also include the case when you only have one band (e.g.
>>>> black and white orthophoto).
>>>>
>>>>
>>>> I need help in understanding this part. because in code, it operates on
>>>> lab color space. first it takes RBG and convert to LAB. This is reason,
>>>> I had to put red,green,blue parameters.
>>>
>>>
>>> I think that the authors come more from pattern recognition in images,
>>> and link that pattern recognition to human recognizable color space. And
>>> LAB color space is considered closer to human perception.
>>>
>>> After perfunctory reading of their technical paper and a glance over the
>>> code, however, I don't really see any reason why going through LAB color
>>> space would be necessary.
>>>
>>> I think you can just skip the whole translation to LAB space part and
>>> use as many bands as you wish. The only difference would be that you
>>> would use kseedsl, kseedsa, kseedsb, but as many kseedsN as you have
>>> bands (possibly an array of arrays with N arrays of seed arrays where N
>>> is the number bands), and spectral distance would be calculated not by
>>>
>>> dist = (L[i] - kseedsl[n])*(L[i] - kseedsl[n]) +
>>> (A[i] - kseedsa[n])*(A[i] - kseedsa[n]) +
>>> (B[i] - kseedsb[n])*(B[i] - kseedsb[n]);
>>>
>>> but through the equivalent for whatever number of bands you have.
>>>
>>> So, for me, the next steps would be:
>>>
>>> - implementation of id output as above
>>> - implementation of the algorithm using any number of bands (without
>>> going through the LAB color space)
>>> - implementation of user defined compactness, and, better yet, SLICO,
>>> i.e. automatic detection of compactness. More info and the source code
>>> of that are at the bottom of http://ivrl.epfl.ch/research/superpixels.
>>>
>>>
>>> Moritz
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> grass-dev mailing list
>>> grass-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>>>
>>
>>
>
>
>
> --
> Regards,
> Rashad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20161220/7945a25f/attachment-0001.html>
More information about the grass-dev
mailing list