[GRASS-dev] i.segment on panchromatic band of Worldview 2 scene: what resources are necessary to complete segmentation ?

[Markus Metz]

On Mon, Jul 8, 2013 at 2:18 PM, Moritz Lennert
<mlennert at club.worldonline.be> wrote:
> After some improvements to the module by MarkusM, I tried again during the
> night, but it got stuck again with heavy swapping going on. I'm definitively
> giving up to segment the panchromatic image in its entirety on my machine.
>
> Question: Does the memory= parameter limit total memory usage of the module,
> or does it only limit the memory usage for certain parts of it ?

The memory= parameter should limit the total memory usage. In my
tests, it used always a bit less memory than specified. It would be a
bug if the module uses more memory than specified.

The only limitations are 1) disk space, 2) how long you want to wait.
Memory should not be a limit because of the memory option.

>
>
> On 07/07/13 11:52, Yann Chemin wrote:
>>
>
>>
>> 2 - Cut in pieces (See Markus Comment about supercomputing) and run 4
>> quads on 4 CPUs instead of one large image in one CPU.
>
>
> I can do that. Will this have any influence on segmentation results ?

Of course. There will be edge effects, objects will not cross region
borders. Therefore you would need one final run with the pieces
patched together, thus using the former output as input seeds for the
last run. This last run should be faster and use less memory because
it uses seeds.

On a standard HPC system, you are usually allowed to compile your own
software in your $HOME. You could compile trunk there, then set up the
tiles and tile management, and launch jobs with the HPC's job manager
(typically gridengine or derivates, torque and/or MPI).

>>
>> 3 - Recode i.segment to work natively on heterogeneous computers...
>> (would be fun!)
>
>
> Would parallization help ? Should i.segment be included into the list of
> OpenMP candidates:
> http://grasswiki.osgeo.org/wiki/OpenMP#Candidates ?

Hmm, because the final global run is needed to remove edge effects, I
would expect the results to be different (not wrong, but different).
But definitively an interesting idea!

Markus M

>
> Moritz
>
>
>> Good luck!
>>
>> On 4 July 2013 14:10, Moritz Lennert<mlennert at club.worldonline.be>  wrote:
>>>
>>> Hello,
>>>
>>> In parallel to the discussion going on in another thread, I have a
>>> question
>>> concering the segmentation of another Worldview 2 scene:
>>>
>>> I first used all 8 multispectral bands and managed to get a series of
>>> results with increasing thresholds in very reasonable running times. The
>>> region was as follows:
>>>
>>>> g.region -p
>>>
>>> projection: 1 (UTM)
>>> zone:       -36
>>> datum:      wgs84
>>> ellipsoid:  wgs84
>>> north:      7251172
>>> south:      7234772
>>> west:       333792
>>> east:       350192
>>> nsres:      2
>>> ewres:      2
>>> rows:       8200
>>> cols:       8200
>>> cells:      67240000
>>>
>>> and the command line:
>>>
>>> i.segment group=xs out=seg_xs minsize=2 memory=3072 threshold=0.05 (and
>>> thresh=0.1 and 0.2 in successive runs using the results of the previous
>>> run
>>> as seeds).
>>>
>>> Now, I would like to test segmentation of just the panchromatic band.
>>> This
>>> means the following region settings:
>>>
>>> projection: 1 (UTM)
>>> zone:       -36
>>> datum:      wgs84
>>> ellipsoid:  wgs84
>>> north:      7251172
>>> south:      7234772
>>> west:       333792
>>> east:       350192
>>> nsres:      0.5
>>> ewres:      0.5
>>> rows:       32800
>>> cols:       32800
>>> cells:      1075840000
>>>
>>> Trying to run with the following command line on my i3, 8GB RAM machine:
>>>
>>> i.segment group=pan out=seg_pan_005 threshold=0.05 memory=3072
>>>
>>> had the process running for almost 13 hours with it then becoming
>>> apparently
>>> stuck in the fourth pass at 10%. At that point the percent didn't change
>>> for
>>> over an hour, so I decided to kill the process. Can I assume that I'm
>>> here
>>> above the capacities of my machine ? Is there anything (besides working
>>> on a
>>> smaller subsample of the image) that I can do to make it work ? What kind
>>> of
>>> resources would I need to be able to run such a segmentation?
>>>
>>> I guess I'll have to move these kinds of treatments to our university
>>> supercomputer, but I first have to get them to install GRASS...
>>>
>>> Moritz
>>> _______________________________________________
>>> grass-dev mailing list
>>> grass-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>>
>>
>>
>>
>
>