[GRASS-user] G_malloc errors in i.segment
Moritz Lennert
mlennert at club.worldonline.be
Thu Jun 21 08:10:16 PDT 2018
Hi Paul,
On 21/06/18 16:54, Paul Shapley wrote:
> Hi Marcus,
>
> Thanks again for your patience with this issue. It seems that whatever i
> set in the memory of 'i.segment' to it throws up the same 'G_malloc'
> error with the same 3 gb shortfall. I had another idea which is to make
> a smaller subset image (but if i do this) will the new subset of 'id'
> segments be applicable if applied to the entire image. For example if a
> value of '5134' represents 'peat' could i use 'r.mapcalc' to query the
> entire image for those values of '5134'. Supervised and Unsupervised
> Classification would not yield the differences in the landscape i'm
> looking for which 'i.segment' succeeds.
i.segment does not classify (i.e. label segments with a semantic
meaning). It only creates segments. Across an image you will have many
(without your size of image probably millions of segments which will
correspond to the same class, but each segment will have its own id. In
addition these ids are somewhat random, so you cannot deduct any meaning
from them.
In order to classify your map you will have to either use classification
rules or go for supervised classification of your segments. Currently
GRASS GIS offers the addons i.segment.stats and v.class.mlR to do this.
Check this article [1] and the related Jupyter notebook [2] for a more
detailled explanation.
Another tip linked to your large image: you can cut the image into small
tiles and segment each tile separately. You might have some border
effects, but by using i.cutlines to define the tiles you can reduce
those (or by using overlaps in a rectangular tiling scheme). You can
parallelize the segmentation process in order to speed it up. We use
i.segment.uspo for that here (at the same time optimizing the threshold
segmentation parameter separately for each tile), but you can also use
i.segment.hierarchical or simply create a little python script using
GridModule (which creates rectangular tiles), i.e. something like this:
from grass.pygrass.modules.grid.grid import GridModule
import grass.script as gscript
# Create dictionary with the parameter arguments for i.segment
kwargs = {'group' : 'YourGroup',
'output' : 'YourOutputMap',
'threshold' : YourDesiredThreshold,
'minsize' : YourDesiredMinsize,
etc...
'quiet' : True}
# Define your desired gridding, overlap between grids and number of
processes
grd = GridModule('i.segment',
width=width,
height=height,
overlap=overlap,
processes=processes,
split=False,
**kwargs)
# Run the module on each of the grid cells
grd.run()
Moritz
[1] http://www.mdpi.com/2072-4292/9/4/358
[2] https://github.com/tgrippa/Opensource_OBIA_processing_chain
>
> Thanks,
>
> Paul Shapley
>
> On 21 June 2018 at 08:52, Markus Metz <markus.metz.giswork at gmail.com
> <mailto:markus.metz.giswork at gmail.com>> wrote:
>
> [please keep it in the list]
>
> On Thu, Jun 21, 2018 at 9:12 AM, Paul Shapley <p.shapley at gmail.com
> <mailto:p.shapley at gmail.com>> wrote:
> >
> > Thank you Markus...here are the results of g.region -p and r.info
> <http://r.info>
> >
> > g.region -p
> > projection: 99 (OSGB 1936 / British National Grid)
> > zone: 0
> > datum: osgb36
> > ellipsoid: airy
> > north: 242815
> > south: 200862.25
> > west: 263134.25
> > east: 298833.75
> > nsres: 0.25
> > ewres: 0.25
> > rows: 167811
> > cols: 142798
> > cells: 23963075178
> > (Thu Jun 21 08:03:51 2018) Command finished (0 sec)
> >
> > (Thu Jun 21 08:08:02 2018)
> > r.info <http://r.info> map=APGB_aerial_1.1 at PERMANENT
> >
> +----------------------------------------------------------------------------+
> > | Map: APGB_aerial_1.1 at PERMANENT Date: Tue May 29
> 09:11:31 2018 |
> > | Mapset: PERMANENT Login of Creator:
> administrator |
> > | Location: TempLocation
> |
> > | DataBase: C:\
> |
> > | Title: APGB_aerial_1.1
> |
> > | Timestamp: none
> |
> >
> |----------------------------------------------------------------------------|
> > |
> |
> > | Type of Map: raster Number of Categories: 0
> |
> > | Data Type: CELL
> |
> > | Rows: 167811
> |
> > | Columns: 142798
> |
> > | Total Cells: 23963075178
> |
> > | Projection: OSGB 1936 / British National Grid
> |
> > | N: 242815 S: 200862.25 Res: 0.25
> |
> > | E: 298833.75 W: 263134.25 Res: 0.25
> |
> > | Range of data: min = 0 max = 255
> |
> > |
> |
> > | Data Description:
> |
> > | generated by r.in.gdal
> |
> > |
> |
> > | Comments:
> |
> > | r.in.gdal -o -k input="J:\Spatial
> Data\Aerial\APGB_received_20180416\ |
> > | \source\RGB_25cm\output\test\JPEG.png"
> output="APGB_aerial_1" memory\ |
> > | =300 offset=0 num_digits=0
> |
> > |
> |
> >
> +----------------------------------------------------------------------------+
> > (Thu Jun 21 08:08:02 2018) Command finished (0 sec)
>
> OK, so you really have a group of rasters with 24 billion cells.
> Still, do you really need to run i.segment on the full extents with
> such a high resolution? If yes, try i.segment with memory=25000.
> Even if the system has more RAM, the operating system and any other
> programs also need some RAM, therefore it is safer to not use all
> free memory. Be aware that i.segment will take a long time (several
> days) to process such a large region.
>
> >
> > Should the output image from 'r.in.gdal' have had more memory
> allocated than the 300mb?
>
> The memory option of r.in.gdal applies to the input image, not the
> output image. Sometimes more memory can help, sometimes it does not
> have any effect. However, it does not affect subsequent processing
> steps.
>
> Markus M
>
> >
> > Thanks again Markus...I appreciate your help.
> >
> > Paul Shapley
> >
> >
> >
> >
> > On 20 June 2018 at 16:15, Markus Metz
> <markus.metz.giswork at gmail.com
> <mailto:markus.metz.giswork at gmail.com>> wrote:
> >>
> >>
> >>
> >> On Wed, Jun 20, 2018 at 1:30 PM, Paul Shapley
> <p.shapley at gmail.com <mailto:p.shapley at gmail.com>> wrote:
> >> >
> >> > Hi Users,
> >> >
> >> > Apologies if i'm posting this error again but i need to get
> pass the memory allocation issue of trying to allocate 3gb memory.
> The Grass version is (using file r.contour). Processor has 32 gb ram
> and i've allocated 30000 mb under memory option of 'i.segment'.
> >> >
> >> > System Info
> >> > GRASS version: 7.4.1
> >> > GRASS SVN revision: r72807
> >> > Build date: 2018-06-13
> >> > Build platform: i386-w64-mingw32
> >> > GDAL: 2.2.4
> >> > PROJ.4: 4.9.3
> >> > GEOS: 3.5.0
> >> > SQLite: 3.17.0
> >> > Python: 2.7.4
> >> > wxPython: 2.8.12.1
> >> > Platform: Windows-2008ServerR2-6.1.7601-SP1 (OSGeo4W)
> >> >
> >> >
> >> > ERROR: G_malloc: unable to allocate 2995426350 bytes of memory
> at imagery/i.segment/flag.c:25
> >>
> >> this error appears because all other memory has already been
> allocated, there is less than 3 GB free RAM left.
> >>
> >> It seems that the current region has more than 20 billion grid
> cells. Does this match the input to i.segment? You might want to
> check the current region settings and compare them with extents and
> resolution as reported by r.info <http://r.info> on the input to
> i.segment.
> >>
> >> Please provide the output of g.region -p and r.info
> <http://r.info> if you want more detailed help.
> >>
> >> Markus M
> >>
> >> >
> >> > --
> >> > Paul J. Shapley MSc CGeog (GIS) FRGS
> >> >
> >> >
> >> > _______________________________________________
> >> > grass-user mailing list
> >> > grass-user at lists.osgeo.org <mailto:grass-user at lists.osgeo.org>
> >> > https://lists.osgeo.org/mailman/listinfo/grass-user
> <https://lists.osgeo.org/mailman/listinfo/grass-user>
> >>
> >
> >
> >
> > --
> > Paul J. Shapley MSc CGeog (GIS) FRGS
> >
>
>
>
>
> --
> *Paul J. Shapley *MSc CGeog (GIS) FRGS
> *
> *
>
>
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-user
>
More information about the grass-user
mailing list