[GRASS-dev] r.sun or libgis memory leak?
Markus Neteler
neteler at osgeo.org
Wed Jan 28 06:38:49 EST 2009
I am running a full year cycle of r.sun (the new version, in
6.4.0svn), with all possible output
("for" loop in shell over all days of the year). My test runs show
some (for me) strange
memory consumption:
top - 16:49:47 up 1 day, 4:25, 1 user, load average: 2.77, 2.81, 2.90
Tasks: 250 total, 4 running, 246 sleeping, 0 stopped, 0 zombie
Cpu(s): 18.6%us, 0.1%sy, 0.0%ni, 80.8%id, 0.0%wa, 0.0%hi, 0.5%si, 0.0%st
Mem: 66070020k total, 65966356k used, 103664k free, 59912k buffers
Swap: 31249992k total, 31332k used, 31218660k free, 38135360k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
27806 neteler 25 0 8937m 8.7g 2884 R 100 13.8 140:57.22 r.sun
28966 neteler 25 0 8937m 8.7g 2880 R 100 13.8 125:49.61 r.sun
30021 neteler 25 0 8937m 8.7g 2880 R 100 13.8 112:45.34 r.sun
14029 partimag 15 0 112m 1196 960 S 6 0.0 106:07.05 gkrellmd
3298 neteler 15 0 12720 1180 800 R 1 0.0 0:00.51 top
1 root 15 0 10328 700 588 S 0 0.0 0:02.96 init
2 root RT -5 0 0 0 S 0 0.0 0:00.12 migration/0
3 root 34 19 0 0 0 S 0 0.0 0:03.37 ksoftirqd/0
4 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/0
5 root RT -5 0 0 0 S 0 0.0 0:00.08 migration/1
6 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1
7 root RT -5 0 0 0 S 0 0.0 0:00.00 watchdog/1
8 root RT -5 0 0 0 S 0 0.0 0:00.04 migration/2
Each r.sun job takes 8.9GB RAM to my understanding - but the overall consumption
is 65GB! I see it slowly growing and finally crashing the blade.
Is it possible that r.sun or a libgis function doesn't release all memory?
I have run valgrind on it (Spearfish60, see below) but it doesn't tell
me too much.
Markus
------------ snip --------------
# Spearfish60
g.region rast=elevation.dem -p
projection: 1 (UTM)
zone: 13
datum: nad27
ellipsoid: clark66
north: 4928000
south: 4914020
west: 590010
east: 609000
nsres: 30
ewres: 30
rows: 466
cols: 633
cells: 294978
r.horizon elevin=elevation.dem horizonstep=30 bufferzone=200
horizon=horangle dist=0.7 maxdistance=2000
100%
r.slope.aspect elevation=elevation.dem aspect=aspect.dem slope=slope.dem
100%
Aspect raster map <aspect.dem> complete
Slope raster map <slope.dem> complete
CMD="r.sun -s elevation.dem horizon=horangle horizonstep=30
aspin=aspect.dem slopein=slope.dem glob_rad=global_rad day=180
time=14"
valgrind --tool=memcheck --leak-check=yes --show-reachable=yes $CMD --o
==4353== Memcheck, a memory error detector.
==4353== Copyright (C) 2002-2007, and GNU GPL'd, by Julian Seward et
al.
==4353== Using LibVEX rev 1854, a library for dynamic binary
translation.
==4353== Copyright (C) 2004-2007, and GNU GPL'd, by OpenWorks LLP.
==4353== Using valgrind-3.3.1, a dynamic binary instrumentation
framework.
==4353== Copyright (C) 2000-2007, and GNU GPL'd, by Julian Seward et
al.
==4353== For more details, rerun with: -v
==4353==
Mode 1: instantaneous solar incidence angle & irradiance using a set
local
time
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x405CD6: INPUT_part (main.c:1126)
==4353== by 0x40795D: calculate (main.c:1841)
==4353== by 0x404753: main (main.c:746)
100%
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB4E: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB5B: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB71: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB88: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB1A: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB27: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB34: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB41: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABAB6: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABAC1: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABB0D: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== Conditional jump or move depends on uninitialised value(s)
==4353== at 0x54ABAF8: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AC49C: (within /lib64/libz.so.1.2.3)
==4353== by 0x54AD06F: deflate (in /lib64/libz.so.1.2.3)
==4353== by 0x5053758: G_zlib_compress (flate.c:339)
==4353== by 0x505342C: G_zlib_write (flate.c:213)
==4353== by 0x506F990: G__write_data_compressed (put_row.c:337)
==4353== by 0x506FE82: put_fp_data (put_row.c:468)
==4353== by 0x5070615: put_raster_data (put_row.c:695)
==4353== by 0x5070F2D: put_raster_row (put_row.c:909)
==4353== by 0x506F573: G_put_raster_row (put_row.c:225)
==4353== by 0x506F5B7: G_put_f_raster_row (put_row.c:235)
==4353== by 0x406698: OUTGR (main.c:1247)
==4353==
==4353== ERROR SUMMARY: 34207 errors from 13 contexts (suppressed: 2
from 1)
==4353== malloc/free: in use at exit: 8,341,575 bytes in 3,023 blocks.
==4353== malloc/free: 29,973 allocs, 26,950 frees, 209,667,941 bytes
allocated.
==4353== For counts of detected errors, rerun with: -v
==4353== searching for pointers to 3,023 not-freed blocks.
==4353== checked 10,687,088 bytes.
==4353==
==4353==
==4353== 993 bytes in 13 blocks are still reachable in loss record 1
of 6
==4353== at 0x4C2136E: malloc (vg_replace_malloc.c:207)
==4353== by 0x56EA7E7: pj_malloc (pj_malloc.c:20)
==4353== by 0x56EB516: pj_mkparam (pj_param.c:12)
==4353== by 0x56E9CD1: pj_init (pj_init.c:258)
==4353== by 0x4E27DE5: pj_get_kv (get_proj.c:228)
==4353== by 0x4045E3: main (main.c:725)
==4353==
==4353==
==4353== 2,536 bytes in 1 blocks are definitely lost in loss record 2
of 6
==4353== at 0x4C1F144: calloc (vg_replace_malloc.c:397)
==4353== by 0x504211C: G__calloc (alloc.c:74)
==4353== by 0x50422F8: G_allocate_f_raster_buf (alloc_cell.c:113)
==4353== by 0x406381: OUTGR (main.c:1201)
==4353== by 0x404758: main (main.c:747)
==4353==
==4353==
==4353== 4,760 bytes in 2 blocks are still reachable in loss record 3
of 6
==4353== at 0x4C1F144: calloc (vg_replace_malloc.c:397)
==4353== by 0x504211C: G__calloc (alloc.c:74)
==4353== by 0x506B8A8: G_recreate_command (parser.c:2804)
==4353== by 0x505C042: G_command_history (history.c:259)
==4353== by 0x408DE1: calculate (main.c:2102)
==4353== by 0x404753: main (main.c:746)
==4353==
==4353==
==4353== 6,326 bytes in 1,007 blocks are definitely lost in loss
record 4 of 6
==4353== at 0x4C2136E: malloc (vg_replace_malloc.c:207)
==4353== by 0x504209E: G__malloc (alloc.c:41)
==4353== by 0x507AA95: G_store (store.c:36)
==4353== by 0x5053005: find_file (find_file.c:101)
==4353== by 0x50530BF: find_file1 (find_file.c:126)
==4353== by 0x5053156: G_find_file_misc (find_file.c:167)
==4353== by 0x5059A3F: open_null_read (get_row.c:995)
==4353== by 0x5059CA4: get_null_value_row_nomask (get_row.c:1063)
==4353== by 0x505A1CD: get_null_value_row (get_row.c:1186)
==4353== by 0x505A285: embed_nulls (get_row.c:1204)
==4353== by 0x5059633: get_map_row_no_reclass (get_row.c:650)
==4353== by 0x5059710: get_map_row (get_row.c:678)
==4353==
==4353==
==4353== 39,674 bytes in 5 blocks are still reachable in loss record 5 of 6
==4353== at 0x4C214A1: realloc (vg_replace_malloc.c:429)
==4353== by 0x50421A8: G__realloc (alloc.c:111)
==4353== by 0x5050F12: set_env (env.c:156)
==4353== by 0x5050CA4: read_env (env.c:104)
==4353== by 0x505152E: G__getenv (env.c:317)
==4353== by 0x5051470: G_getenv (env.c:271)
==4353== by 0x505F704: G_location (location.c:63)
==4353== by 0x505F71C: G__location_path (location.c:78)
==4353== by 0x505F6A8: G_location_path (location.c:41)
==4353== by 0x505AB6B: G__gisinit (gisinit.c:57)
==4353== by 0x402F3B: main (main.c:223)
==4353==
==4353==
==4353== 8,287,286 bytes in 1,995 blocks are still reachable in loss
record 6 of 6
==4353== at 0x4C2136E: malloc (vg_replace_malloc.c:207)
==4353== by 0x504209E: G__malloc (alloc.c:41)
==4353== by 0x507AA95: G_store (store.c:36)
==4353== by 0x506EB45: G_set_program_name (progrm_nme.c:52)
==4353== by 0x505AB32: G__gisinit (gisinit.c:51)
==4353== by 0x402F3B: main (main.c:223)
==4353==
==4353== LEAK SUMMARY:
==4353== definitely lost: 8,862 bytes in 1,008 blocks.
==4353== possibly lost: 0 bytes in 0 blocks.
==4353== still reachable: 8,332,713 bytes in 2,015 blocks.
==4353== suppressed: 0 bytes in 0 blocks.
More information about the grass-dev
mailing list