[GRASS-dev] [GRASS GIS] #3365: r.in.gdal: crash on large dataset
GRASS GIS
trac at osgeo.org
Mon Jul 3 11:06:57 PDT 2017
#3365: r.in.gdal: crash on large dataset
-----------------------+---------------------------------
Reporter: neteler | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.2.2
Component: Raster | Version: svn-releasebranch72
Keywords: r.in.gdal | CPU: x86-64
Platform: Linux |
-----------------------+---------------------------------
Trying to import a large VRT dataset, I get a crash:
{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > r.in.gdal
input=/mnt/kvm7-storage/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
Segmentation fault (core dumped)
}}}
Debugging:
{{{
GRASS 7.2.2svn (latlong):~/software/grass72_svn_debugmode/bin.x86_64-pc-
linux-gnu > gdb r.in.gdal
GNU gdb (GDB) Fedora 7.12.1-48.fc25
Copyright (C) 2017 Free Software Foundation, Inc.
[...]
Reading symbols from r.in.gdal...done.
(gdb) r
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Starting program:
/home/mundialis/software/grass72_svn_debugmode/dist.x86_64-pc-linux-
gnu/bin/r.in.gdal
input=/mnt//GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Missing separate debuginfos, use: dnf debuginfo-install
glibc-2.24-8.fc25.x86_64
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Proceeding with import of 1 raster bands...
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
370 compressed_buf[0] = work_buf[0] = nbytes;
(gdb) bt full
#0 0x00007ffff77a1ab9 in put_data (fd=0, null_buf=0x7fffffe9b020 "",
cell=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0) at put_row.c:370
wk = 0x7fffff91cb31 ""
nbytes = 1
compressed_buf = 0x7fffff7bd220 <error: Cannot access memory at
address 0x7fffff7bd220>
total = 1440000
fcb = 0x691080
compressed = 1
len = 4
work_buf = 0x7fffff91cb30 "\001"
wk = 0x7fffff91cb31 ""
nwrite = 0
#1 0x00007ffff77a1f98 in put_raster_data (fd=0, null_buf=0x7fffffe9b020
"", rast=0x7fffc62ac010, row=0, n=1440000, zeros_r_nulls=0, map_type=0)
at put_row.c:476
fcb = 0x691080
#2 0x00007ffff77a2bdb in put_raster_row (fd=0, buf=0x7fffc62ac010,
data_type=0, zeros_r_nulls=0) at put_row.c:699
convert_and_write_FtypeOtype = {{0x0, 0x7ffff77a23d9
<convert_and_write_if>, 0x7ffff77a25f5 <convert_and_write_id>}, {
0x7ffff77a280f <convert_and_write_fi>, 0x0, 0x7ffff77a26f3
<convert_and_write_fd>}, {0x7ffff77a2928 <convert_and_write_di>,
0x7ffff77a24d7 <convert_and_write_df>, 0x0}}
fcb = 0x691080
null_buf = 0x7fffffe9b020 ""
#3 0x00007ffff77a0f8b in Rast_put_row (fd=0, buf=0x7fffc62ac010,
data_type=0) at put_row.c:67
No locals.
#4 0x0000000000406e4d in ImportBand (hBand=0x7dbcb0, output=0x627dc0
"loss_global_forest_cover_loss_2000_2015", group_ref=0x0) at main.c:1141
data_type = 0
eGDT = GDT_Int32
eRawGDT = GDT_Byte
row = 1
nrows = 560000
ncols = 1440000
complex = 0
cf = 0
cfR = 32767
cfI = -145625496
bNoDataEnabled = 0
indx = 32767
cell = 0x7fffc62ac010
cellReal = 0x77
cellImg = 0x7fffffffac40
bufComplex = 0x7fffffffab40
dfNoData = -10000
outputReal =
"0\341\377\367\377\177\000\000\260\254\377\377\377\177\000\000\071\031@\000\000\000\000\000\243\366\251\242\000\000\000\000\377\377\377\377",
'\000' <repeats 12 times>,
"H\017\064\366\377\177\000\000\000\240\374\367\377\177\000\000\000\000\000\000\000\000\360?",
'\000' <repeats 14 times>, "\360?", '\000' <repeats 30 times>,
"\360?\000\000\000\000\000\000\000\000\220\062@\000\000\000\000\000\314\345w\367\377\177\000\000P\255\377\377\377\177\000\000\350\262`\000\000\000\000\000\220\062@\000\000\000\000\000\320\332\377\377\377\177",
'\000' <repeats 18 times>, "\360\331\377\377\377\177\000\000"...
outputImg = '\000' <repeats 32 times>,
"\377\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\000\000\000\000\000\000\377\377\377\377\377\377\000\377\377\377",
'\000' <repeats 22 times>, "\360?", '\000' <repeats 14 times>, "\360?",
'\000' <repeats 14 times>,
"\360?\000\000\000\000\000\000\000\000\210\345w\367\377\177", '\000'
<repeats 26 times>,
"\001\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\200\256\377\377\377\177\000\000\240\224z\367\377\177\000\000\000\273}",
'\000' <repeats 13 times>...
nullFlags = 0x0
history = {fields = {0x7ffff7fca000 "", 0x7fffffffabe8 "",
0x7fffffffabe4 "", 0x7ffff7ff7b78 "\350\343\377\367\377\177",
0x7fffffffaba0 "\300\254\377\377\377\177", 0x401939
"GDALGetRasterBand", 0x400be8 "\261\a", 0x7fffffffabe8 ""}, nlines =
-1565919581,
lines = 0x28aa7da}
GDALmetadata = 0x7ffff77aa094
have_colors = 0
gdal_rat = 0x7ffff752f25f <find_file1+198>
#5 0x0000000000405231 in main (argc=3, argv=0x7fffffffdad8) at main.c:619
nBand = 1
input = 0x627d20
"/mnt/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015.vrt"
output = 0x627dc0 "loss_global_forest_cover_loss_2000_2015"
title = 0x0
cellhd = {format = 0, compressed = 0, rows = 560000, rows3 =
560000, cols = 1440000, cols3 = 1440000, depths = 1, proj = 3, zone = 0,
ew_res = 0.00025000000000000017, ew_res3 =
0.00025000000000000017, ns_res = 0.00025000000000000017, ns_res3 =
0.00025000000000000017, tb_res = 1,
north = 80, south = -60.000000000000085, east =
180.00000000000023, west = -180, top = 1, bottom = 0}
loc_wind = {format = 0, compressed = -1, rows = 1, rows3 = 1, cols
= 1, cols3 = 1, depths = 1, proj = 3, zone = 0, ew_res = 1, ew_res3 = 1,
ns_res = 1, ns_res3 = 1, tb_res = 1, north = 1, south = 0, east
= 1, west = 0, top = 1, bottom = 0}
cur_wind = {format = 0, compressed = 0, rows = 0, rows3 = 0, cols
= 0, cols3 = 0, depths = 0, proj = 0, zone = 0, ew_res = 0, ew_res3 = 0,
ns_res = 0, ns_res3 = 0, tb_res = 0, north = 0, south = 0, east
= 0, west = 0, top = 0, bottom = 0}
proj_info = 0x68ced0
proj_units = 0x68cbf0
loc_proj_info = 0x68e5f0
loc_proj_units = 0x7db4f0
hDS = 0x7dbb00
hDriver = 0x62ef90
hBand = 0x7dbcb0
adfGeoTransform = {-180, 0.00025000000000000017, 0, 80, 0,
-0.00025000000000000017}
n_bands = 0
force_imagery = 0
error_msg = "\253\000\000\000\022", '\000' <repeats 19 times>,
"i\000\000\000\022", '\000' <repeats 19 times>, "\222\000\000\000\022",
'\000' <repeats 19 times>, "\231\000\000\000\022", '\000' <repeats 19
times>, "\001\000\000\000
\000\000\000@\323\377\377\377\177\000\000\260\327\377\377\377\177\000\000\350
Y\372\367\377\177\000\000\060U\372\367\377\177\000\000\001\000\000\000\000\000\000\000\235\066\313\335\377\177\000\000\260Q\336\367\377\177\000\000\000\000\000\000\000\000\000\000D|\336\367\377\177\000\000`O\372\367\377\177\000\000\210\337\377\367\377\177\000\000\000\000\000\000\000\000\000\000"...
projcomp_error = 1
overwrite = 0
offset = 0
suffix = 0x629130 ""
num_digits = 0
module = 0x7ffff777ea28 <state+40>
parm = {input = 0x7ffff777ea88 <state+136>, output = 0x626ea0,
target = 0x6270e0, title = 0x6271a0, outloc = 0x6274a0, band = 0x626f60,
memory = 0x627020, offset = 0x627260, num_digits = 0x627320,
map_names_file = 0x6273e0, rat = 0x627560}
flag_o = 0x7ffff777ea58 <state+88>
flag_e = 0x626840
flag_k = 0x627ab0
flag_f = 0x627a10
flag_l = 0x627a60
flag_c = 0x627b00
flag_p = 0x627b50
flag_j = 0x626550
(gdb)
}}}
Doing a convert operation with GDAL, no apparent issues (I CTRL-C'd it
then):
{{{
gdal_translate
/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
/scratch/bla.tif
Input file size is 1440000, 560000
0..
}}}
**Data source:**
http://earthenginepartners.appspot.com/science-2013-global-
forest/download_v1.3.html
I created a VRT from the "loss" layer tiles which I then tried to import
as above:
{{{
gdalbuildvrt loss_global_forest_cover_loss_2000_2015.vrt *.tif
gdalinfo loss_global_forest_cover_loss_2000_2015.vrt
Driver: VRT/Virtual Raster
[...]
Size is 1440000, 560000
Coordinate System is:
GEOGCS["WGS 84",
DATUM["WGS_1984",
SPHEROID["WGS 84",6378137,298.257223563,
AUTHORITY["EPSG","7030"]],
AUTHORITY["EPSG","6326"]],
PRIMEM["Greenwich",0],
UNIT["degree",0.0174532925199433],
AUTHORITY["EPSG","4326"]]
Origin = (-180.000000000000000,80.000000000000000)
Pixel Size = (0.000250000000000,-0.000250000000000)
Corner Coordinates:
Upper Left (-180.0000000, 80.0000000) (180d 0' 0.00"W, 80d 0' 0.00"N)
Lower Left (-180.0000000, -60.0000000) (180d 0' 0.00"W, 60d 0' 0.00"S)
Upper Right ( 180.0000000, 80.0000000) (180d 0' 0.00"E, 80d 0' 0.00"N)
Lower Right ( 180.0000000, -60.0000000) (180d 0' 0.00"E, 60d 0' 0.00"S)
Center ( 0.0000000, 10.0000000) ( 0d 0' 0.00"E, 10d 0' 0.00"N)
Band 1 Block=128x128 Type=Byte, ColorInterp=Gray
}}}
Test with GRASS GIS 7.3:
{{{
GRASS 7.3.svn (latlong):~/software/grass73_svn > r.in.gdal
input=/mnt/geodata/global_forest/GFC-2015-v1.3/loss_global_forest_cover_loss_2000_2015/loss_global_forest_cover_loss_2000_2015.vrt
output=loss_global_forest_cover_loss_2000_2015
Importing raster map <loss_global_forest_cover_loss_2000_2015>...
0% Segmentation fault (core dumped)
}}}
System:
{{{
gcc -v
...
gcc version 6.3.1 20161221 (Red Hat 6.3.1-1) (GCC)
uname -a
Linux fedora-calc 4.9.12-200.fc25.x86_64 #1 SMP Thu Feb 23 19:31:49 UTC
2017 x86_64 x86_64 x86_64 GNU/Linux
}}}
--
Ticket URL: <https://trac.osgeo.org/grass/ticket/3365>
GRASS GIS <https://grass.osgeo.org>
More information about the grass-dev
mailing list