[GRASS-user] importing Mars DEM takes too long

[Helmut Kudrnovsky]

Anna Petrášová wrote
> On Sun, Nov 20, 2016 at 5:02 PM, Helmut Kudrnovsky <

> hellik@

> > wrote:
>>>11 seconds can be pretty long depending on the size of the file. Try
>>>to convert it to tif and load again, it will be probably much faster.
>>
>> the gdal_translated uncompressed geotif has about ~150MB (jp2 ~48 MB).
>>
>> the import takes about 7 seconds.
>>
>> there is some difference in importing time ; maybe cause of compressed
>> vs.
>> uncompressed.
>>
>> it's worth to ask on the GDAL ML about performance of GDAL'S jpeg2000
>> driver.
> 
> I just tested it on in wingrass both standalone and from OSGeo4W and
> it's very slow with ~50MB JP2 file. I tested this one:
> http://www.uahirise.org/PDS/DTM/ESP/ORB_020600_020699/ESP_020673_1750_ESP_020528_1750/ESP_020528_1750_RED_C_01_ORTHO.JP2
> 
> How could you import it in 11 s? Maybe the problem appears only with
> certain files?

tested your file with
------------------------------------------
System Info                                                                     
GRASS version: 7.3.svn                                                          
GRASS SVN revision: r69851                                                      
Build date: 2016-11-20                                                          
Build platform: x86_64-w64-mingw32                                              
GDAL: 2.1.2                                                                     
PROJ.4: 4.9.3                                                                   
GEOS: 3.5.0                                                                     
SQLite: 3.14.1                                                                  
Python: 2.7.5                                                                   
wxPython: 2.8.12.1                                                              
Platform: Windows-8-6.2.9200 (OSGeo4W) 
------------------------------------------
gdalinfo --formats | grep JPEG2000
  JP2ECW -raster,vector- (rov): ERDAS JPEG2000 (SDK 5.0)
  JP2MrSID -raster- (rov): MrSID JPEG2000
------------------------------------------
gdal_translate -co COMPRESS=LZW -co PREDICTOR=2
ESP_020528_1750_RED_C_01_ORTHO.JP2
ESP_020528_1750_RED_C_01_ORTHO_compressed.tif
Input file size is 7552, 19113
0...10...20...30...40...50...60...70...80...90...100 - done.
------------------------------------------
gdal_translate ESP_020528_1750_RED_C_01_ORTHO.JP2
ESP_020528_1750_RED_C_01_ORTHO_uncompressed.tif
Input file size is 7552, 19113
0...10...20...30...40...50...60...70...80...90...100 - done.
------------------------------------------
ls -l
total 268597
-rw-r--r-- 1  49609275 Nov 20 22:58 ESP_020528_1750_RED_C_01_ORTHO.JP2
-rw-r--r-- 1  80937523 Nov 20 23:00
ESP_020528_1750_RED_C_01_ORTHO_compressed.tif
-rw-r--r-- 1 144495106 Nov 20 23:01
ESP_020528_1750_RED_C_01_ORTHO_uncompressed.tif
------------------------------------------
r.import input=D:\dl\mars\ESP_020528_1750_RED_C_01_ORTHO.JP2
output=ESP_020528_1750_RED_C_01_ORTHO
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
Proceeding with import of 1 raster bands...
Importing raster map <ESP_020528_1750_RED_C_01_ORTHO>...
(Sun Nov 20 23:59:03 2016) Command finished (11 sec)  
------------------------------------------
r.import input=D:\dl\mars\ESP_020528_1750_RED_C_01_ORTHO_compressed.tif
output=ESP_020528_1750_RED_C_01_ORTHO_compressed
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
Proceeding with import of 1 raster bands...
Importing raster map <ESP_020528_1750_RED_C_01_ORTHO_compressed>...
(Mon Nov 21 00:04:39 2016) Command finished (6 sec)
------------------------------------------
r.import input=D:\dl\mars\ESP_020528_1750_RED_C_01_ORTHO_uncompressed.tif
output=ESP_020528_1750_RED_C_01_ORTHO_uncompressed
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
WARNING: Datum <unknown> not recognised by GRASS and no parameters found
Proceeding with import of 1 raster bands...
Importing raster map <ESP_020528_1750_RED_C_01_ORTHO_uncompressed>...
(Mon Nov 21 00:05:41 2016) Command finished (5 sec) 
------------------------------------------

no special tuning here on my side, just used the OSGeo4W settings on a 8GB
RAM win 10 box.

there are differences in file size (compressed vs. uncompresssed) and
importing time (also compressed vs. uncompressed).

just have a look at GDAL website regarding jpeg2000 driver implementations
(http://www.gdal.org/formats_list.html):

JPEG2000 (.jp2, .j2k) 	JPEG2000 	Yes 	Yes 	2GiB 	No, needs libjasper
JPEG2000 (.jp2, .j2k) 	JP2ECW 	Yes 	Yes 	500MB 	No, needs ECW SDK
JPEG2000 (.jp2, .j2k) 	JP2KAK 	Yes 	Yes 	No limits 	No, needs Kakadu library
JPEG2000 (.jp2, .j2k) 	JP2MrSID 	Yes 	Yes 		No, needs MrSID SDK
JPEG2000 (.jp2, .j2k) 	JP2OpenJPEG 	Yes 	Yes 		No, needs OpenJPEG library
(v2) 

OSGeo4W here on my side uses ERDAS JPEG2000 (SDK 5.0)/MrSID JPEG2000; thus
there may be perfomance issues between these two and JP2OpenJPEG?

still worth to ask on the GDAL ML.



-----
best regards
Helmut
--
View this message in context: http://osgeo-org.1560.x6.nabble.com/importing-Mars-DEM-takes-too-long-tp5296530p5296687.html
Sent from the Grass - Users mailing list archive at Nabble.com.