I've been in contact with Christian the author of gdal_retile and he told me this problem is in the main GDAL library.<br>He also suggested using some of the other GDAL tools to see if the problem persists.<br><br>So I started using gdal_translate to copy my ecw to a new ecw using this statement:<br>
gdal_translate -a_srs EPSG:28992 -of ECW -co "LARGE_OK=YES" large.ecw Large_translate.ecw<br><br>After 6 hours (!) I got a new ecw file, this time 4.2GB with this info:<br>Driver: ECW/ERMapper Compressed Wavelets<br>
Files: Large_translate.ecw<br>Size is 173334, 160000<br>Coordinate System is:<br>PROJCS["NUTM31",<br> GEOGCS["RIJKDRIEHOEKSMETING",<br> DATUM["RD",<br> SPHEROID["BESS1841",6377397.155,299.1528128]],<br>
PRIMEM["Greenwich",0],<br> UNIT["degree",0.0174532925199433]],<br> PROJECTION["Transverse_Mercator"],<br> PARAMETER["latitude_of_origin",0],<br> PARAMETER["central_meridian",3],<br>
PARAMETER["scale_factor",0.9996],<br> PARAMETER["false_easting",500000],<br> PARAMETER["false_northing",0],<br> UNIT["Meter",1]]<br>Origin = (33000.000000000000000,381000.000000000000000)<br>
Pixel Size = (0.150000000001455,-0.150000000023283)<br>Corner Coordinates:<br>Upper Left ( 33000.000, 381000.000) ( 1d12'3.77"W, 3d26'16.85"N)<br>Lower Left ( 33000.000, 357000.000) ( 1d12'0.46"W, 3d13'17.23"N)<br>
Upper Right ( 59000.100, 381000.000) ( 0d58'3.16"W, 3d26'20.47"N)<br>Lower Right ( 59000.100, 357000.000) ( 0d58'0.03"W, 3d13'20.61"N)<br>Center ( 46000.050, 369000.000) ( 1d 5'1.86"W, 3d19'48.81"N)<br>
Band 1 Block=173334x1 Type=Byte, ColorInterp=Red<br> Overviews: arbitrary<br>Band 2 Block=173334x1 Type=Byte, ColorInterp=Green<br> Overviews: arbitrary<br>Band 3 Block=173334x1 Type=Byte, ColorInterp=Blue<br> Overviews: arbitrary<br>
<br>I tried to retile this ecw file using this command:<br>gdal_retile -v -s_srs EPSG:28992 -of ECW -ps 17335 16000 -targetDir tiles Large_translate.ecw >retile.txt<br>But got the same memory error.<br><br>Next I tried converting to GeoTiff using this command:<br>
gdal_translate -a_srs EPSG:28992 -of GTiff Large.ecw Large.tiff<br>After 2.5 hours I got a 77.5GB Tiff file with this info:<br>Driver: GTiff/GeoTIFF<br>Files: Large.tiff<br>Size is 173334, 160000<br>Coordinate System is:<br>
PROJCS["Amersfoort / RD New",<br> GEOGCS["Amersfoort",<br> DATUM["Amersfoort",<br> SPHEROID["Bessel 1841",6377397.155,299.1528128000009,<br> AUTHORITY["EPSG","7004"]],<br>
AUTHORITY["EPSG","6289"]],<br> PRIMEM["Greenwich",0],<br> UNIT["degree",0.0174532925199433],<br> AUTHORITY["EPSG","4289"]],<br> PROJECTION["Oblique_Stereographic"],<br>
PARAMETER["latitude_of_origin",52.15616055555555],<br> PARAMETER["central_meridian",5.387638888888889],<br> PARAMETER["scale_factor",0.9999079],<br> PARAMETER["false_easting",155000],<br>
PARAMETER["false_northing",463000],<br> UNIT["metre",1,<br> AUTHORITY["EPSG","9001"]],<br> AUTHORITY["EPSG","28992"]]<br>Origin = (33000.000000000000000,381000.000000000000000)<br>
Pixel Size = (0.150000000001455,-0.150000000023283)<br>Metadata:<br> AREA_OR_POINT=Area<br>Image Structure Metadata:<br> INTERLEAVE=PIXEL<br>Corner Coordinates:<br>Upper Left ( 33000.000, 381000.000) ( 3d38'2.16"E, 51d24'21.02"N)<br>
Lower Left ( 33000.000, 357000.000) ( 3d38'31.86"E, 51d11'24.60"N)<br>Upper Right ( 59000.100, 381000.000) ( 4d 0'27.21"E, 51d24'39.10"N)<br>Lower Right ( 59000.100, 357000.000) ( 4d 0'50.58"E, 51d11'42.56"N)<br>
Center ( 46000.050, 369000.000) ( 3d49'27.95"E, 51d18'2.36"N)<br>Band 1 Block=173334x1 Type=Byte, ColorInterp=Red<br>Band 2 Block=173334x1 Type=Byte, ColorInterp=Green<br>Band 3 Block=173334x1 Type=Byte, ColorInterp=Blue<br>
<br>When I try to retile this tiff using:<br>gdal_retile -v -s_srs EPSG:28992 -of GTiff -ps 17335 16000 -targetDir tiles Large.tiff >retile.txt<br>I got again the memory error.<br><br>I don't know anymore how to to this. How can I split my large ecw file into smaller tiles?<br>
<br>I also noticed that even if I use the same EPSG code I get different projection info, even differences when using only gdal_translate.<br>The projection is the same, they only have different naming. Why is that?<br><br>
I also noticed both gdal_translate and gdal_retile not use the full potential of my hardware. I'm running Vista 64Bit on a AMD3 2.6Ghz quadcore with 8GB RAM and only 2-2.5GB RAM was used by gdal_translate, leaving more than 5 GB free and only 35-55% of my CPU was used. Do I need to set some settings to speed things up?<br>
<br>Thanks,<br><br>Paul<br><br><br><br>From: Paul Meems <bontepaarden <at> <a href="http://gmail.com">gmail.com</a>><br>
Subject: <a target="_top" rel="nofollow" href="http://news.gmane.org/find-root.php?message_id=%3c870e056a0908101422x5f38bb6fh32c2bd848ff6c868%40mail.gmail.com%3e">Memory Error in _gdal.Band_ReadRaster(*args, **kwargs)</a><br>
Newsgroups: <a href="http://news.gmane.org/gmane.comp.gis.gdal.devel" target="_top">gmane.comp.gis.gdal.devel</a><br>
Date: 2009-08-10 21:22:06 GMT<br>
<p>
</p>I'm trying to split a large ECW file into smaller files.<br>I've downloaded and installed OSGeo4W with gdal16<br>I managed to split a small ecw file.<br>That ecw file had this info:<br><br>Driver: ECW/ERMapper Compressed Wavelets<br>
Files: tilertest.ecw<br>Size is 6800, 7500<br>Coordinate System is `'<br>Origin = (13000.000000000000000,381250.000000000000000)<br>Pixel Size = (2.500000000000000,-2.500000000000000)<br>Corner Coordinates:<br>Upper Left ( 13000.000, 381250.000) <br>
Lower Left ( 13000.000, 362500.000) <br>Upper Right ( 30000.000, 381250.000) <br>Lower Right ( 30000.000, 362500.000) <br>Center ( 21500.000, 371875.000) <br>Band 1 Block=6800x1 Type=Byte, ColorInterp=Red<br>
Overviews: arbitrary<br>Band 2 Block=6800x1 Type=Byte, ColorInterp=Green<br> Overviews: arbitrary<br>Band 3 Block=6800x1 Type=Byte, ColorInterp=Blue<br> Overviews: arbitrary<br><br>I used this statement:<br>gdal_retile -v -s_srs EPSG:28992 -of ECW -ps 3700 3750 -targetDir tiles TilerTest.ecw<br>
<br>But when I try to split the large ecw file (3.4GB) I get an error after a few minutes.<br>The info is:<br>Driver: ECW/ERMapper Compressed Wavelets<br>Files: Large.ecw<br>Size is 173334, 160000<br>Coordinate System is:<br>
LOCAL_CS["LOCAL - (unsupported)",<br> UNIT["Meter",1]]<br>Origin = (33000.000000000000000,381000.000000000000000)<br>Pixel Size = (0.150000000001455,-0.150000000023283)<br>Corner Coordinates:<br>Upper Left ( 33000.000, 381000.000) <br>
Lower Left ( 33000.000, 357000.000) <br>Upper Right ( 59000.100, 381000.000) <br>Lower Right ( 59000.100, 357000.000) <br>Center ( 46000.050, 369000.000) <br>Band 1 Block=173334x1 Type=Byte, ColorInterp=Red<br>
Overviews: arbitrary<br>Band 2 Block=173334x1 Type=Byte, ColorInterp=Green<br> Overviews: arbitrary<br>Band 3 Block=173334x1 Type=Byte, ColorInterp=Blue<br> Overviews: arbitrary<br><br>The statement is:<br>gdal_retile -v -s_srs EPSG:28992 -of ECW -ps 86667 80000 -targetDir tiles large.ecw >retile.txt<br>
<br>The logfile (retile.txt) shows:<br>Building internal Index for 1 tile(s) ... finished<br>Filename: Terneuzen_15cm.ecw<br>File Size: 173334x160000x3<br>Pixel Size: 0.150000 x -0.150000<br>UL:(33000.000000,381000.000000) LR:(59000.100000,356999.999996)<br>
tileWidth 86667<br>tileHeight 80000<br>countTilesX: 2<br>countTilesY: 2<br>lastTileWidth: 86667<br>lastTileHeight: 80000<br><br>And the error is:<br>Traceback (most recent call last):<br> File "C:\OSGeo4W\apps\gdal-16\bin\gdal_retile.py", line 938, in <module> sys.exit(main(sys.argv))<br>
File "C:\OSGeo4W\apps\gdal-16\bin\gdal_retile.py", line 855, in main dsCreatedTileIndex = tileImage(minfo,ti)<br>
File "C:\OSGeo4W\apps\gdal-16\bin\gdal_retile.py", line 335, in
tileImage createTile(minfo, offsetX, offsetY, width,
height,tilename,OGRDS)<br>
File "C:\OSGeo4W\apps\gdal-16\bin\gdal_retile.py", line 467, in createTile dec.uly+offsetY*dec.scaleY)<br>
File "C:\OSGeo4W\apps\gdal-16\bin\gdal_retile.py", line 250, in
getDataSet data = s_band.ReadRaster(
readOffsetX,readOffsetY,readX,readY, readX,readY,self.band_type )<br>
File "C:\OSGeo4W\apps\gdal-16\pymod\osgeo\gdal.py", line 760, in
ReadRaster return _gdal.Band_ReadRaster(*args, **kwargs) MemoryError<br><br>I've searched using Google and found a similar error: <br><a href="http://article.gmane.org/gmane.comp.gis.gdal.devel/18936">http://article.gmane.org/gmane.comp.gis.gdal.devel/18936</a> and contacted Sarah, but she's still having the problem.<br>
<br>Does anybody know how to solve this?<br><br>I run it on a Vista 64-Bit 4-core, with 8GB RAM. I'm getting the same error on a WinXP sp3 2-core, with 2 GB RAM<br><br>Thanks,<br><br>Paul Meems<br>The Netherlands