<div dir="ltr"><div>Hello folks.</div><div>This is my first post to the GDAL list :).
So I am running into an error using `Rasterio` 1.1.7 and `Gdal` 3.1.3 on
Windows and Mac, but not Linux. The code opens a raster GeoTIFF file
and compresses the data to some smaller format. Here is the code below
and then a picture of the error message. Note that the code is written
in Python 3.8 with `rasterio` but the error seems to come from the GDAL
call. <br></div><div><br></div><div>The error message is <code>rasterio.errors.RasterioIOError: Read or write failed. An error
occurred while writing a dirty block from GDALRasterBand::IRasterIO <font face="arial,sans-serif">. I googled around and found that I needed to ensure Libtiff > 4.0 is installed, which I did--but the error still persists. </font><br></code></div><div><br></div><div>One
thing to note is that the GeoTIFF is rather large. I am working with
aerial drone imagery for a philanthropic group in Tanzania, so the
images can be like 2-6GB even after compressed. Here is a sample image
that I use for testing <a href="https://github.com/00krishna-tools/nyuki/releases/download/v0.0.1/sample_image_big.tif" target="_blank">https://github.com/00krishna-tools/nyuki/releases/download/v0.0.1/sample_image_big.tif</a></div><div><br></div><div><pre><code>import rasterio
dat = 'sample_image_big.tif'
target_file = 'sample_image_big_compressed_jpeg.tif'
with rasterio.Env():
profile.update(
compress="JPEG",
BIGTIFF = "YES")
with rasterio.open(target_file, 'w', **profile) as dst:
for ji, window in dat.block_windows(1):
dst.write(dat.read(window=window), window=window)<br><br></code></pre><pre><code><font face="arial,sans-serif">I am using "windowed" reads and writes, etc., so that seems to be the most efficient way to do it. <br><br></font></code></pre><pre><code><font face="arial,sans-serif">Here is a picture of the error with traceback:<br><br><span style="font-family:monospace">Traceback (most recent call last):<br style="box-sizing:inherit"> File "rasterio/_io.pyx", line 1369, in rasterio._io.DatasetWriterBase.write<br style="box-sizing:inherit"> File "rasterio/shim_rasterioex.pxi", line 133, in rasterio._shim.io_multi_band<br style="box-sizing:inherit"> File "rasterio/_err.pyx", line 182, in rasterio._err.exc_wrap_int<br style="box-sizing:inherit">rasterio._err.CPLE_AppDefinedError: An error occurred while writing a dirty block from GDALRasterBand::IRasterIO<span class="gmail-c-mrkdwn__br" style="box-sizing:inherit;display:block;height:unset"></span>During handling of the above exception, another exception occurred:<span class="gmail-c-mrkdwn__br" style="box-sizing:inherit;display:block;height:unset"></span>Traceback (most recent call last):<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/bin/nyuki", line 10, in <module><br style="box-sizing:inherit"> sys.exit(nyuki())<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/click/core.py", line 829, in __call__<br style="box-sizing:inherit"> return self.main(*args, **kwargs)<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/click/core.py", line 782, in main<br style="box-sizing:inherit"> rv = self.invoke(ctx)<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke<br style="box-sizing:inherit"> return _process_result(sub_ctx.command.invoke(sub_ctx))<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/click/core.py", line 1259, in invoke<br style="box-sizing:inherit"> return _process_result(sub_ctx.command.invoke(sub_ctx))<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/click/core.py", line 1066, in invoke<br style="box-sizing:inherit"> return ctx.invoke(self.callback, **ctx.params)<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/click/core.py", line 610, in invoke<br style="box-sizing:inherit"> return callback(*args, **kwargs)<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/nyuki/nyuki.py", line 160, in compress<br style="box-sizing:inherit"> compressor(sourcetiff, target_compression, yes)<br style="box-sizing:inherit"> File "/Users/user/mdconda/anaconda3/envs/nyuki-env/lib/python3.7/site-packages/nyuki/geotiff_compressor.py", line 52, in compressor<br style="box-sizing:inherit"> dst.write(dat.read(window=window), window=window)<br style="box-sizing:inherit"> File "rasterio/_io.pyx", line 1371, in rasterio._io.DatasetWriterBase.write<br style="box-sizing:inherit">rasterio.errors.RasterioIOError: Read or write failed. An error occurred while writing a dirty block from GDALRasterBand::IRasterIO</span></font></code></pre><pre><code><font face="arial,sans-serif">So the real question is whether there is a workaround for this issue, or is there something different<br></font></code></pre><pre><code><font face="arial,sans-serif">I should do with my code? Any suggestions for getting this code to work on Mac and Windows, as it<br></font></code></pre><pre><code><font face="arial,sans-serif">works just fine on Linux. Thanks so much for any insights. <font color="#888888"><br></font></font></code></pre></div></div>