<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>Margherita</div>
<div> </div>
<div>thanks, now that you say this, I remeber a special GDAL option "GDAL_MAX_DATASET_POOL_SIZE" which by default is 100. The operating system limit itself is high enough on our server (65k). When running the processing again with less memory cache (and hence likely less concurrent opened files) it worked without errors. I will set this GDAL option and see if I ever run into this errors again.</div>
<div> </div>
<div>armin</div>
<div>
<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"><b>Gesendet:</b> Mittwoch, 21. Dezember 2016 um 07:29 Uhr<br/>
<b>Von:</b> "Margherita Di Leo" <diregola@gmail.com><br/>
<b>An:</b> armin.burger@gmx.net<br/>
<b>Betreff:</b> Re: [gdal-dev] Read error on large VRT files</div>
<div name="quoted-content">
<div>Hey Armin,</div>
<div> </div>
<div>Could it be a limitation in number of open files at same time? See <a href="https://grasswiki.osgeo.org/wiki/Large_raster_data_processing#Troubleshooting" target="_blank">https://grasswiki.osgeo.org/wiki/Large_raster_data_processing#Troubleshooting</a></div>
<div>
<div class="gmail_quote">
<div>Il giorno mar 20 dic 2016 alle 22:21 Armin Burger <<a href="mailto:armin.burger@gmx.net" onclick="parent.window.location.href='armin.burger@gmx.net'; return false;" target="_blank">armin.burger@gmx.net</a>> ha scritto:</div>
<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">Hi all<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
I sometimes get read errors when trying to convert large input VRT files<br class="gmail_msg"/>
<br/>
into a single BigTIFF file, typically applying an outsize of e.g. 50% or<br class="gmail_msg"/>
<br/>
25%. Errors are like<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
ERROR 1: TIFFFillTile:Read error at row 11264, col 11264; got 46 by<br class="gmail_msg"/>
<br/>
tes, expected 84<br class="gmail_msg"/>
<br/>
ERROR 1: TIFFReadEncodedTile() failed.<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
ERROR 1: 2352/2592_BUCNFD.tif, band 1: IReadBlock failed at X offset 45,<br class="gmail_msg"/>
<br/>
Y offset 47<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
The tile that returns the error can be converted alone without problems.<br class="gmail_msg"/>
<br/>
All together the number of tiles of the VRT can be towards 6000, having<br class="gmail_msg"/>
<br/>
100 GB in total, tiles are LZW or Deflate compressed TIFF's.<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
Is there a possible source of errors when trying to read and convert<br class="gmail_msg"/>
<br/>
VRT's of compressed TIFF of that size?<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
Thanks for any hint<br class="gmail_msg"/>
<br/>
<br class="gmail_msg"/>
<br/>
Armin<br class="gmail_msg"/>
<br/>
_______________________________________________<br class="gmail_msg"/>
<br/>
gdal-dev mailing list<br class="gmail_msg"/>
<br/>
<a class="gmail_msg" href="mailto:gdal-dev@lists.osgeo.org" onclick="parent.window.location.href='gdal-dev@lists.osgeo.org'; return false;" target="_blank">gdal-dev@lists.osgeo.org</a><br class="gmail_msg"/>
<br/>
<a class="gmail_msg" href="http://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">http://lists.osgeo.org/mailman/listinfo/gdal-dev</a></blockquote>
</div>
</div>
</div>
</div>
</div>
</div></div></body></html>