<div dir="ltr">More specifically, I'm guessing the MOST efficient is to read whole blocks from one band, which presumably avoids crossing compression boundaries etc.<div><br></div><div>A block may be a scanline, or it may be a tile, depending on the format.</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 1, 2021 at 2:29 PM Simon Eves <<a href="mailto:simon.eves@omnisci.com">simon.eves@omnisci.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Band by band makes sense. I shall do that instead. Thank you! :)</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Nov 1, 2021 at 2:05 PM Even Rouault <<a href="mailto:even.rouault@spatialys.com" target="_blank">even.rouault@spatialys.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>Yes regarding multithreading. Regarding GRIB and performance
issues, you must be aware that the GRIB driver when accessing a
single pixel of a band needs to decompress data for the whole
band. Hence there's a per-dataset cache of band data which default
to 100 MB (you can increase it by setting the GRIB_CACHEMAX config
option to a number in megabytes). So the most performance access
pattern for GRIB is to read band per band, and no
all-bands-of-a-line<br>
</p>
<div>Le 01/11/2021 à 21:58, Simon Eves a
écrit :<br>
</div>
<blockquote type="cite">
<div dir="ltr">You can ignore this.
<div><br>
</div>
<div>I have rather belatedly found the documentation that says
that one must open a GDALDataset per thread, even if it's on
the same file.</div>
<div><br>
</div>
<div>The multi-threading now works just fine.</div>
<div><br>
</div>
<div>Interestingly, we're not actually doing that with our
existing geo importer. I guess it's OK because we're pulling
the OGRFeatures out with the process thread, and only
converting and loading them with the child threads. I guess I
really ought to rewrite that code too now. Sigh.</div>
<div><br>
</div>
<div>As you were...</div>
<div><br>
</div>
<div>Simon</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Sun, Oct 31, 2021 at 4:27
PM Simon Eves <<a href="mailto:simon.eves@omnisci.com" target="_blank">simon.eves@omnisci.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">We are writing a raster importer, and
finding that GDALRasterBand::RasterIO() is unexpectedly slow
for some GRIB2 files.
<div><br>
</div>
<div>We have a file which is about 1800x1000 pixels, with 49
bands of type DOUBLE. The file is about 47MB on disc.</div>
<div><br>
</div>
<div>Reading all the bands of a single scanline from this
file takes about 1300ms, which is about 26ms per band,
hence the entire file takes around 20 minutes to import.
All the time seems to be spent in the RasterIO() call,
even though it's not doing any raster rescaling or data
format conversion (1:1 pixels, fetching as GDT_Float64).</div>
<div><br>
</div>
<div>So, I figured we'd try multi-threading it, but
evidently the call is not thread-safe. Here is just one of
various stack traces it will throw.</div>
<div><br>
</div>
<div>libc.so.6!raise (Unknown Source:0)<br>
libc.so.6!abort (Unknown Source:0)<br>
libc.so.6![Unknown/Just-In-Time compiled code] (Unknown
Source:0)<br>
libgdal.so.28!GRIBRasterBand::UncacheData(GRIBRasterBand *
const this)
(/build/scripts/gdal-3.2.2/frmts/grib/gribdataset.cpp:948)<br>
libgdal.so.28!GRIBRasterBand::LoadData(GRIBRasterBand *
const this)
(/build/scripts/gdal-3.2.2/frmts/grib/gribdataset.cpp:730)<br>
libgdal.so.28!GRIBRasterBand::LoadData(GRIBRasterBand *
const this)
(/build/scripts/gdal-3.2.2/frmts/grib/gribdataset.cpp:697)<br>
libgdal.so.28!GRIBRasterBand::IReadBlock(GRIBRasterBand *
const this, int nBlockYOff, void * pImage)
(/build/scripts/gdal-3.2.2/frmts/grib/gribdataset.cpp:803)<br>
libgdal.so.28!GDALRasterBand::GetLockedBlockRef(int
bJustInitialize, int nYBlockOff, int nXBlockOff,
GDALRasterBand * const this)
(/build/scripts/gdal-3.2.2/gcore/gdal_priv.h:963)<br>
libgdal.so.28!GDALRasterBand::GetLockedBlockRef(GDALRasterBand * const
this, int nXBlockOff, int nYBlockOff, int bJustInitialize)
(/build/scripts/gdal-3.2.2/gcore/gdalrasterband.cpp:1238)<br>
libgdal.so.28!GDALRasterBand::IRasterIO(GDALRasterBand *
const this, GDALRWFlag eRWFlag, int nXOff, int nYOff, int
nXSize, int nYSize, void * pData, int nBufXSize, int
nBufYSize, GDALDataType eBufType, GSpacing nPixelSpace,
GSpacing nLineSpace, GDALRasterIOExtraArg * psExtraArg)
(/build/scripts/gdal-3.2.2/gcore/rasterio.cpp:149)<br>
libgdal.so.28!GDALRasterBand::RasterIO(GDALRasterBand *
const this, GDALRWFlag eRWFlag, int nXOff, int nYOff, int
nXSize, int nYSize, void * pData, int nBufXSize, int
nBufYSize, GDALDataType eBufType, GSpacing nPixelSpace,
GSpacing nLineSpace, GDALRasterIOExtraArg * psExtraArg)
(/build/scripts/gdal-3.2.2/gcore/gdalrasterband.cpp:372)<br>
import_export::Importer::<lambda(size_t,
int)>::operator()(size_t, int) const(const
import_export::Importer::<lambda(size_t, int)> *
const __closure, const size_t thread_id, const int y)
(/home/simon.eves/work/omniscidb-internal/ImportExport/Importer.cpp:5721)<br>
</div>
<div>...</div>
<div><br>
</div>
<div>All of the parameters to the call are either constant
or uncontended simple variables, and obviously there is a
unique data buffer (pData) per thread.</div>
<div><br>
</div>
<div>Is there anything we can do to make this work?</div>
<div><br>
</div>
<div>I was intending to look into the lower level
block-based API, in the hope that it will be faster, but
have not yet done so.</div>
<div><br>
</div>
<div>This is all with a local static build of GDAL 3.2.2 on
Ubuntu 20.04 with GCC 9.<br>
</div>
<div>
<div><br>
</div>
<div>Yours,</div>
<div><br>
</div>
<div>Simon Eves</div>
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div style="margin:0px;padding:0px 0px 20px;width:2544px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:medium">
<div>
<div style="font-size:12.8px;margin:8px 0px 0px;padding:0px">
<div>
<div dir="ltr"><span><font color="#888888">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<table style="font-family:Times;width:2544px" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="vertical-align:top;font-size:0px" align="left">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0px 15px 0px 0px;vertical-align:middle" align="left"><font size="2" face="arial,
helvetica,
sans-serif"><a href="http://www.omnisci.com/" target="_blank"><img src="http://www2.omnisci.com/l/298412/2018-09-18/3sqpg/298412/61753/OmniSci_Email_Header2.png"></a><br>
</font></td>
<td style="padding:0px 0px 0px 15px;vertical-align:top" align="left">
<table style="width:215px" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><span style="color:rgb(14,76,144);font-weight:700"><font size="2" face="arial,
helvetica,
sans-serif">Simon
Eves</font></span></span></td>
</tr>
<tr>
<td style="vertical-align:top" align="left">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><font size="2" face="arial,
helvetica,
sans-serif">Senior
Graphics
Engineer,
Rendering
Group<br>
100 Montgomery
St (5th
Floor), San
Francisco, CA
94104, USA<br>
</font></span></td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="vertical-align:top" align="left">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0px;vertical-align:top" align="left"><br>
</td>
<td style="padding:0px;vertical-align:top" align="left"><br>
</td>
</tr>
<tr>
<td style="padding:0px;vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><font size="2" face="arial,
helvetica,
sans-serif">Email: <a href="mailto:simon.eves@omnisci.com" target="_blank">simon.eves@omnisci.com</a> |
Cell: </font></span></td>
<td style="padding:0px;vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><font size="2" face="arial,
helvetica,
sans-serif">+1
(415) 902-1996</font></span></td>
</tr>
</tbody>
</table>
<br>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</font></span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div style="margin:0px;padding:0px 0px 20px;width:2544px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:medium">
<div>
<div style="font-size:12.8px;margin:8px 0px 0px;padding:0px">
<div>
<div dir="ltr"><span><font color="#888888">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<table style="font-family:Times;width:2544px" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="vertical-align:top;font-size:0px" align="left">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0px 15px 0px 0px;vertical-align:middle" align="left"><font size="2" face="arial,
helvetica,
sans-serif"><a href="http://www.omnisci.com/" target="_blank"><img src="http://www2.omnisci.com/l/298412/2018-09-18/3sqpg/298412/61753/OmniSci_Email_Header2.png"></a><br>
</font></td>
<td style="padding:0px 0px 0px 15px;vertical-align:top" align="left">
<table style="width:215px" cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><span style="color:rgb(14,76,144);font-weight:700"><font size="2" face="arial,
helvetica,
sans-serif">Simon
Eves</font></span></span></td>
</tr>
<tr>
<td style="vertical-align:top" align="left">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><font size="2" face="arial,
helvetica,
sans-serif">Senior
Graphics
Engineer,
Rendering
Group<br>
100 Montgomery
St (5th
Floor), San
Francisco, CA
94104, USA<br>
</font></span></td>
</tr>
</tbody>
</table>
</td>
</tr>
<tr>
<td style="vertical-align:top" align="left">
<table cellspacing="0" cellpadding="0" border="0">
<tbody>
<tr>
<td style="padding:0px;vertical-align:top" align="left"><br>
</td>
<td style="padding:0px;vertical-align:top" align="left"><br>
</td>
</tr>
<tr>
<td style="padding:0px;vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><font size="2" face="arial,
helvetica,
sans-serif">Email: <a href="mailto:simon.eves@omnisci.com" target="_blank">simon.eves@omnisci.com</a> |
Cell: </font></span></td>
<td style="padding:0px;vertical-align:top" align="left"><span style="white-space:nowrap;color:rgb(0,0,1)"><font size="2" face="arial,
helvetica,
sans-serif">+1
(415) 902-1996</font></span></td>
</tr>
</tbody>
</table>
<br>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</font></span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
gdal-dev mailing list
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre cols="72">--
<a href="http://www.spatialys.com" target="_blank">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div style="margin:0px;padding:0px 0px 20px;width:2544px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:medium"><div><div style="font-size:12.8px;margin:8px 0px 0px;padding:0px"><div><div dir="ltr"><span><font color="#888888"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><table cellpadding="0" cellspacing="0" border="0" style="font-family:Times;width:2544px"><tbody><tr><td align="left" style="vertical-align:top;font-size:0px"><table cellpadding="0" cellspacing="0" border="0"><tbody><tr><td align="left" style="padding:0px 15px 0px 0px;vertical-align:middle"><font face="arial, helvetica, sans-serif" size="2"><a href="http://www.omnisci.com/" target="_blank"><img src="http://www2.omnisci.com/l/298412/2018-09-18/3sqpg/298412/61753/OmniSci_Email_Header2.png"></a><br></font></td><td align="left" style="padding:0px 0px 0px 15px;vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="width:215px"><tbody><tr><td align="left" style="vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><span style="color:rgb(14,76,144);font-weight:700"><font face="arial, helvetica, sans-serif" size="2">Simon Eves</font></span></span></td></tr><tr><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0"><tbody><tr><td align="left" style="vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><font face="arial, helvetica, sans-serif" size="2">Senior Graphics Engineer, Rendering Group<br>100 Montgomery St (5th Floor), San Francisco, CA 94104, USA<br></font></span></td></tr></tbody></table></td></tr><tr><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0"><tbody><tr><td align="left" style="padding:0px;vertical-align:top"><br></td><td align="left" style="padding:0px;vertical-align:top"><br></td></tr><tr><td align="left" style="padding:0px;vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><font face="arial, helvetica, sans-serif" size="2">Email: <a href="mailto:simon.eves@omnisci.com" target="_blank">simon.eves@omnisci.com</a> | Cell: </font></span></td><td align="left" style="padding:0px;vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><font face="arial, helvetica, sans-serif" size="2">+1 (415) 902-1996</font></span></td></tr></tbody></table><br></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table><br></div></div></div></div></div></div></div></div></div></div></font></span></div><div></div></div></div><div></div></div></div></div></div></div></div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div style="margin:0px;padding:0px 0px 20px;width:2544px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:medium"><div><div style="font-size:12.8px;margin:8px 0px 0px;padding:0px"><div><div dir="ltr"><span><font color="#888888"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><table cellpadding="0" cellspacing="0" border="0" style="font-family:Times;width:2544px"><tbody><tr><td align="left" style="vertical-align:top;font-size:0px"><table cellpadding="0" cellspacing="0" border="0"><tbody><tr><td align="left" style="padding:0px 15px 0px 0px;vertical-align:middle"><font face="arial, helvetica, sans-serif" size="2"><a href="http://www.omnisci.com/" target="_blank"><img src="http://www2.omnisci.com/l/298412/2018-09-18/3sqpg/298412/61753/OmniSci_Email_Header2.png"></a><br></font></td><td align="left" style="padding:0px 0px 0px 15px;vertical-align:top"><table cellpadding="0" cellspacing="0" border="0" style="width:215px"><tbody><tr><td align="left" style="vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><span style="color:rgb(14,76,144);font-weight:700"><font face="arial, helvetica, sans-serif" size="2">Simon Eves</font></span></span></td></tr><tr><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0"><tbody><tr><td align="left" style="vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><font face="arial, helvetica, sans-serif" size="2">Senior Graphics Engineer, Rendering Group<br>100 Montgomery St (5th Floor), San Francisco, CA 94104, USA<br></font></span></td></tr></tbody></table></td></tr><tr><td align="left" style="vertical-align:top"><table cellpadding="0" cellspacing="0" border="0"><tbody><tr><td align="left" style="padding:0px;vertical-align:top"><br></td><td align="left" style="padding:0px;vertical-align:top"><br></td></tr><tr><td align="left" style="padding:0px;vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><font face="arial, helvetica, sans-serif" size="2">Email: <a href="mailto:simon.eves@omnisci.com" target="_blank">simon.eves@omnisci.com</a> | Cell: </font></span></td><td align="left" style="padding:0px;vertical-align:top"><span style="white-space:nowrap;color:rgb(0,0,1)"><font face="arial, helvetica, sans-serif" size="2">+1 (415) 902-1996</font></span></td></tr></tbody></table><br></td></tr></tbody></table></td></tr></tbody></table></td></tr></tbody></table><br></div></div></div></div></div></div></div></div></div></div></font></span></div><div></div></div></div><div></div></div></div></div></div></div></div></div></div></div></div>