<div dir="ltr"><div dir="ltr">Dear Colleagues, Patrick,</div><div dir="ltr"><br></div><div>thanks, this is a VERY relevant issue, and might be the future concept for GIS, of sorts (=more data volumes).</div><div>While I have no direct answer to your architecture question (sorry),</div><div>I have a sample dataset and application on the topic and can share it with people as they see fit.</div><div>Here the manuscript and data, unpublished</div><div><a href="https://www.earth-syst-sci-data-discuss.net/essd-2016-65/">https://www.earth-syst-sci-data-discuss.net/essd-2016-65/</a> <br></div><div><br></div><div>The 104 GIS data layers (rasters) I can share via a google drive etc with people on request and for more</div><div>progress and completion of the topic.</div><div><br></div><div>Arguably,</div><div>people are shopping around in the GIS world for a solution and platform</div><div>for such type of (massive multi) layer questions/analysis, and classic R has usually a memory issue,</div><div>and supercomputing and 'the cloud' lacks the 'usual' GIS platform etc etc.</div><div>So here comes QGIS to the rescue, more or less</div><div>(consider Metadata creation in such projects also).</div><div>I am happy to learn.</div><div><br></div><div>Please keep me posted on this topic, offline as needed.</div><div><br></div><div>Thanks so much, very best</div><div> Falk Huettmann</div><div><br></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Nov 27, 2019 at 6:35 PM Patrick Dunford <<a href="mailto:enzedrailmaps@gmail.com">enzedrailmaps@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Hello,</p>
<p>the issue is not with the performance drawing the raster, it is
with the system only being able to have a certain number of file
based rasters loaded in a project before it will crash.<br>
</p>
<div>On 22/11/19 11:08 AM, Alexandre Neto
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">AFAIK,
<div><br>
</div>
<div>WMST creates pyramids in cache as needed, hence it may do a
better balance between disk usage and performance. You can
control the number of pyramid layers you create and the
compression format of it to save some space. But having raster
files with 4000x4000 pixels with no pyramids will make a full
read of the raster file to show even only a bit of it... That
will make QGIS performance really poor.</div>
<div><br>
</div>
<div>Thereºs this blog post from 2010 that does some comparison
between several formats and their sizes and performances. It's
in portuguese, but I think you can find ways of translating
it.</div>
<div><br>
</div>
<div><br>
</div>
<div><a href="https://blog.viasig.com/2010/01/mosaicos-de-imagens-em-mapserver-com-gdal/" target="_blank">https://blog.viasig.com/2010/01/mosaicos-de-imagens-em-mapserver-com-gdal/</a><br>
</div>
<div><br>
</div>
<div>Hope it helps</div>
<div><br>
</div>
<div>Alexandre Neto</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Nov 21, 2019 at 2:05
PM Patrick Dunford <<a href="mailto:enzedrailmaps@gmail.com" target="_blank">enzedrailmaps@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Building pyramids gobbles up heaps of disk space. I tried
pyramids on 157 MB layer, which created an extra file 1.6
GiB.</p>
<p>I can't see the WMTS building lots of temporary file
pyramids using up disk<br>
</p>
<div>On 18/11/19 10:50 PM, Alexandre Neto wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto">Hi Patrick,
<div dir="auto"><br>
</div>
<div dir="auto">Sorry for asking, did you create
pyramids (overlays) for your raster "tiles"?</div>
<div dir="auto"><br>
</div>
<div dir="auto">See more information about pyramids on
QGIS official documentation:</div>
<div dir="auto"><br>
</div>
<div dir="auto"><a href="https://docs.qgis.org/3.4/en/docs/user_manual/working_with_raster/raster_properties.html#pyramids-properties" target="_blank">https://docs.qgis.org/3.4/en/docs/user_manual/working_with_raster/raster_properties.html#pyramids-properties</a><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">To create pyramids on many files, it's
probably better to use GDAL directly to process all
files in a folder:</div>
<div dir="auto"><br>
</div>
<div dir="auto"><a href="https://gdal.org/programs/gdaladdo.html" target="_blank">https://gdal.org/programs/gdaladdo.html</a><br>
</div>
<div dir="auto"><br>
</div>
<div dir="auto">My feeling is that the WMS service is
doing that for you, and that's the reason why it works
well with a service.</div>
<div dir="auto"><br>
</div>
<div dir="auto">Alexandre Neto</div>
<div dir="auto">QGIS Support</div>
<div dir="auto"><a href="http://www.qcooperative.net" target="_blank">www.qcooperative.net</a></div>
</div>
<br>
</blockquote>
</div>
_______________________________________________<br>
Qgis-user mailing list<br>
<a href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a></blockquote>
</div>
</blockquote>
</div>
_______________________________________________<br>
Qgis-user mailing list<br>
<a href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a></blockquote></div></div>