[Qgis-user] Handling a large number of raster layers with Qgis architectural limitations

[Alexandre Neto]

If the problem is from the number of files (and not actually it's size)
then creating a virtual raster may help to solve it.

https://docs.qgis.org/3.4/en/docs/user_manual/processing_algs/gdal/rastermiscellaneous.html#build-virtual-raster

Alexandre Neto
QGIS Support
www.qcooperative.net

A segunda, 18/11/2019, 19:36, Nicolas Cadieux <nicolas.cadieux at archeotec.ca>
escreveu:

> Hi,
>
> I ran into same problems when dealing with a few thousand files.  The idea
> was to load the very small shp files created by another process and then
> merge then.  I solved the problem by loading and merge it then I SAGA.  It
> generally loads everything into memory and probably does not keep a file
> handle once the file are open.  I believe this issue is created by Windows
> and not QGIS.  I remember being told to go on Linux as this maximum
> operating system file max can be modified with a script.
>
> Nicolas
>
> > Le 18 nov. 2019 à 04:59, Jésahel Benoist <djes1975 at gmail.com> a écrit :
> >
> > From my experience, GeoTIFF has a long history and is a more
> > appropriate format to handle multiple large rasters. As a container,
> > it could handle misc compression format (JPEG an other), misc
> > representation at different scales (resolution is not a problem), misc
> > color modes with raster/vectorial alpha layer, and so on. In one of my
> > projects I handle more than 400 raster files (4000x4000x32) without
> > any problem.
> > Of course, a better and final choice would be to tile everything, but
> > it is sometimes difficult with older maps.
> >
> >> Le lun. 18 nov. 2019 à 10:12, Patrick Dunford
> >> <enzedrailmaps at gmail.com> a écrit :
> >>
> >> Good day to all
> >>
> >> One of the user experiences I have had from using the Qgis software has
> been with projects using large numbers of raster tile layers. These layers
> are generally tiles that have a size of 4800x7200 pixels in GeoJPEG format
> and have either been downloaded directly from tile servers to these locally
> stored files, or created from downloaded tiles with other layers overlaid
> in Gimp projects.
> >>
> >> There appears to be some architectural limit in Qgis desktop software
> relating to either the total number of raster layer [files] in a project or
> to the total number of pixels in raster layer [files] in a project. This is
> unrelated to the number of layers or pixels currently enabled for display
> in the map canvas. In practice, the appearance of this limit is that it is
> kicking in long before the host computer's own physical resources are
> anywhere near fully engaged. Map digitising and editing is done on systems
> with 32 GB of physical memory (RAM) and 200 GB of SSD-based virtual memory
> (swap) and these systems are able to edit very large Gimp projects for user
> tile creation that often engage all of the system's physical memory and
> around 100 GB of the virtual memory without problems. But these types of
> numbers are in practice never seen with Qgis projects when the raster layer
> limit is being seen.
> >>
> >> The appearance of a raster layer limit is generally experienced in
> older versions of the software by layers being displayed on the canvas as
> garbage, and in newer versions by the software crashing. It will only start
> working again if raster layers are removed from the project. However, when
> layers are loaded from WMTS servers, no appe
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20191119/514a9dd8/attachment.html>