[QGIS-Developer] [Qgis-user] Handling a large number of raster layers with Qgis architectural limitations
Alexandre Neto
senhor.neto at gmail.com
Mon Nov 18 01:50:08 PST 2019
Hi Patrick,
Sorry for asking, did you create pyramids (overlays) for your raster
"tiles"?
See more information about pyramids on QGIS official documentation:
https://docs.qgis.org/3.4/en/docs/user_manual/working_with_raster/raster_properties.html#pyramids-properties
To create pyramids on many files, it's probably better to use GDAL directly
to process all files in a folder:
https://gdal.org/programs/gdaladdo.html
My feeling is that the WMS service is doing that for you, and that's the
reason why it works well with a service.
Alexandre Neto
QGIS Support
www.qcooperative.net
A segunda, 18/11/2019, 09:11, Patrick Dunford <enzedrailmaps at gmail.com>
escreveu:
> Good day to all
>
> One of the user experiences I have had from using the Qgis software has
> been with projects using large numbers of raster tile layers. These layers
> are generally tiles that have a size of 4800x7200 pixels in GeoJPEG format
> and have either been downloaded directly from tile servers to these locally
> stored files, or created from downloaded tiles with other layers overlaid
> in Gimp projects.
>
> There appears to be some architectural limit in Qgis desktop software
> relating to either the total number of raster layer [files] in a project or
> to the total number of pixels in raster layer [files] in a project. This is
> unrelated to the number of layers or pixels currently enabled for display
> in the map canvas. In practice, the appearance of this limit is that it is
> kicking in long before the host computer's own physical resources are
> anywhere near fully engaged. Map digitising and editing is done on systems
> with 32 GB of physical memory (RAM) and 200 GB of SSD-based virtual memory
> (swap) and these systems are able to edit very large Gimp projects for user
> tile creation that often engage all of the system's physical memory and
> around 100 GB of the virtual memory without problems. But these types of
> numbers are in practice never seen with Qgis projects when the raster layer
> limit is being seen.
>
> The appearance of a raster layer limit is generally experienced in older
> versions of the software by layers being displayed on the canvas as
> garbage, and in newer versions by the software crashing. It will only start
> working again if raster layers are removed from the project. However, when
> layers are loaded from WMTS servers, no appearance of limitation is seen.
>
> The question to be answered, then, is which of any possible range of
> resolutions would be appropriate or useful to this predicament. With only
> limited understanding of the architectural design of the software, it would
> seem the following options exist:
>
> 1. File a bug report for the software concerning a possible issue with
> the design of the product
> 2. Amalgamate smaller tiles into larger ones (e.g. 48 tiles at
> 4800x7200 can be put into one tile at 57600x28800). This only works if the
> software issue is related to the number of file based tile layers and not
> to the total number of pixels in those layers.
> 3. Post a feature request for sub-project capabilities. This would
> allow a project that combines vector and raster layers, to be split into
> one project containing the vector layers and a number of projects each of
> which contains the vector project as a subproject and a certain subset of
> all the raster layers that is smaller than the observable limit.
> 4. Set up my own local WMTS server to serve all the raster layers to
> my map editing projects.
> 5. Explore the possibilities of preconfigured limits in the operating
> system that may need to be increased to overcome file based layer limits in
> projects (such as the NOFILES limit in Linux, currently set at 10,000 hard
> and soft on map editing computers)
>
> Is anyone who is knowledgeable about the architecture of the Qgis desktop
> software able to comment with some detail about possible resolutions.
>
>
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20191118/523519d5/attachment.html>
More information about the QGIS-Developer
mailing list