[SoC] GSoC 2021 - Final Report: New Virtual Raster Data Provider for Raster Calculator in QGIS
Francesco Bursi
francesco.bursi at hotmail.it
Thu Aug 19 23:23:07 PDT 2021
Hello everyone,
With GSoC coming to an end, I would like to share my final report of the work I have done under OSGeo organization. It has been an amazing journey, and I've learned a lot from mentors and from other developers of the QGIS community. I followed the guidelines set by Google [a] and I wrote a public GitHub gist that can be found in [b].
[a] https://developers.google.com/open-source/gsoc/help/work-product
[b] https://gist.github.com/Franc-Brs/b533fde3bec4ba16978c8fd2ef5e064f#file-gsoc2021-md
*Title of the GSoC Project*
New Virtual Raster Data Provider for Raster Calculator in QGIS.
*Organization*
QGIS under OSGeo.
*Before the coding period*
* I set up the work environment;
* I read the existing documentation for QGIS developers and I started to dive into the codebase;
* I made a pull request for a small bugfix, you can find the link to the PR in [1].
*Abstract*
The Raster Calculator in the Raster menu of QGSIS allows the user to perform calculations on the basis of existing raster pixel values. The results are written to a new raster layer in a GDAL-supported format.
The feature I've worked on allow the user to use the QGIS raster calculator without the need of writing the result to a file. I developed a virtual data provider for raster used by the raster calculator. With this feature it is possible to add a virtual (raster) layer to QGIS avoiding the creation of derived files from the geo-processes and without occupying disk space. The new raster is created on-the-fly, and it is added to the TOC of QGIS, the same raster can be used for analysis purposes or only for rendering, and it can be also used other times in the raster calculator.
*State of the QGIS Project before my GSoC*
As it is written in the documentation [2] and in the abstract above, in QGIS it is possible to perform raster algebra, but an output file is needed to write the results. Before this work, there wasn't the possibility to use this calculator without the need of writing a file to the disk.
*The addition that my project brought to QGIS*
The Pull Request I submitted and that covers all the work I've done during my GSoC can be found in [3] and the changes in the code can be found in [4], precisely. In [6] it's possible to find the new branch I created for the development of this feature. As I've described in the proposal [5] and in the above chapters, the new feature I've worked on in the last two months involves the usage of the existing raster calculator of QGIS with the possibility of computing a new raster on-the-fly without the need to write the result on a file. To achieve and to develop this functionality, I accomplished the following steps:
* Create a data provider that takes the raster data from files or from layers loaded in QGIS and use them to perform the computation. So I subclassed the QgsRasterDataProvider class and I create a QgsVirtualRasterProvider class that will take the data and load them in QGIS. This new virtual raster data provider takes advantage of some properties of the raster calculator and therefore providing to it the right information, it's possible to obtain the desired result;
* During the process of development, I also added a test class to verify that my work was consistent and the QgsVirtualRasterProvider behaved in the right way. Moreover, the TestQgsVirtualRasterProvider I created helped me to understand where the code had some flaws. The test class can be found in [4]
* After the development of the new virtual raster data provider it was possible to use the new feature via python console, since the existing raster calculator already has an amazing dialog for the user interface I made some changes to it in order to enable the user to work with my new feature through the dialog.
As it is reported by the PR at [3] it is possible to use and to test this new feature by choosing "raster calculator" under the "Raster" tab and by checking the first checkbox of the dialog near the label: "Create on-the-fly raster instead of writing layer to disk". After the check, the next things will happen:
* It will be given the possibility to choose a name for the layer that will be automatically added to the project, if nothing is typed the name is automatically taken from the formula;
* The possibility to choose a folder and the format for the output file will be hidden;
* The "Add result to project" checkbox will be checked without any other possibility.
In the next GIF is shown a simple example with the creation of an on-the-fly raster that takes the name from the formula that is used to compute it.
https://user-images.githubusercontent.com/79576081/129976410-330593ad-7d9b-4f13-80e7-eaa8e0c2e733.gif
As I mentioned, it is also possible to choose a name, as it is done in the next GIF (firstGenerationRaster).
https://user-images.githubusercontent.com/79576081/129976537-4fe91ed3-213e-4cf7-b6bc-d0e863ca6e29.gif
By selecting the checkbox "Create on-the-fly raster instead of writing layer to disk" as it has been described, there will be given the possibility to choose the name. It is also possible to compute a raster that is created by using this feature, so starting from an on-the-fly computed raster it is possible to derive another on-the-fly raster. This task is showed in the next GIF, where the on-the-fly generetad raster (called "firstGenerationRaster") is used to compute a "second generation" on-the-fly raster (that is called "secondGenerationRaster").
https://user-images.githubusercontent.com/79576081/129977013-911794a6-f79a-4943-b08f-64582412f98e.gif
The new data provider accepts a URI that must contains all the information needed, the parameters are the following:
* crs;
* extent;
* width;
* height;
* formula;
* rasterX:uri;
* rasterX:provider;
* (other rasters uri and providers if needed).
Where rasterX:uri is URI the of the raster it's referenced in the formula, and rasterX:provider is its provider. If there is the need to target multiple rasters, the URI and the provider for every raster (in the test class there are some examples of URI) should be given.
At [9] you can find the OSGeo wiki page of the project with the work I've done every week and with this final report.
*Potential future Work*
* Since I had some time left, I started a new branch [7] to enhance the raster calculator capabilities by adding the possibility of using a conditional statement in the following form: if(raster at 1>100, 400, 10). Where the computed raster pixel will have value 400 if raster at 1 is greater than 100, or it will have value 10 in any other case. This enhancement was not part of the initial proposal, and therefore it's not included in the final submission, from the PR [8] it is possible to check what I changed and what I've done so far;
* An enhancement for the developed provider can be the possibility to take advantage of OpenCL acceleration, as it has also been suggested in the dev mailing list;
* Another enhancement that concerns more the raster calculator and only partially the virtual raster provider would be the possibility to support the creation of output raster with multiple bands with the declaration of multiple formulas.
*Useful links*
I've collected all the links that I mentioned in this report and all the links that can be useful to understand my work.
[1] https://github.com/qgis/QGIS/pull/43469#event-4832189934 (small bug-fix)
[2] https://docs.qgis.org/3.16/en/docs/user_manual/working_with_raster/raster_analysis.html#raster-calculator (state-of-the-art before this GSoC)
[3] https://github.com/qgis/QGIS/pull/44195 (Pull request to the main QGIS repository)
[4] https://github.com/qgis/QGIS/pull/44195/files (Files changed and created during my GSoC)
[5] https://docs.google.com/document/d/1u8L_L1IJjGCZ3d8Vq8duxHbsHZeyzwqOe3tXa1JcKbs/edit (GSoC Proposal)
[6] https://github.com/Franc-Brs/QGIS/tree/VRProvider (New branch for the Virtual Raster Provider)
[7] https://github.com/Franc-Brs/QGIS/tree/rastercalc-conditional-statements (New branch for conditional statement in raster calculator)
[8] https://github.com/Franc-Brs/QGIS/pull/4 (PR to my personal remote repository for the new branch for the conditional statement)
[9] https://wiki.osgeo.org/wiki/New_Virtual_Raster_Data_Provider_for_Raster_Calculator_in_QGIS (Main project page)
[10] https://summerofcode.withgoogle.com/projects/#5286802463653888 (Official Google page)
I'm grateful for being able to partecipate to this amazing program and I feel that I learned a lot. I hope to continue to contribute to the QGIS project in the future.
Thank you for the support and for the advices!
My best regards,
Francesco Bursi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/soc/attachments/20210820/1c52e80e/attachment-0001.html>
More information about the SoC
mailing list