[QGIS-Developer] Building QGIS with Visual Studio 2019 CE and vcpkg
Uhrig, Stefan
stefan.uhrig at sap.com
Mon Mar 1 07:35:50 PST 2021
Hi Benjamin,
Thanks for the feedback.
AFAIK, QGIS has only recently be bumped to C++17, and the version of spatialindex in vcpkg is still 1.9.0. The fix for the binary_function issue made it to a later version only. So, the bad news is that the build with vcpkg is broken for now.
I’ve read Mathias Kuhn’s mail, and his idea of having a curated vcpkg fork seems the best approach to me to solve these kind of issues. On the one hand, we can prevent changes in upstream from breaking the build. On the other hand, we can update dependencies on our own (like the spatialindex library), merge it to the fork and then open a pull request to upstream. We would not be blocked by the upstream review process that way.
I’ll fork vcpkg and I’ll try to update the spatialindex library. If I succeed, I’ll mail you the link to the repository.
Best regards,
Stefan
From: Benjamin Jakimow <benjamin.jakimow at geo.hu-berlin.de>
Sent: Sunday, February 28, 2021 9:03 AM
To: Mats Taraldsvik <mats.taraldsvik at gmail.com>
Cc: Uhrig, Stefan <stefan.uhrig at sap.com>; qgis-developer <qgis-developer at lists.osgeo.org>
Subject: Re: [QGIS-Developer] Building QGIS with Visual Studio 2019 CE and vcpkg
Hi Stefan,
thanks for your extensive description. So far it worked out very well to setup VS, and I can confirm that geos_c.lib is detected automatically.
Unfortunately my build failed, because MovingRegion.h (which comes with vcpkg) wants to use std::binary_function, which was removed with C++17:
<vcpkg root>\installed\x64-windows\include\spatialindex\MovingRegion.h(155): error C2039: 'binary_function': is not a member of 'std'
I wonder why, because this issue has been solved in libspatialindex since Oct 2019 (https://github.com/libspatialindex/libspatialindex/pull/153)
Best regards,
Benjamin
On 2021-02-11 11:29, Mats Taraldsvik wrote:
Hi Stefan,
Thank you for this excellent guide to compile with vcpkg, it worked right away after me struggling with the official guide+CMake compilation for a couple of hours.
I did make a few modifications that might help:
1. Release build:
- geos_c.lib was detected automatically, and for some reason I did not need to explicitly set GEOS_LIBRARY
2. Debug build:
I noticed: 1> [CMake] -- Found ZSTD: C:/Users/mattar/Source/Repos/vcpkg_qgis/installed/x64-windows/lib/zstd.lib in the logs and you wrote:
> The build will fail because the execution of crssync.exe fails. That's because crssync.exe tries to load zstd.dll instead of zstdd.dll. I haven't figured out why, yet. An easy workaround is copying zstdd.dll to zstd.dll in QGIS\out\build\x64-Debug\output\bin. Then restart the build and it should succeed.
By setting ZSTD_LIBRARY explicitly to <your-vcpkg-root>/installed/x64-windows/debug/lib/zstdd.lib it picked the right lib and dll:
Then log changed to: 1> [CMake] -- Found ZSTD: C:/Users/mattar/Source/Repos/vcpkg_qgis/installed/x64-windows/debug/lib/zstdd.lib
I hope this does help a bit.
I'm an outsider to this community, but my humble suggestion _when this work is complete_ (what's missing except for python support?):
- add your CMakeSettings.json to the build description or to src/CMakeSettings.json
- vcpkg has a new feature "manifests", where a file vcpkg.json describes all dependencies. this should also be added or described (instead of the vcpkg install command).
- vcpkg just got versioning, which might help align the requirements with the rest of qgis
- I see that vcpkg already has flex and bison (<your-vcpkg-root>/downloads/tools/winflexbison), so if there is a way for win_bison.exe and win_flex.exe to be copied to the build directory, these could be used instead of cygwin, making the build even easier)
- vcpkg does have binary caching, perhaps making a github actions CI build using this method feasible?
Anyway, thank you for your efforts! It made the first QGIS-step more pleasant. :)
Best regards,
Mats Taraldsvik
On Mon, Feb 1, 2021 at 4:24 PM Uhrig, Stefan <stefan.uhrig at sap.com<mailto:stefan.uhrig at sap.com>> wrote:
Hi all,
In my opinion, the build with Visual Studio 2019 and vcpkg is still too experimental to add it to the INSTALL.md file. It might break any day if QGIS requires a package or package version that is not available from vcpkg.
Actually, I like the idea to add it to the "Show & tell" category. We can ask the community for their experiences with that setup. Does it work, is it getting used? If it is used by some "critical mass" and works over a longer time without issues, we can add instructions to the INSTALL.md file. Maybe we can add a hint to the INSTALL.md file that the not so faint-hearted developers might give it a try and provide a link to the "Show & tell" post.
However, to be really useful, the debug build should also work with Python bindings and QGIS processing. I got it working for release builds (just needed to install the required Python dependencies via pip install <package>), but that does not work for debug builds. Someone solved that some years ago in another context (https://www.riverbankcomputing.com/pipermail/pyqt/2012-April/031385.html), so I might get it working. I'll give it a try.
Furthermore, I'd like to investigate the zstd.dll vs zstdd.dll load issue before publishing the build instructions.
Best regards,
Stefan
From: QGIS-Developer <qgis-developer-bounces at lists.osgeo.org<mailto:qgis-developer-bounces at lists.osgeo.org>> On Behalf Of DelazJ
Sent: Monday, February 1, 2021 10:49 AM
To: Jorge Gustavo Rocha <jgr at di.uminho.pt<mailto:jgr at di.uminho.pt>>
Cc: qgis-developer <qgis-developer at lists.osgeo.org<mailto:qgis-developer at lists.osgeo.org>>
Subject: Re: [QGIS-Developer] Building QGIS with Visual Studio 2019 CE and vcpkg
Hi,
Why not in the build instructions of the code repo instead: https://github.com/qgis/QGIS/blob/master/INSTALL.md? If it's a working process, easy to replicate, this is the place I'd expect this kind of information.
Regards,
Harrissou
Le dim. 31 janv. 2021 à 15:55, Jorge Gustavo Rocha <jgr at di.uminho.pt<mailto:jgr at di.uminho.pt>> a écrit :
Hi Stefan,
We have now a new tab on github called 'discussions'[1]. Your detailed
instructions are a good candidate to write down a new entry there, under
'Show and tell' category. The goal is to keep this information next to
the repo. You can use the markdown syntax to enhance the writing format.
You can also add some print screens.
This is just a suggestion. Feel free to put it there or not.
Regards (and thank you!),
Jorge Gustavo
[1] https://github.com/qgis/QGIS/discussions
Às 14:13 de 31/01/21, Uhrig, Stefan escreveu:
> Install Visual Studio Community 2019: Select "Desktop development with
> C++" under "Workloads" and "C++ MFC for latest v142 build tools (x86 &
> x64)" under "Individual components". Verify that you install "Windows 10
> SDK (10.0.18362.0)" or higher.
>
>
>
> Install Git for Windows and Python 3.
>
>
>
> Download
> https://sourceforge.net/projects/winflexbison/files/win_flex_bison3-latest.zip/download
> and extract it to a location of your choice (avoid space characters in
> the target path).
>
>
>
> Follow the vcpkg "Quick Start: Windows" instructions at
> https://github.com/microsoft/vcpkg#quick-start-windows. Make sure to
> execute the "vcpkg integrate install" command.
>
>
>
> Install the required dependencies via:
>
>
>
> vcpkg --triplet=x64-windows install exiv2 gdal gsl libspatialindex
> libspatialite libzip opencl protobuf qca qscintilla qt5 qt5-serialport
> qt5-location qt5-winextras qtkeychain qwt zstd
>
>
>
> This may take a while as this creates a Debug and a Release build of
> each dependency from source.
>
>
>
> Clone the QGIS repostiory. The path to the QGIS repository should not
> contain any space characters.
>
>
>
> Start Visual Studio and open the QGIS CMakeLists.txt file via "File -->
> Open --> CMake...". This will start the CMake generation, which will
> fail. You will need several configuration rounds until everything is
> setup properly.
>
>
>
> Let's start with a Release build because that requires less tweaking. Go
> to "Project --> CMake Settings". Delete the default configuration
> ("x64-Debug (default)").
>
>
>
> Add a new "x64-Release" configuration and set "Configuration type" to
> "Release". Save the CMakeSettings.json file, which will start a new
> CMake generation. "FIND_FLEX" will fail, so you need to set the
> "FLEX_EXECUTABLE" path manually in section "CMake variables and cache".
> Browse to your win_flex.exe file. Additionally, disable "WITH_BINDINGS"
> and "WITH_QGIS_PROCESS". Save the file. Next set "BISON_EXECUTABLE" to
> your win_bison.exe file and save again. Disable "WITH_QTWEBKIT" and
> save. Set "QCA_LIBRARY" to
> "<vcpkg-root>\installed\x64-windows\lib\qca.lib" and save. Configuration
> should now succeed, but you have to change "GEOS_LIBRARY" from
> "geos.lib" to "geos_c.lib". Save and start the build after CMake
> generation has finished.
>
>
>
> Select "qgis.exe (output\bin\qgis.exe)" as start-up item in the toolbar
> and start QGIS.
>
>
>
> Now let's do the same for a Debug build. Add a "x64-Debug" configuration
> and save. Change to the "x64-Debug" configuration in the toolbar. Repeat
> the steps from above (set flex and bison paths etc.). When selecting
> libraries, take the libraries from
> "<vcpkg-root>\installed\x64-windows\debug\lib". Most libraries will have
> a "d" or "_d" as name suffix. The CMake generation will not always
> select the debug versions of libraries. Hence, go over the CMake
> variables and change release library paths to debug library paths, e.g.
> "<vcpkg-root>\installed\x64-windows\debug\lib\geos_cd.lib" for
> GEOS_LIBRARY. I'll append my CMakeSettings.json file as reference to
> this mail. Save again and start the build.
>
>
>
> The build will fail because the execution of crssync.exe fails. That's
> because crssync.exe tries to load zstd.dll instead of zstdd.dll. I
> haven't figured out why, yet. An easy workaround is copying zstdd.dll to
> zstd.dll in QGIS\out\build\x64-Debug\output\bin. Then restart the build
> and it should succeed.
>
>
>
> You should then be able to start and debug qgis.exe and its
> dependencies. Note that some vcpkg portfiles forget to copy over the
> .pdb files. For example, proj is affected. If you find such a library
> with a missing .pdb file, check whether the portfile (e.g.
> "<vcpkg-root>\ports\proj4\portfile.cmake") contains the
> "vcpkg_copy_pdbs()" call. If not, just add it at the end, re-install the
> dependency and copy over the .pdb file to the QGIS output directory.
>
>
>
> I'm currently trying to get QGIS working with Python bindings. I got the
> build working, however I'm struggling with the debug build execution.
> The debug Python library seems to be unable to load the Release build Qt
> libraries that come with PyQt5. Well, I'll keep trying and if I succeed,
> I'll update the instructions.
>
>
>
> Here is my CMakeSettings.json file for reference:
>
>
>
> {
>
> "configurations": [
>
> {
>
> "name": "x64-Release",
>
> "generator": "Ninja",
>
> "configurationType": "Release",
>
> "buildRoot": "${projectDir}\\out\\build\\${name}",
>
> "installRoot": "${projectDir}\\out\\install\\${name}",
>
> "cmakeCommandArgs": "",
>
> "buildCommandArgs": "",
>
> "ctestCommandArgs": "",
>
> "inheritEnvironments": [ "msvc_x64_x64" ],
>
> "variables": [
>
> {
>
> "name": "WITH_BINDINGS",
>
> "value": "False",
>
> "type": "BOOL"
>
> },
>
> {
>
> "name": "WITH_QGIS_PROCESS",
>
> "value": "False",
>
> "type": "BOOL"
>
> },
>
> {
>
> "name": "FLEX_EXECUTABLE",
>
> "value": "C:/src/tools/win_flex.exe",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "BISON_EXECUTABLE",
>
> "value": "C:/src/tools/win_bison.exe",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "WITH_QTWEBKIT",
>
> "value": "False",
>
> "type": "BOOL"
>
> },
>
> {
>
> "name": "QCA_LIBRARY",
>
> "value": "C:/src/vcpkg/installed/x64-windows/lib/qca.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "GEOS_LIBRARY",
>
> "value": "C:/src/vcpkg/installed/x64-windows/lib/geos_c.lib",
>
> "type": "FILEPATH"
>
> }
>
> ]
>
> },
>
> {
>
> "name": "x64-Debug",
>
> "generator": "Ninja",
>
> "configurationType": "Debug",
>
> "buildRoot": "${projectDir}\\out\\build\\${name}",
>
> "installRoot": "${projectDir}\\out\\install\\${name}",
>
> "cmakeCommandArgs": "",
>
> "buildCommandArgs": "",
>
> "ctestCommandArgs": "",
>
> "inheritEnvironments": [ "msvc_x64_x64" ],
>
> "variables": [
>
> {
>
> "name": "FLEX_EXECUTABLE",
>
> "value": "C:/src/tools/win_flex.exe",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "BISON_EXECUTABLE",
>
> "value": "C:/src/tools/win_bison.exe",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "WITH_QTWEBKIT",
>
> "value": "False",
>
> "type": "BOOL"
>
> },
>
> {
>
> "name": "WITH_BINDINGS",
>
> "value": "False",
>
> "type": "BOOL"
>
> },
>
> {
>
> "name": "WITH_QGIS_PROCESS",
>
> "value": "False",
>
> "type": "BOOL"
>
> },
>
> {
>
> "name": "QCA_LIBRARY",
>
> "value": "C:/src/vcpkg/installed/x64-windows/debug/lib/qcad.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "GEOS_LIBRARY",
>
> "value":
> "C:/src/vcpkg/installed/x64-windows/debug/lib/geos_cd.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "GSL_LIB",
>
> "value": "C:/src/vcpkg/installed/x64-windows/debug/lib/gsld.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "GSLCBLAS_LIB",
>
> "value":
> "C:/src/vcpkg/installed/x64-windows/debug/lib/gslcblasd.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "PROJ_LIBRARY",
>
> "value":
> "C:/src/vcpkg/installed/x64-windows/debug/lib/proj_d.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "QSCINTILLA_LIBRARY",
>
> "value":
> "C:/src/vcpkg/installed/x64-windows/debug/lib/qscintilla2_qt5d.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "QTKEYCHAIN_LIBRARY",
>
> "value":
> "C:/src/vcpkg/installed/x64-windows/debug/lib/qt5keychaind.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "QWT_LIBRARY",
>
> "value": "C:/src/vcpkg/installed/x64-windows/debug/lib/qwtd.lib",
>
> "type": "FILEPATH"
>
> },
>
> {
>
> "name": "SPATIALINDEX_LIBRARY",
>
> "value":
> "C:/src/vcpkg/installed/x64-windows/debug/lib/spatialindex-64d.lib",
>
> "type": "FILEPATH"
>
> }
>
> ]
>
> }
>
> ]
>
> }
>
>
>
>
>
>
>
>
>
> *From:*i-s-o <46.i.s.o.64 at gmail.com<mailto:46.i.s.o.64 at gmail.com>>
> *Sent:* Saturday, January 30, 2021 5:58 PM
> *To:* Uhrig, Stefan <stefan.uhrig at sap.com<mailto:stefan.uhrig at sap.com>>
> *Cc:* qgis-developer <QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>>
> *Subject:* Re: [QGIS-Developer] Building QGIS with Visual Studio 2019 CE
> and vcpkg
>
>
>
> I am very interested in trying out your solution. Could you share the
> required steps?
>
>
>
> Thx.
>
>
>
> On Fri, Jan 29, 2021, 09:25 Uhrig, Stefan <stefan.uhrig at sap.com<mailto:stefan.uhrig at sap.com>
> <mailto:stefan.uhrig at sap.com<mailto:stefan.uhrig at sap.com>>> wrote:
>
> TL;DR: It is currently possible to build the QGIS core app with
> Visual Studio 2019 and vcpkg, which makes debugging QGIS
> dependencies easy.
>
>
>
>
>
> Hi all,
>
>
>
> Some time ago I discovered vcpkg
> (https://github.com/microsoft/vcpkg). vcpkg is a package manager
> that downloads package source code to your local machine and builds
> the package locally. Recently, I discovered that vcpkg should be
> able to provide all dependencies to build at least the QGIS core
> application. Hence, I gave it a try.
>
>
>
> Basically, it worked out of the box. I started with a fresh Windows
> 10 installation, installed Visual Studio 2019 Community Edition,
> Git, vcpkg, Python 3 and flex and bison for Windows. I fetched all
> other dependencies via vcpkg. It was not necessary to even touch a
> single file in the repository. I could just open the main
> CMakeLists.txt file in Visual Studio and only had to tweak the CMake
> cache (the CMake find macros that come with QGIS are not aware of
> vcpkg, so I had to set some paths manually). I had to switch off
> some extensions though as the required dependencies were not
> available via vcpkg (WITH_BINDINGS, WITH_QGIS_PROCESS,
> WITH_QTWEBKIT). The build did not report any errors, I could start
> the application and it seems to work, but I did some light testing only.
>
>
>
> I mainly tried it because I enjoy debugging with Visual Studio more
> than with gdb (or gdb wrapped in some IDE). In my experience, the
> performance of the Visual Studio debugger is better and it is more
> stable, especially in long debug sessions.
>
>
>
> I don't want to promote official building support of QGIS with
> vcpkg. Providing the dependencies via OSGeo4W is much more reliable.
> However, if you don't mind the experimental nature of this setup and
> you want to be able to debug into QGIS' dependencies, you might give
> it a try. Especially, if you want to track the cause of a crash in
> one of QGIS' dependencies, this setup might be helpful. You have the
> source code and debug versions of the dependencies, so the debugger
> will jump to the crashing code line and you can inspect all the
> variables of the dependency.
>
>
>
> If someone is interested in trying it, give me a note. I can then
> assemble detailed instructions on how to make it work. It took me a
> while to figure out which packages are needed and how the CMake
> cache needs to be tweaked.
>
>
>
> Best regards,
>
> Stefan
>
>
>
>
>
>
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org> <mailto:QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>>
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
J. Gustavo
--
Jorge Gustavo Rocha
Departamento de Informática
Universidade do Minho
4710-057 Braga
Gabinete 3.29 (Piso 3)
Tel: +351 253604480
Fax: +351 253604471
Móvel: +351 910333888
skype: nabocudnosor
_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
_______________________________________________
QGIS-Developer mailing list
QGIS-Developer at lists.osgeo.org<mailto:QGIS-Developer at lists.osgeo.org>
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
--
--
Benjamin Jakimow, Doctoral Researcher
Earth Observation Lab | Geography Department | Humboldt-Universität zu Berlin
e-mail: benjamin.jakimow at geo.hu-berlin.de<mailto:benjamin.jakimow at geo.hu-berlin.de>
phone: +49 (0) 30 2093 6894
mobile: +49 (0) 157 5656 8477
fax: +49 (0) 30 2093 6848
mail: Unter den Linden 6 | 10099 Berlin | Germany
room: 2'222
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20210301/a3204662/attachment-0001.html>
More information about the QGIS-Developer
mailing list