[QGIS-Developer] Building QGIS with Visual Studio 2019 CE and vcpkg
Matthias Kuhn
matthias at opengis.ch
Sun Feb 28 00:51:55 PST 2021
Hi All,
Thanks for the pointer Stefan, this is very interesting. I spent a bit of
time this week experimenting with it. And the first impression is positive
(a bit bumpy sometimes when digging deeper, but I think that's in the
nature of every build system)
Some additional findings:
- The manifest approach mentioned by Mats is really nice, you can create a
fully configured VS project from the command line. It will download and
install deps and subsequently build the app without any manual steps. This
will also create a vs project to open and work with.
- I started to work on (or rather continued to revive a stalled) port (
https://github.com/m-kuhn/vcpkg/tree/qgis/ports/qgis). Likely the
vcpkg.json from here can be dropped into the root of a QGIS source dir and
then built with
cmake.exe -S [path_to_qgis_source] -B [path_to_build_dir]
-DCMAKE_TOOLCHAIN_FILE=[path_to_vcpkg]\scripts\buildsystems\vcpkg.cmake
cmake.exe --build [path_to_build_dir]
- Experience shows that systems like this suffer often from library
updates that suddenly break systems. Version pinning (or a curated fork?)
might help here. I haven't looked into that yet.
Matthias
On Thu, Feb 11, 2021 at 11:30 AM Mats Taraldsvik <mats.taraldsvik at gmail.com>
wrote:
> Hi Stefan,
>
> Thank you for this excellent guide to compile with vcpkg, it worked right
> away after me struggling with the official guide+CMake compilation for a
> couple of hours.
>
> I did make a few modifications that might help:
>
> 1. Release build:
> - geos_c.lib was detected automatically, and for some reason I did not
> need to explicitly set GEOS_LIBRARY
>
> 2. Debug build:
>
> I noticed: 1> [CMake] -- Found ZSTD:
> C:/Users/mattar/Source/Repos/vcpkg_qgis/installed/x64-windows/lib/zstd.lib
> in the logs and you wrote:
> > The build will fail because the execution of crssync.exe fails. That's
> because crssync.exe tries to load zstd.dll instead of zstdd.dll. I haven't
> figured out why, yet. An easy workaround is copying zstdd.dll to zstd.dll
> in QGIS\out\build\x64-Debug\output\bin. Then restart the build and it
> should succeed.
>
> By setting ZSTD_LIBRARY explicitly to
> <your-vcpkg-root>/installed/x64-windows/debug/lib/zstdd.lib it picked the
> right lib and dll:
> Then log changed to: 1> [CMake] -- Found ZSTD:
> C:/Users/mattar/Source/Repos/vcpkg_qgis/installed/x64-windows/debug/lib/zstdd.lib
>
> I hope this does help a bit.
>
> I'm an outsider to this community, but my humble suggestion _when this
> work is complete_ (what's missing except for python support?):
> - add your CMakeSettings.json to the build description or to
> src/CMakeSettings.json
> - vcpkg has a new feature "manifests", where a file vcpkg.json describes
> all dependencies. this should also be added or described (instead of the
> vcpkg install command).
> - vcpkg just got versioning, which might help align the requirements with
> the rest of qgis
> - I see that vcpkg already has flex and bison
> (<your-vcpkg-root>/downloads/tools/winflexbison), so if there is a way for
> win_bison.exe and win_flex.exe to be copied to the build directory, these
> could be used instead of cygwin, making the build even easier)
> - vcpkg does have binary caching, perhaps making a github actions CI build
> using this method feasible?
>
> Anyway, thank you for your efforts! It made the first QGIS-step more
> pleasant. :)
>
> Best regards,
> Mats Taraldsvik
>
> On Mon, Feb 1, 2021 at 4:24 PM Uhrig, Stefan <stefan.uhrig at sap.com> wrote:
>
>> Hi all,
>>
>>
>>
>> In my opinion, the build with Visual Studio 2019 and vcpkg is still too
>> experimental to add it to the INSTALL.md file. It might break any day if
>> QGIS requires a package or package version that is not available from vcpkg.
>>
>>
>>
>> Actually, I like the idea to add it to the “Show & tell” category. We can
>> ask the community for their experiences with that setup. Does it work, is
>> it getting used? If it is used by some “critical mass” and works over a
>> longer time without issues, we can add instructions to the INSTALL.md file.
>> Maybe we can add a hint to the INSTALL.md file that the not so
>> faint-hearted developers might give it a try and provide a link to the
>> “Show & tell” post.
>>
>>
>>
>> However, to be really useful, the debug build should also work with
>> Python bindings and QGIS processing. I got it working for release builds
>> (just needed to install the required Python dependencies via pip install
>> <package>), but that does not work for debug builds. Someone solved that
>> some years ago in another context (
>> https://www.riverbankcomputing.com/pipermail/pyqt/2012-April/031385.html),
>> so I might get it working. I’ll give it a try.
>>
>>
>>
>> Furthermore, I’d like to investigate the zstd.dll vs zstdd.dll load issue
>> before publishing the build instructions.
>>
>>
>>
>> Best regards,
>>
>> Stefan
>>
>>
>>
>>
>>
>>
>>
>> *From:* QGIS-Developer <qgis-developer-bounces at lists.osgeo.org> *On
>> Behalf Of *DelazJ
>> *Sent:* Monday, February 1, 2021 10:49 AM
>> *To:* Jorge Gustavo Rocha <jgr at di.uminho.pt>
>> *Cc:* qgis-developer <qgis-developer at lists.osgeo.org>
>> *Subject:* Re: [QGIS-Developer] Building QGIS with Visual Studio 2019 CE
>> and vcpkg
>>
>>
>>
>> Hi,
>>
>>
>>
>> Why not in the build instructions of the code repo instead:
>> https://github.com/qgis/QGIS/blob/master/INSTALL.md? If it's a working
>> process, easy to replicate, this is the place I'd expect this kind of
>> information.
>>
>>
>>
>> Regards,
>>
>> Harrissou
>>
>>
>>
>> Le dim. 31 janv. 2021 à 15:55, Jorge Gustavo Rocha <jgr at di.uminho.pt> a
>> écrit :
>>
>> Hi Stefan,
>>
>> We have now a new tab on github called 'discussions'[1]. Your detailed
>> instructions are a good candidate to write down a new entry there, under
>> 'Show and tell' category. The goal is to keep this information next to
>> the repo. You can use the markdown syntax to enhance the writing format.
>> You can also add some print screens.
>>
>> This is just a suggestion. Feel free to put it there or not.
>>
>> Regards (and thank you!),
>>
>> Jorge Gustavo
>>
>> [1] https://github.com/qgis/QGIS/discussions
>>
>> Às 14:13 de 31/01/21, Uhrig, Stefan escreveu:
>> > Install Visual Studio Community 2019: Select "Desktop development with
>> > C++" under "Workloads" and "C++ MFC for latest v142 build tools (x86 &
>> > x64)" under "Individual components". Verify that you install "Windows 10
>> > SDK (10.0.18362.0)" or higher.
>> >
>> >
>> >
>> > Install Git for Windows and Python 3.
>> >
>> >
>> >
>> > Download
>> >
>> https://sourceforge.net/projects/winflexbison/files/win_flex_bison3-latest.zip/download
>> > and extract it to a location of your choice (avoid space characters in
>> > the target path).
>> >
>> >
>> >
>> > Follow the vcpkg "Quick Start: Windows" instructions at
>> > https://github.com/microsoft/vcpkg#quick-start-windows. Make sure to
>> > execute the "vcpkg integrate install" command.
>> >
>> >
>> >
>> > Install the required dependencies via:
>> >
>> >
>> >
>> > vcpkg --triplet=x64-windows install exiv2 gdal gsl libspatialindex
>> > libspatialite libzip opencl protobuf qca qscintilla qt5 qt5-serialport
>> > qt5-location qt5-winextras qtkeychain qwt zstd
>> >
>> >
>> >
>> > This may take a while as this creates a Debug and a Release build of
>> > each dependency from source.
>> >
>> >
>> >
>> > Clone the QGIS repostiory. The path to the QGIS repository should not
>> > contain any space characters.
>> >
>> >
>> >
>> > Start Visual Studio and open the QGIS CMakeLists.txt file via "File -->
>> > Open --> CMake...". This will start the CMake generation, which will
>> > fail. You will need several configuration rounds until everything is
>> > setup properly.
>> >
>> >
>> >
>> > Let's start with a Release build because that requires less tweaking. Go
>> > to "Project --> CMake Settings". Delete the default configuration
>> > ("x64-Debug (default)").
>> >
>> >
>> >
>> > Add a new "x64-Release" configuration and set "Configuration type" to
>> > "Release". Save the CMakeSettings.json file, which will start a new
>> > CMake generation. "FIND_FLEX" will fail, so you need to set the
>> > "FLEX_EXECUTABLE" path manually in section "CMake variables and cache".
>> > Browse to your win_flex.exe file. Additionally, disable "WITH_BINDINGS"
>> > and "WITH_QGIS_PROCESS". Save the file. Next set "BISON_EXECUTABLE" to
>> > your win_bison.exe file and save again. Disable "WITH_QTWEBKIT" and
>> > save. Set "QCA_LIBRARY" to
>> > "<vcpkg-root>\installed\x64-windows\lib\qca.lib" and save. Configuration
>> > should now succeed, but you have to change "GEOS_LIBRARY" from
>> > "geos.lib" to "geos_c.lib". Save and start the build after CMake
>> > generation has finished.
>> >
>> >
>> >
>> > Select "qgis.exe (output\bin\qgis.exe)" as start-up item in the toolbar
>> > and start QGIS.
>> >
>> >
>> >
>> > Now let's do the same for a Debug build. Add a "x64-Debug" configuration
>> > and save. Change to the "x64-Debug" configuration in the toolbar. Repeat
>> > the steps from above (set flex and bison paths etc.). When selecting
>> > libraries, take the libraries from
>> > "<vcpkg-root>\installed\x64-windows\debug\lib". Most libraries will have
>> > a "d" or "_d" as name suffix. The CMake generation will not always
>> > select the debug versions of libraries. Hence, go over the CMake
>> > variables and change release library paths to debug library paths, e.g.
>> > "<vcpkg-root>\installed\x64-windows\debug\lib\geos_cd.lib" for
>> > GEOS_LIBRARY. I'll append my CMakeSettings.json file as reference to
>> > this mail. Save again and start the build.
>> >
>> >
>> >
>> > The build will fail because the execution of crssync.exe fails. That's
>> > because crssync.exe tries to load zstd.dll instead of zstdd.dll. I
>> > haven't figured out why, yet. An easy workaround is copying zstdd.dll to
>> > zstd.dll in QGIS\out\build\x64-Debug\output\bin. Then restart the build
>> > and it should succeed.
>> >
>> >
>> >
>> > You should then be able to start and debug qgis.exe and its
>> > dependencies. Note that some vcpkg portfiles forget to copy over the
>> > .pdb files. For example, proj is affected. If you find such a library
>> > with a missing .pdb file, check whether the portfile (e.g.
>> > "<vcpkg-root>\ports\proj4\portfile.cmake") contains the
>> > "vcpkg_copy_pdbs()" call. If not, just add it at the end, re-install the
>> > dependency and copy over the .pdb file to the QGIS output directory.
>> >
>> >
>> >
>> > I'm currently trying to get QGIS working with Python bindings. I got the
>> > build working, however I'm struggling with the debug build execution.
>> > The debug Python library seems to be unable to load the Release build Qt
>> > libraries that come with PyQt5. Well, I'll keep trying and if I succeed,
>> > I'll update the instructions.
>> >
>> >
>> >
>> > Here is my CMakeSettings.json file for reference:
>> >
>> >
>> >
>> > {
>> >
>> > "configurations": [
>> >
>> > {
>> >
>> > "name": "x64-Release",
>> >
>> > "generator": "Ninja",
>> >
>> > "configurationType": "Release",
>> >
>> > "buildRoot": "${projectDir}\\out\\build\\${name}",
>> >
>> > "installRoot": "${projectDir}\\out\\install\\${name}",
>> >
>> > "cmakeCommandArgs": "",
>> >
>> > "buildCommandArgs": "",
>> >
>> > "ctestCommandArgs": "",
>> >
>> > "inheritEnvironments": [ "msvc_x64_x64" ],
>> >
>> > "variables": [
>> >
>> > {
>> >
>> > "name": "WITH_BINDINGS",
>> >
>> > "value": "False",
>> >
>> > "type": "BOOL"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "WITH_QGIS_PROCESS",
>> >
>> > "value": "False",
>> >
>> > "type": "BOOL"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "FLEX_EXECUTABLE",
>> >
>> > "value": "C:/src/tools/win_flex.exe",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "BISON_EXECUTABLE",
>> >
>> > "value": "C:/src/tools/win_bison.exe",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "WITH_QTWEBKIT",
>> >
>> > "value": "False",
>> >
>> > "type": "BOOL"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "QCA_LIBRARY",
>> >
>> > "value": "C:/src/vcpkg/installed/x64-windows/lib/qca.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "GEOS_LIBRARY",
>> >
>> > "value": "C:/src/vcpkg/installed/x64-windows/lib/geos_c.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > }
>> >
>> > ]
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "x64-Debug",
>> >
>> > "generator": "Ninja",
>> >
>> > "configurationType": "Debug",
>> >
>> > "buildRoot": "${projectDir}\\out\\build\\${name}",
>> >
>> > "installRoot": "${projectDir}\\out\\install\\${name}",
>> >
>> > "cmakeCommandArgs": "",
>> >
>> > "buildCommandArgs": "",
>> >
>> > "ctestCommandArgs": "",
>> >
>> > "inheritEnvironments": [ "msvc_x64_x64" ],
>> >
>> > "variables": [
>> >
>> > {
>> >
>> > "name": "FLEX_EXECUTABLE",
>> >
>> > "value": "C:/src/tools/win_flex.exe",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "BISON_EXECUTABLE",
>> >
>> > "value": "C:/src/tools/win_bison.exe",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "WITH_QTWEBKIT",
>> >
>> > "value": "False",
>> >
>> > "type": "BOOL"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "WITH_BINDINGS",
>> >
>> > "value": "False",
>> >
>> > "type": "BOOL"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "WITH_QGIS_PROCESS",
>> >
>> > "value": "False",
>> >
>> > "type": "BOOL"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "QCA_LIBRARY",
>> >
>> > "value":
>> "C:/src/vcpkg/installed/x64-windows/debug/lib/qcad.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "GEOS_LIBRARY",
>> >
>> > "value":
>> > "C:/src/vcpkg/installed/x64-windows/debug/lib/geos_cd.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "GSL_LIB",
>> >
>> > "value":
>> "C:/src/vcpkg/installed/x64-windows/debug/lib/gsld.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "GSLCBLAS_LIB",
>> >
>> > "value":
>> > "C:/src/vcpkg/installed/x64-windows/debug/lib/gslcblasd.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "PROJ_LIBRARY",
>> >
>> > "value":
>> > "C:/src/vcpkg/installed/x64-windows/debug/lib/proj_d.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "QSCINTILLA_LIBRARY",
>> >
>> > "value":
>> > "C:/src/vcpkg/installed/x64-windows/debug/lib/qscintilla2_qt5d.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "QTKEYCHAIN_LIBRARY",
>> >
>> > "value":
>> > "C:/src/vcpkg/installed/x64-windows/debug/lib/qt5keychaind.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "QWT_LIBRARY",
>> >
>> > "value":
>> "C:/src/vcpkg/installed/x64-windows/debug/lib/qwtd.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > },
>> >
>> > {
>> >
>> > "name": "SPATIALINDEX_LIBRARY",
>> >
>> > "value":
>> > "C:/src/vcpkg/installed/x64-windows/debug/lib/spatialindex-64d.lib",
>> >
>> > "type": "FILEPATH"
>> >
>> > }
>> >
>> > ]
>> >
>> > }
>> >
>> > ]
>> >
>> > }
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > *From:*i-s-o <46.i.s.o.64 at gmail.com>
>> > *Sent:* Saturday, January 30, 2021 5:58 PM
>> > *To:* Uhrig, Stefan <stefan.uhrig at sap.com>
>> > *Cc:* qgis-developer <QGIS-Developer at lists.osgeo.org>
>> > *Subject:* Re: [QGIS-Developer] Building QGIS with Visual Studio 2019 CE
>> > and vcpkg
>> >
>> >
>> >
>> > I am very interested in trying out your solution. Could you share the
>> > required steps?
>> >
>> >
>> >
>> > Thx.
>> >
>> >
>> >
>> > On Fri, Jan 29, 2021, 09:25 Uhrig, Stefan <stefan.uhrig at sap.com
>> > <mailto:stefan.uhrig at sap.com>> wrote:
>> >
>> > TL;DR: It is currently possible to build the QGIS core app with
>> > Visual Studio 2019 and vcpkg, which makes debugging QGIS
>> > dependencies easy.
>> >
>> >
>> >
>> >
>> >
>> > Hi all,
>> >
>> >
>> >
>> > Some time ago I discovered vcpkg
>> > (https://github.com/microsoft/vcpkg). vcpkg is a package manager
>> > that downloads package source code to your local machine and builds
>> > the package locally. Recently, I discovered that vcpkg should be
>> > able to provide all dependencies to build at least the QGIS core
>> > application. Hence, I gave it a try.
>> >
>> >
>> >
>> > Basically, it worked out of the box. I started with a fresh Windows
>> > 10 installation, installed Visual Studio 2019 Community Edition,
>> > Git, vcpkg, Python 3 and flex and bison for Windows. I fetched all
>> > other dependencies via vcpkg. It was not necessary to even touch a
>> > single file in the repository. I could just open the main
>> > CMakeLists.txt file in Visual Studio and only had to tweak the CMake
>> > cache (the CMake find macros that come with QGIS are not aware of
>> > vcpkg, so I had to set some paths manually). I had to switch off
>> > some extensions though as the required dependencies were not
>> > available via vcpkg (WITH_BINDINGS, WITH_QGIS_PROCESS,
>> > WITH_QTWEBKIT). The build did not report any errors, I could start
>> > the application and it seems to work, but I did some light testing
>> only.
>> >
>> >
>> >
>> > I mainly tried it because I enjoy debugging with Visual Studio more
>> > than with gdb (or gdb wrapped in some IDE). In my experience, the
>> > performance of the Visual Studio debugger is better and it is more
>> > stable, especially in long debug sessions.
>> >
>> >
>> >
>> > I don’t want to promote official building support of QGIS with
>> > vcpkg. Providing the dependencies via OSGeo4W is much more reliable.
>> > However, if you don’t mind the experimental nature of this setup and
>> > you want to be able to debug into QGIS’ dependencies, you might give
>> > it a try. Especially, if you want to track the cause of a crash in
>> > one of QGIS’ dependencies, this setup might be helpful. You have the
>> > source code and debug versions of the dependencies, so the debugger
>> > will jump to the crashing code line and you can inspect all the
>> > variables of the dependency.
>> >
>> >
>> >
>> > If someone is interested in trying it, give me a note. I can then
>> > assemble detailed instructions on how to make it work. It took me a
>> > while to figure out which packages are needed and how the CMake
>> > cache needs to be tweaked.
>> >
>> >
>> >
>> > Best regards,
>> >
>> > Stefan
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > QGIS-Developer at lists.osgeo.org <mailto:
>> QGIS-Developer at lists.osgeo.org>
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe:
>> https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >
>> >
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > QGIS-Developer at lists.osgeo.org
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> >
>>
>> J. Gustavo
>> --
>> Jorge Gustavo Rocha
>> Departamento de Informática
>> Universidade do Minho
>> 4710-057 Braga
>> Gabinete 3.29 (Piso 3)
>> Tel: +351 253604480
>> Fax: +351 253604471
>> Móvel: +351 910333888
>> skype: nabocudnosor
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20210228/1db843ae/attachment-0001.html>
More information about the QGIS-Developer
mailing list