[QGIS-Developer] MacOS packaging

Sat Oct 27 10:07:50 PDT 2018

More notes -

There may be some terminology confusion here - packaging as I see it is making a distributable installation, either an installer or a simple drag-n-drop, on a disk image or zipped.

The install step of the build process is where the bundling happens.  Since this is part of the make install step, bundling needs to be done with cmake.  It can be customized per build with a user cmake script with the QGIS_MACAPP_BUNDLE_USER option, that's what I do for bundling GRASS and SAGA and a couple other extra libs.  If cmake is not enough, the user cmake can run some other script (sh or python or whatever) and pass on needed cmake variables to it.

Adding more bundling options would have to be flexible enough to not be stuck on a specific distribution of external stuff (ie I prefer the standard Python distribution, others may want Anaconda or something else).  Note that Python bundling would also have to cover extra modules needed by built-in plugins.

Packaging is not covered in the build process.  It's really a personal preference thing, but I suppose there could be an official script.  My main suggestion here is that it should be a standard Apple installer package, not a 3rd party installer system, and especially not as an executable.  Non-Apple installers may use a proprietary or obscured storage/compression system, and I know what the Apple installer executable is doing.  There are tools to extract files from packages if I want to look at the guts before installing.

Side rant - it's unfortunate that Apple is pushing people into the app store mindset.  That's not a good comparison for QGIS, since the QGIS license doesn't work for the Apple app store, and there are many external compenents of QGIS that are usable on their own and in different combinations that would be harder (and more confusing) if combined into a single app.  Also, comparing different systems is not fair either, Linux and Windows and macOS are very different.  Just because others do it shouldn't be the main reason to do something (does Linux have an "all in one" installer, or does it just seem like it because of the packaging system?).

Also note, a single "pkg" installer can still install components outside the main app, though App Store apps are restricted from this, I think.

One option could be to still have separate external components (GDAL, Python, maybe GRASS), but merge them into one mega-installer.  I think it may be possible with the Apple installer packages but it may need a 3rd-party solution or a script (Applescript or Automator might work, so it's a double-clickable app).

One more note: building for older macOS versions is a bit tricky. I'm doing some hackery to use an older SDK than is available in the Xcode on my system (on my 10.12, Xcode only has the 10.12 SDK, but I'm building for 10.10 using an old Xcode 10.10 SDK).  Building with a stock Xcode has only the latest SDK, so building for an earlier system might be a problem.

It used to be a problem, with different system versions having some different symbol names, but this was because Apple was updating their POSIX support, and I think that's all done.  I've heard that you can build for an earlier target than the SDK version now, but I haven't tried it.  And it still might be tricky if a library forces a SDK version (ie Qt 5.11 adds in a 10.11 SDK deployment target)

So, after all that, my suggestions:

- expand the current cmake bundling options to more external libraries, but leave the big items like Python (complex, many choices) and GRASS (complex, normally it's own app) to packager's user bundle scripts

- work on an actual "packaging" script - purpose is to package QGIS app (with whatever is bundled in the app) into an Apple installer package, this would be an all-in-one if user bundling includes EVERYTHING

- consider a mega-packaging script/app to handle separate external companion installers, in the case where separate external component installations are desired

> On Oct 26, 2018, at 6:53 PM, William Kyngesburye <woklist at kyngchaos.com> wrote:
> 
> I created the Mac app bundle option.  I never fully extended it to 3rd party libraries because I don't have much interest in hiding the GDAL stuff.  Rather than do something new, I think this should be expanded.
> 
> Also, there is the problem of extra GDAL formats.  I include everything non-GPL or commercial that I can in my GDAL Complete, with extra plugins for the GPL and commercial stuff - ECW, MrSID, GRASS (not completely sure about that one, GPL-wise), FileGDB, GeoPDF (Poppler).
> 
> In a full bundled QGIS, I don't think bundling ECW or MrSID would work.  So you'd still need a plugin installer (not a QGIS plugin) to add those to QGIS in some sort of global application support location like I do with GDAL now.
> 
>> On Oct 26, 2018, at 10:56 AM, Denis Rouzaud <denis.rouzaud at gmail.com <mailto:denis.rouzaud at gmail.com>> wrote:
>> 
>> Hi Peter,
>> 
>> This is great news!
>> 
>> Le ven. 26 oct. 2018 à 11:26, Peter Petrik <peter.petrik at lutraconsulting.co.uk <mailto:peter.petrik at lutraconsulting.co.uk>> a écrit :
>> Hello,
>> 
>> we have been asked to create standalone QGIS package for MacOS. By standalone I mean that there will be a single package (.pkg) file that will be extracted to /Application folder and will contain all dependencies (GDAL, Python3, PyQT, Qt libraries, ...) and will be working without any additional installation steps (similar to any application you install via App Store). 
>> 
>> As there is no such open-sourced solution I could use or enhance, I started some prototyping here: https://github.com/lutraconsulting/qgis-mac-packager <https://github.com/lutraconsulting/qgis-mac-packager> . I hope I can wrap the last bits next week and be able to produce QGIS 3.4 release and QGIS master nightlies on some Mac Cloud server. I used osgeo4mac homebrew for dependencies, since it looks like it is the most maintained package manager with osgeo libraries for MacOS. Usage of Conda packages could be better, but the number of downloads and the activity in any available repositories is not convincing.
>> 
>> Is Qt Mac Deploy of any help here?
>> http://doc.qt.io/qt-5/osx-deployment.html#macdeploy <http://doc.qt.io/qt-5/osx-deployment.html#macdeploy>
>>  
>> 
>> The aim is to eventually have QGIS bundled and shipped similar to Linux and Windows. Once we finish the work, we will send an email to the PSC and see if this is something they'd be happy to bring it under their umbrella.
>> 
>> I am open to any suggestions or cooperation for either packaging or distribution. Feel free to 
>> write me PM or reply here. Thanks
>> 
>> I'd be happy to take part of the effort. 
>> 
>> Now its time to celebrate new QGIS release during weekend! 
>> 
>> Cheers,
>> Peter
>> 
>> Note: CMAKE scripts try to achieve similar tasks (qgis/mac/cmake/*.in). But it seems to me that only bundling of Qt libraries is actively maintained [QGIS_MACAPP_BUNDLE=1]and bundling of rest of libs (gdal, libzip, geos, etc.. ) [QGIS_MACAPP_BUNDLE=2 and 3] is not implemented/maintained. Also I am not convinced that CMake scripting language is best tool for such task. (due to reconfiguration on change, syntax/readability compared to python, tools available for path handling, ...)
>> 
>> CMake syntax is not really nice to read (maybe it's personal), but the point is that all the logic is already written and accessible there, mainly in terms of dependancy and library location.
>> I think it makes sense to try to keep the logic in a single place if reasonably possible.
>> 
>> The QGIS_MACAPP_BUNDLE is indeed an non completed work from Larry. But as pointed out, he's also working on something lately.
>> 
>> More generally speaking, and even though nothing is written in QGIS source code yet, I think that this project is a very good candidate for a QEP. 
>> Do you have any intention to do one?
>> Is the decision to go for Python and not CMake definitive for your project or was it more a first quick try?
>> 
>> I hope my message did not sound too pessimistic. Moving towards a solution to get a complete package for QGIS that is openly maintained in the community is really great and I'm really glad you started this!
>> 
>> Cheers,
>> Denis
>> 
>> -- 
>> Denis Rouzaud
>> denis at opengis.ch  <mailto:denis at opengis.ch>
>> +41 76 370 21 22 <>
>> 
>>  <>_______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org <mailto:QGIS-Developer at lists.osgeo.org>
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> 
> -----
> William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
> http://www.kyngchaos.com/ <http://www.kyngchaos.com/>
> 
> "Time is an illusion - lunchtime doubly so."
> 
> - Ford Prefect
> 
> 
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

-----
William Kyngesburye <kyngchaos*at*kyngchaos*dot*com>
http://www.kyngchaos.com/

"History is an illusion caused by the passage of time, and time is an illusion caused by the passage of history."

- Hitchhiker's Guide to the Galaxy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20181027/22a9bdd0/attachment-0001.html>