[GRASS-dev] g.extension, svn, and Mac

Thu Aug 6 10:35:04 PDT 2020

Hi Nicklas,

I've been building these packages for a couple years with Anaconda and never needed to export CONDA_BUILD_SYSROOT previously. And I don't need it to build 7.8. So something has changed that requires this in 7.9.

The issue with compiling extensions may have existed previously and just no one reported it. I'm trying to check this now. This was the case with old non-inclusive package. You had to have the command line dev tools installed. The new apps are supposed to fix that by having tools needed for compiling installed inside the app.

The reason to build with Anaconda is that I wanted to create an all-inclusive binary package that would work as most Mac users expect, without having to install anything extra. This could not be done with the old (and largely obsolete) packaging tools that come with GRASS. There may be other ways to do it, but Anaconda works pretty well and is comparatively easy to manage, especially for getting all of the many dependencies that GRASS needs without me having to compile and update them from scratch.

Maybe there is no need to point to an SDK at all, as you mention. I've been doing it as a legacy from old build protocols that helped ensure that GRASS would be backward compatible with at least a few older OS versions. Maybe with everything inside, it isn't needed and just mucks things up. So I'll try building without it. If  CONDA_BUILD_SYSROOT is now required, could it point to the /Contents/Resources folder inside the app? This should have all the needed compiling tools (in /bin, /include, /lib, etc).

While we can tell Mac users that they need to install command line tools (or equivalent in Homebrew or Macports) if they want to compile some extensions, it will be better and cleaner if there is a way to make all this happen in the app.

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton, https://complexity.asu.edu/csdc'

On Aug 6, 2020, at 2:07 AM, Nicklas Larsson <n_larsson at yahoo.com<mailto:n_larsson at yahoo.com>> wrote:

Michael,

You may try without setting SDK at all (remove --with-macosx-sdk= and CONDA_BUILD_SYSROOT), but I doubt it will. CONDA_BUILD_SYSROOT is not a workaround, it’s needed for conda.

(If indeed necessary to set SDK, it might be better to use:
CONDA_BUILD_SYSROOT=$(xcrun --show-sdk-path) to use present SDK while building GRASS, but make sure to set MACOSX_DEPLOYMENT_TARGET.)

What seems to work for compiling extensions, is modifying the file:

/Applications/GRASS-7.9.app/Contents/Resources/include/Make/Platform.make

deleting the 4 instances of the text:

-isysroot /Users/cmbarton/SDKs/MacOSX10.14.sdk

I cannot however tell whether this works without Xcode.

Best,
Nicklas

On 6 Aug 2020, at 00:48, Michael Barton <Michael.Barton at asu.edu<mailto:Michael.Barton at asu.edu>> wrote:

This has to do with some kind of change in master (not in v. 7.8.4dev as of 25 July) that caused configure to fail with a bogus missing zlib message. Nicklas found a workaround by adding

export CONDA_BUILD_SYSROOT=[hard path to SDK]

to the configure script. But that is now causing problems with compiling through g.extension. So we need to find out why configure is failing.

I've filed a bug report at: https://github.com/OSGeo/grass/issues/880<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_OSGeo_grass_issues_880&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=Cf3V6jNzJPw3ozGPhFY2SqZ28HxqNLzLT7ECqdPEQdA&s=xRD0BLuZbnEJwrUrZEeNww7x58-w0Ms0rCdUDJs1_wI&e=>

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=Cf3V6jNzJPw3ozGPhFY2SqZ28HxqNLzLT7ECqdPEQdA&s=zdRYDg2IV4BWgT-rtAnRzjKUv99_edmJ14t0vyeKuj8&e=>, https://complexity.asu.edu/csdc'<https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=Cf3V6jNzJPw3ozGPhFY2SqZ28HxqNLzLT7ECqdPEQdA&s=3KjRDwjkh-clz1HkTmu2RQKbBhZWkdGazv3e-pgY08E&e=>

On Aug 5, 2020, at 3:16 PM, Michael Barton <Michael.Barton at asu.edu<mailto:Michael.Barton at asu.edu>> wrote:

It's better if the binary does not need any SDK so people can use it without installing Xcode. It really shouldn't be necessary. There should be all the tools needed for compiling within the miniconda package installed in the app. I'm pretty sure I know what line in configure is causing this. Need to see how to get around it.

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=Cf3V6jNzJPw3ozGPhFY2SqZ28HxqNLzLT7ECqdPEQdA&s=zdRYDg2IV4BWgT-rtAnRzjKUv99_edmJ14t0vyeKuj8&e=>, https://complexity.asu.edu/csdc'<https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=Cf3V6jNzJPw3ozGPhFY2SqZ28HxqNLzLT7ECqdPEQdA&s=3KjRDwjkh-clz1HkTmu2RQKbBhZWkdGazv3e-pgY08E&e=>

On Aug 5, 2020, at 3:04 PM, Steven Pawley <dr.stevenpawley at gmail.com<mailto:dr.stevenpawley at gmail.com>> wrote:

I’m on 10.15 (on a relatively new Mac) and officially Xcode only ships with the latest SDK. I think unofficially it might be possible to copy older SDKs into the Xcode app bundle but presumably not ideal.

On Aug 5, 2020, at 3:46 PM, Michael Barton <Michael.Barton at asu.edu<mailto:Michael.Barton at asu.edu>> wrote:

This error implies that you need the 10.14.sdk folder. Why would that be so? That would be a problem.

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=HdCFQffudEp5qahnvPaiTLzJsh65rn-5Ab710fbU1vY&s=fyXN3VStjHZGCQbsYN2_xQ13D7SMjIs8kHrGU5a2_jY&e=>, https://complexity.asu.edu/csdc'<https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=HdCFQffudEp5qahnvPaiTLzJsh65rn-5Ab710fbU1vY&s=m8TFRpKA72VXur99soNzYcxoGw5_Ss7Q1bEWk22hE6U&e=>

On Aug 5, 2020, at 2:39 PM, Steven Pawley <dr.stevenpawley at gmail.com<mailto:dr.stevenpawley at gmail.com>> wrote:

Hi Michael,

Thanks for updating. I can still install python-based extensions without any error, however I still get the same error for compiled code:

g.extension extension=r.fill.gaps
Fetching <r.fill.gaps> from GRASS GIS Addons repository (be patient)...
Compiling...
clang-10: warning: no such sysroot directory:
'/Users/cmbarton/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
cell_funcs.c:21:10: fatal error: 'math.h' file not found
#include <math.h>
         ^~~~~~~~
1 error generated.
make: *** [OBJ.x86_64-apple-darwin19.6.0/cell_funcs.o] Error
1
ERROR: Compilation failed, sorry. Please check above error messages.

Steve

On Aug 5, 2020, at 2:00 PM, Michael Barton <Michael.Barton at asu.edu<mailto:Michael.Barton at asu.edu>> wrote:

Steve,

I just now posted a new updated GRASS 7.9dev that includes svn in the app. It solved the g.extension problem for me. Can you test and see if it solves it for you too?

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=bm8GNsnlGrRMgk5Aw-PILxKCHgti6ue1ipDNLN5bS8o&e=>, https://complexity.asu.edu/csdc'<https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=h-op3BcIQNfbjiQKiuUA4VsQRB09LrvWPs0cx0vmmAc&e=>

On Aug 4, 2020, at 11:59 AM, Michael Barton <Michael.Barton at asu.edu<mailto:Michael.Barton at asu.edu>> wrote:

I'm pretty sure this happened when I installed developer tools 11.5

I want to make sure that anyone using the binaries can do so without having to separately install any dependencies. So I'll see what happens if I just include svn in the app. I will need to test on a system that suffers from this lack.

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=bm8GNsnlGrRMgk5Aw-PILxKCHgti6ue1ipDNLN5bS8o&e=>, https://complexity.asu.edu/csdc'<https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=2Gr42Gr6-mCS6vSIKkL1gNxGKhgP9akeCkWc5-eOGZ8&s=h-op3BcIQNfbjiQKiuUA4VsQRB09LrvWPs0cx0vmmAc&e=>

On Aug 4, 2020, at 11:56 AM, Steven Pawley <dr.stevenpawley at gmail.com<mailto:dr.stevenpawley at gmail.com>> wrote:

Hi Michael,

Right, I do have svn installed via Homebrew (I get  'Type 'svn help' for usage.’), so that probably explains why g.extension works for me overall, apart from the issue with finding the MacOS SDK.

Steve

On Aug 4, 2020, at 11:27 AM, Michael Barton <Michael.Barton at asu.edu<mailto:Michael.Barton at asu.edu>> wrote:

Steven,

What happens if you open a terminal window and type 'svn' followed by a return?

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=3OtUJxnVDVfConnURZzc-TyoKzTbZWrxoF7zL8E2qZ0&s=6KybHpYJMo7V86xk_1VxAmD-_UolxXaQfLvYShRFgpk&e=>, https://complexity.asu.edu/csdc'<https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=3OtUJxnVDVfConnURZzc-TyoKzTbZWrxoF7zL8E2qZ0&s=abWZQP3dzmM5KTmBUXPwvqbzQUlMgfhQzUGEr0HevTo&e=>

On Aug 3, 2020, at 6:53 PM, Steven Pawley <dr.stevenpawley at gmail.com<mailto:dr.stevenpawley at gmail.com>> wrote:

Perhaps a different issue,  but I can install extensions, but only pure python-based extensions. Anything that requires compilation of C code fails.

For example, when using the latest GRASS GIS binary (7.9, 25th July 2020 build), this works with r.texture.tiled:

Fetching <r.texture.tiled> from GRASS GIS Addons repository (be patient)...
Compiling...
Installing...
Updating extensions metadata file...
Updating extension modules metadata file...
Installation of <r.texture.tiled> successfully finished

But for example, r.fill.gaps does not:

g.extension extension=r.fill.gaps
Fetching <r.fill.gaps> from GRASS GIS Addons repository (be patient)...
Compiling...
clang-10: warning: no such sysroot directory:
'/Users/cmbarton/SDKs/MacOSX10.14.sdk' [-Wmissing-sysroot]
cell_funcs.c:21:10: fatal error: 'math.h' file not found
#include <math.h>
         ^~~~~~~~
1 error generated.
make: *** [OBJ.x86_64-apple-darwin19.5.0/cell_funcs.o] Error
1
ERROR: Compilation failed, sorry. Please check above error messages.

The 7.9-dev build throws an error because it appears to be still looking for components based on your system, specifically.,The 7.8.4 binary (16th July 2020 build) also throws the same error. I’m running the MacOS 10.15.6.

Steve

On Aug 2, 2020, at 4:44 PM, Michael Barton <Michael.Barton at asu.edu<mailto:Michael.Barton at asu.edu>> wrote:

I just learned yesterday, when I tried to install something from g.extension that:

1. g.extension uses svn to download packages, and
2. Apple stopped including svn with the current OS (10.15.x) and its developer tools

I don't know if anyone else has run into this yet, but it could be an increasingly sticky issue going forward. Unless there are plans to change how g.extension gets packages, I suppose I need to include it in the Mac app bundle.

Has anyone else hit this yet?

Michael
____________________
C. Michael Barton
Director, Center for Social Dynamics & Complexity
Director, Network for Computational Modeling in Social & Ecological Sciences
Professor of Anthropology, School of Human Evolution & Social Change
Head, Graduate Faculty in Complex Adaptive Systems Science
Arizona State University

voice:  480-965-6262 (SHESC), 480-965-8130/727-9746 (CSDC)
fax: 480-965-7671 (SHESC),  480-727-0709 (CSDC)
www: 'http://www.public.asu.edu/~cmbarton<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.public.asu.edu_-7Ecmbarton&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=esHRRZODAbgeKk6vYVpJkYSszGdc2vYZgvM7W5jtUjk&s=wLo54OaOggTCfa9dMnza-vU7d8tllnb8G9z7MW3tF38&e=>, https://complexity.asu.edu/csdc'<https://urldefense.proofpoint.com/v2/url?u=https-3A__complexity.asu.edu_csdc-27&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=esHRRZODAbgeKk6vYVpJkYSszGdc2vYZgvM7W5jtUjk&s=-cLS2YOXOnAMIbnQqtJQAbmXsjh9CVUVr2PzI6yj6FE&e=>

_______________________________________________
grass-dev mailing list
grass-dev at lists.osgeo.org<mailto:grass-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/grass-dev<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.osgeo.org_mailman_listinfo_grass-2Ddev&d=DwMFaQ&c=l45AxH-kUV29SRQusp9vYR0n1GycN4_2jInuKy6zbqQ&r=lk-7X7CEOMDN8GaGVhiDsuO6gEp1wbG6nfT1XEEEtR0&m=3OtUJxnVDVfConnURZzc-TyoKzTbZWrxoF7zL8E2qZ0&s=4UVbmUdiztW2twuvUPIEUWiez0RLGyHRTdL0TV5DVfw&e=>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20200806/cd3ba9bb/attachment-0001.html>