[gdal-dev] Build GDAL 3.5 for iOS error: forward declaration of 'stat64'
Nik Sands
nik at nixanz.com
Tue Jul 5 16:29:18 PDT 2022
Thanks Even,
After applying the patch and then running the ‘cmake' and 'cmake —build' again, it stops at the same point but produces much fewer errors (and I have found a dodgy work-around those - see below). Only two errors this time, both for the same file:
==========
[ 6%] Building CXX object port/CMakeFiles/cpl.dir/cpl_compressor.cpp.o
[ 7%] Building CXX object port/CMakeFiles/cpl.dir/cpl_float.cpp.o
[ 7%] Building CXX object port/CMakeFiles/cpl.dir/cpl_vsil_unix_stdio_64.cpp.o
/Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:793:22: error: variable has incomplete type 'struct statvfs64'
struct statvfs64 buf;
^
/Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:793:12: note: forward declaration of 'statvfs64'
struct statvfs64 buf;
^
1 error generated.
make[2]: *** [port/CMakeFiles/cpl.dir/cpl_vsil_unix_stdio_64.cpp.o] Error 1
make[1]: *** [port/CMakeFiles/cpl.dir/all] Error 2
make: *** [all] Error 2
==========
So the patch has fixed the vast majority of the errors, but this one still persists.
DODGY WORK AROUND:
I then modified the file 'port/cpl_vsil_unix_stdio_64.cpp’ and deleted the ‘if' section for ‘HAVE_STATVFS64', forcing it to run the stavfs ‘else’ part instead, and this time the build continued on (as far as the “98%” mark before producing another error which I might post seperately if I can’t resolve it). Obviously this is not the ‘correct’ way to fix the issue, but I hope that it demonstrates (part of) where the issue lies and hopefully might help to identify the correct solution?
So for some reason, when the 'port/cpl_vsil_unix_stdio_64.cpp’ file is used, it appears that HAVE_STATVFS64 is defined, despite the patch having been applied.
Following the patch, my cmake/helpers/configure.cmake file shows that the patch has been applied. The relevent portion of that file now looks like this:
==========
check_function_exists(ftruncate64 HAVE_FTRUNCATE64)
if (HAVE_FTRUNCATE64)
set(VSI_FTRUNCATE64 "ftruncate64")
else ()
set(VSI_FTRUNCATE64 "ftruncate")
endif ()
if (${CMAKE_SYSTEM_NAME} MATCHES "iOS")
set(VSI_FOPEN64 "fopen")
set(VSI_FTRUNCATE64 "ftruncate")
set(VSI_FTELL64 "ftell")
set(VSI_FSEEK64 "fseek")
set(VSI_STAT64 stat)
set(VSI_STAT64_T stat)
unset(HAVE_FOPEN64)
unset(HAVE_FTRUNCATE64)
unset(HAVE_FTELL64)
unset(HAVE_FSEEK64)
unset(HAVE_STATVFS64)
endif()
set(UNIX_STDIO_64 TRUE)
set(INCLUDE_XLOCALE_H)
if(HAVE_XLOCALE_H)
set(INCLUDE_XLOCALE_H "#include <xlocale.h>")
endif()
==========
Cheers,
Nik.
> On 5 Jul 2022, at 9:23 pm, Even Rouault <even.rouault at spatialys.com> wrote:
>
> (re-adding the list)
>
> ok, it's a bit weird CMake detects symbols that are not available at build time, but there might be some subtelties in iOS SDK
>
> Can you try the following patch which basically forces to remap all "foo64" functions to "foo". I assume that iOS Unix I/O is 64-bit enabled by default...
>
> diff --git a/cmake/helpers/configure.cmake b/cmake/helpers/configure.cmake
> index c38604e23e..e5be264bf9 100644
> --- a/cmake/helpers/configure.cmake
> +++ b/cmake/helpers/configure.cmake
> @@ -241,6 +241,20 @@ else ()
> set(VSI_FTRUNCATE64 "ftruncate")
> endif ()
>
> + if (${CMAKE_SYSTEM_NAME} MATCHES "iOS")
> + set(VSI_FOPEN64 "fopen")
> + set(VSI_FTRUNCATE64 "ftruncate")
> + set(VSI_FTELL64 "ftell")
> + set(VSI_FSEEK64 "fseek")
> + set(VSI_STAT64 stat)
> + set(VSI_STAT64_T stat)
> + unset(HAVE_FOPEN64)
> + unset(HAVE_FTRUNCATE64)
> + unset(HAVE_FTELL64)
> + unset(HAVE_FSEEK64)
> + unset(HAVE_STATVFS64)
> + endif()
> +
> set(UNIX_STDIO_64 TRUE)
>
> set(INCLUDE_XLOCALE_H)
>
>
>
>
> Le 05/07/2022 à 13:07, Nik Sands a écrit :
>> Hi Even,
>>
>> Thanks again for your advice. The debug message did produce what I believe was the expected output as follows:
>>
>> CMake Warning at cmake/helpers/configure.cmake:175 (message):
>> CMAKE_SYSTEM_NAME=iOS
>>
>> Making the suggested change to the configure.cmake file (line 175 in the version 3.5 distribution I have) and re-building certainly progresses further than my previous build attempt. I now get the errors in the ouput listed below. Note that I found one other location in the same configure.cmake file where it was checking for “Darwin” and I made the same change to that line, but it made no difference to these errors below.
>>
>> Cheers,
>> Nik.
>>
>>
>> ==========
>> …
>> …
>> [ 7%] Building CXX object port/CMakeFiles/cpl.dir/cpl_vsil_unix_stdio_64.cpp.o
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:139:23: error: use of undeclared identifier 'ftell64'
>> static_assert( sizeof(VSI_FTELL64(nullptr)) == sizeof(vsi_l_offset),
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:152:21: note: expanded from macro 'VSI_FTELL64'
>> #define VSI_FTELL64 ftell64
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:315:25: error: use of undeclared identifier 'fseek64'
>> const int nResult = VSI_FSEEK64( fp, nOffsetIn, nWhence );
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:149:21: note: expanded from macro 'VSI_FSEEK64'
>> #define VSI_FSEEK64 fseek64
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:355:25: error: use of undeclared identifier 'ftell64'
>> m_nOffset = VSI_FTELL64( fp );
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:152:21: note: expanded from macro 'VSI_FTELL64'
>> #define VSI_FTELL64 ftell64
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:422:13: error: use of undeclared identifier 'fseek64'
>> if( VSI_FSEEK64( fp, m_nOffset, SEEK_SET ) != 0 )
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:149:21: note: expanded from macro 'VSI_FSEEK64'
>> #define VSI_FSEEK64 fseek64
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:456:35: error: use of undeclared identifier 'ftell64'
>> vsi_l_offset nNewOffset = VSI_FTELL64( fp );
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:152:21: note: expanded from macro 'VSI_FTELL64'
>> #define VSI_FTELL64 ftell64
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:484:13: error: use of undeclared identifier 'fseek64'
>> if( VSI_FSEEK64( fp, m_nOffset, SEEK_SET ) != 0 )
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:149:21: note: expanded from macro 'VSI_FSEEK64'
>> #define VSI_FSEEK64 fseek64
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:532:12: error: use of undeclared identifier 'ftruncate64'; did you mean 'ftruncate'?
>> return VSI_FTRUNCATE64( fileno(fp), nNewSize );
>> ^~~~~~~~~~~~~~~
>> ftruncate
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:146:25: note: expanded from macro 'VSI_FTRUNCATE64'
>> #define VSI_FTRUNCATE64 ftruncate64
>> ^
>> /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk/usr/include/unistd.h:611:6: note: 'ftruncate' declared here
>> int ftruncate(int, off_t);
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:645:16: error: use of undeclared identifier 'fopen64'; did you mean 'fopen'?
>> FILE *fp = VSI_FOPEN64( pszFilename, pszAccess );
>> ^~~~~~~~~~~
>> fopen
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:143:21: note: expanded from macro 'VSI_FOPEN64'
>> #define VSI_FOPEN64 fopen64
>> ^
>> /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS15.5.sdk/usr/include/stdio.h:153:7: note: 'fopen' declared here
>> FILE *fopen(const char * __restrict __filename, const char * __restrict __mode) __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fopen));
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:793:22: error: variable has incomplete type 'struct statvfs64'
>> struct statvfs64 buf;
>> ^
>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsil_unix_stdio_64.cpp:793:12: note: forward declaration of 'statvfs64'
>> struct statvfs64 buf;
>> ^
>> 9 errors generated.
>> make[2]: *** [port/CMakeFiles/cpl.dir/cpl_vsil_unix_stdio_64.cpp.o] Error 1
>> make[1]: *** [port/CMakeFiles/cpl.dir/all] Error 2
>> make: *** [all] Error 2
>> ==========
>>
>>
>>
>>> On 5 Jul 2022, at 5:00 pm, Even Rouault <even.rouault at spatialys.com <mailto:even.rouault at spatialys.com>> wrote:
>>>
>>> Nik,
>>>
>>> in cmake/helpers/configure.cmake, there's a special case of Darwin at line 183 in master
>>>
>>> if (${CMAKE_SYSTEM_NAME} MATCHES "Darwin")
>>>
>>> Can you test if replacing it with
>>>
>>> if (${CMAKE_SYSTEM_NAME} MATCHES "Darwin" OR ${CMAKE_SYSTEM_NAME} MATCHES "iOS")
>>>
>>> works better? (I assume CMAKE_SYSTEM_NAME = iOS from https://cmake.org/cmake/help/latest/variable/IOS.html <https://cmake.org/cmake/help/latest/variable/IOS.html> , but you might want to add a 'message(WARNING "CMAKE_SYSTEM_NAME=${CMAKE_SYSTEM_NAME}")' debug statement to check
>>>
>>> Even
>>>
>>> Le 05/07/2022 à 06:14, Nik Sands a écrit :
>>>> Hi GDAL devs,
>>>>
>>>> As per my earlier emails, I’m attempting to build GDAL 3.5 for iOS. The complete process (so far) for this is below, at the end of this email.
>>>>
>>>> A quick summary of some relevant points is:
>>>> • Using a 3rd party cmake toolchain file which caters for iOS, macOS (as well as other Apple OSs) and can even build a fat binary that works for both iOS device and simulator (arm64/x86_64).
>>>> • Using a 3rd party mirror of SQLite which includes cmake configuration, so that I can incorporate the same methodology to build iOS SQLite that I’m using for other dependencies (the standard Apple bundled SQLite throws an error when building GDAL)
>>>>
>>>> The 'cmake <options> ..’ succeeds to configure the GDAL build. However, when I run ‘cmake —build .’ I get the following output:
>>>>
>>>> ==========
>>>> [ 0%] Built target generate_gdal_version_h
>>>> [ 0%] Building CXX object apps/CMakeFiles/appslib.dir/gdalinfo_lib.cpp.o
>>>> [ 0%] Building CXX object apps/CMakeFiles/appslib.dir/gdalbuildvrt_lib.cpp.o
>>>> [ 1%] Building CXX object apps/CMakeFiles/appslib.dir/gdal_grid_lib.cpp.o
>>>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/apps/gdal_grid_lib.cpp:1213:26: error: variable has incomplete type 'VSIStatBufL' (aka 'stat64')
>>>> VSIStatBufL sStat;
>>>> ^
>>>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/port/cpl_vsi.h:195:16: note: forward declaration of 'stat64'
>>>> typedef struct VSI_STAT64_T VSIStatBufL;
>>>> ^
>>>> /Users/nsands/Documents/Development/3rdParty/GDAL3/gdal-3.5.0/build/port/cpl_config.h:41:22: note: expanded from macro 'VSI_STAT64_T'
>>>> #define VSI_STAT64_T stat64
>>>> ^
>>>> 1 error generated.
>>>> make[2]: *** [apps/CMakeFiles/appslib.dir/gdal_grid_lib.cpp.o] Error 1
>>>> make[1]: *** [apps/CMakeFiles/appslib.dir/all] Error 2
>>>> make: *** [all] Error 2
>>>> ==========
>>>>
>>>> I found a GDAL bug that appears to be related, but it was closed as ‘fixed’ about 8 years ago:
>>>> https://trac.osgeo.org/gdal/ticket/1005 <https://trac.osgeo.org/gdal/ticket/1005>
>>>>
>>>> This may also be relevant:
>>>> https://www.spinics.net/lists/dash/msg02117.html <https://www.spinics.net/lists/dash/msg02117.html>
>>>>
>>>> But this is somewhat beyond my (lack of) expertise to resolve. Does anybody have any suggestions for resolving this issue?
>>>>
>>>> Cheers,
>>>> Nik.
>>>>
>>>>
>>>> COMPLETE PROCESS TO BUILD iOS GDAL (so far):
>>>>
>>>>
>>>> Install Homebrew package manager:
>>>> /usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install <https://raw.githubusercontent.com/Homebrew/install/master/install>)"
>>>>
>>>> Install cmake using Homebrew:
>>>> brew install cmakecmake --build . --target install
>>>>
>>>> Install ios.toolchain.cmake:
>>>> Download from: https://github.com/leetal/ios-cmake <https://github.com/leetal/ios-cmake>
>>>> Install at: $HOME/Documents/Development/3rdParty/ios-cmake-master/ios.toolchain.cmake
>>>>
>>>> Environment Variables:
>>>> export PREFIX=$HOME/build/arm64
>>>> export CMTOOLCHAIN=$HOME/Documents/Development/3rdParty/ios-cmake-master/ios.toolchain.cmake
>>>>
>>>> Build SQLite:
>>>> Download CMAKE-compatible SQLite amalgamation from:
>>>> https://github.com/azadkuh/sqlite-amalgamation <https://github.com/azadkuh/sqlite-amalgamation>
>>>>
>>>> cd sqlite-amalgamation-master
>>>> mkdir build
>>>> cd build
>>>> cmake -DCMAKE_TOOLCHAIN_FILE=$CMTOOLCHAIN -DPLATFORM=OS64 -DCMAKE_INSTALL_PREFIX=$PREFIX -DSQLITE_ENABLE_RTREE=ON ..
>>>> cmake --build .
>>>> cmake --build . --target install
>>>>
>>>> Build Proj:
>>>> cd proj-{VERSION}
>>>> mkdir build
>>>> cd build
>>>> cmake -DCMAKE_TOOLCHAIN_FILE=$CMTOOLCHAIN -DPLATFORM=OS64 -DCMAKE_INSTALL_PREFIX=$PREFIX -DENABLE_TIFF=OFF -DENABLE_CURL=OFF -DBUILD_PROJSYNC=OFF -DSQLITE3_INCLUDE_DIR=$PREFIX/include -DSQLITE3_LIBRARY=$PREFIX/lib/libsqlite3.a ..
>>>> cmake --build .
>>>> cmake --build . --target install
>>>>
>>>> Build GDAL:
>>>> (Specifying the path to PROJ is not required because cmake automatically adds CMAKE_INSTALL_PREFIX to CMAKE_SYSTEM_PREFIX_PATH
>>>>
>>>> cd gdal-{VERSION}
>>>> mkdir build
>>>> cd build
>>>> cmake -DCMAKE_TOOLCHAIN_FILE=$CMTOOLCHAIN -DPLATFORM=OS64 -DCMAKE_INSTALL_PREFIX=$PREFIX/arm64 -DBUILD_APPS=OFF -DBUILD_SHARED_LIBS=OFF -DBUILD_PYTHON_BINDINGS=OFF -DSQLITE3_INCLUDE_DIR=$PREFIX/include -DSQLITE3_LIBRARY=$PREFIX/lib/libsqlite3.a -DCMAKE_BUILD_TYPE=Release ..
>>>> cmake --build .
>>>> cmake --build . --target install
>>>> _______________________________________________
>>>> gdal-dev mailing list
>>>> gdal-dev at lists.osgeo.org <mailto:gdal-dev at lists.osgeo.org>
>>>> https://lists.osgeo.org/mailman/listinfo/gdal-dev <https://lists.osgeo.org/mailman/listinfo/gdal-dev>
>>>
>>> --
>>> http://www.spatialys.com <http://www.spatialys.com/>
>>> My software is free, but my time generally not.
>>>
>>
>>
>> ========================================================
>> NIK SANDS
>> Line Tamer | Time Traveller | Space Cadet
>>
> --
> http://www.spatialys.com <http://www.spatialys.com/>
> My software is free, but my time generally not.
========================================================
NIK SANDS
Line Tamer | Time Traveller | Space Cadet
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220706/4456df4e/attachment-0001.htm>
More information about the gdal-dev
mailing list