[PROJ] PROJ 9.4.1RC1 & PROJ-data 1.18RC1

[Greg Troxel]

Mike Taves <mwtoews at gmail.com> writes:

> Thanks for testing and reporting clear feedback. I've added a few
> comments inline:
>
> On Tue, 28 May 2024 at 11:36, Greg Troxel via PROJ <proj at lists.osgeo.org> wrote:
>>   -- Could NOT find GTest (missing: GTEST_LIBRARY GTEST_INCLUDE_DIR GTEST_MAIN_LIBRARY) (Required is at least version "1.8.1")
>>   -- Fetching GTest
>>   -- PROJ: Configured 'dist' target
>
> There is an error after "Fetching GTest" which is not shown. For the
> record, the relevant changes are from:
> https://github.com/OSGeo/PROJ/pull/4006
>
> I'm not able to replicate this with various Docker images (it's not
> easy to switch off the network). What version of cmake is this with?

wow, that's interesting that it isn't easy to turn off the network.
That is very surprising to me (not a docker user) as it seems many
people want to isolate things for security purposes.

3.29.3

But pkgsrc has proxy variables intended to make wget/curl/etc. fail
during the build phase.  However we are not doing anything that I know
of to hide errors.  I am having trouble reading the code tonight, and
it's also possible that it isn't that, but rather curl not being in the
path because it was not declared as a build tool.  (We try hard to hide
everything that is not a declared dependency, as part of getting
repeatable builds.)

Not really a big deal and not the problem, but dependencies need to be
specified as being available for the target (to include/link against)
and for the host (to be used in the build) so that cross building works.

>>   - proj seems wrong to error out without GTest, given the install docs.
>
> The behaviour should disable tests if the minimum GTest is not
> available, showing non-fatal error messages. It should allow it to be
> built, without test capabilities.

Hmm, well that is not what happened to me.  Makes sense that it is the
plan.  I'll dig in.

>>   - it's a bug to try to use the network at build time.  I'm not sure
>>     what "Fetching GTest" is but it looks like that's what's happening.
>>     pkgsrc arranges for proxy variables to prevent this from working.
>
> Is there a general consensus if "tests should not use the network at
> build time" is a bug or not? There is a related and undocumented
> "RUN_NETWORK_DEPENDENT_TESTS" configuration:
> https://github.com/OSGeo/PROJ/blob/3b7a6a9b2e5ca9441cd589f552d64a9ec196678f/test/CMakeLists.txt#L3

This all really depends on whether a person
  - thinks software should work without the network, unless it is trying
    to use it
  - cares about repeatability across environments
  - cares about security
vs
  - wanting things to "work" anyway for people that have systems set
    up wrong, who lack the skills to install dependencies, or who are
    using platforms where it's super hard


My biases come from packaging, from a professional focus on security,
and having worked on a group of networked computers that were
disconnected from the internet for security reasons, where distribution
tarballs were brought in on CD.  People with these sorts of environments
tend not to say much about them :-)

So from this viewpoint

  it is fine to expect the release tarball to be present

  it is fine to declare that prereqs must be installed

  it is a bug to try to use the network at all during the configure,
  build, or install phases, period.

  for tests, I can understand that there might be some tests that use
  the network, because the program in normal use uses the network.  By
  default, there should be no use of the network, as that violates a
  security expectation that the tests run within the build tree, and
  would fail if there is no network or it is heavily firewalled.  It is
  certainly fine to document that if one has a network where one can do
  X, then one can RUN_NETWORK_DEPENDENT_TESTS=ON and they'll be run.

> Should there be a CMake variable to control if a network should be
> available or not?

I think network tests should be optional and default off, so the
existing variable is fine.

I do not see any reason to use the network at configure, build or
install time.  (To address the elephant, I'm not ncounting "user might
not have prereqs and user would not be able to install them because they
are running windows" as a good reason.)   So no variables about
configure/build/install network use are needed.

>>   - it doesn't make sense to printing out that GTest was not found
>>     (which is fine if optional) and then try to use it.
>
> Agree, but I'll need to sort-out how to replicate this behaviour.

I'm very glad to hear you think it's a bug.  I'll see what I can figure
out.  I hadn't tried to turn up verbosity yet.

>>   - even if the docs are wrong, it should be specified as a test
>>     dependency, not a regular build dependency.  Packaging systems are
>>     careful to minimize dependencies
>>
>>   - not really related, but it found nlohmann-json which was not
>>     documented as a dependency.   In pkgsrc, as I would expect in every
>>     other system (unless they *only* build in chroot) to need, for each
>>     possible dependency, to either provide it or to configure it off
>
> Also good suggestions. The docs should be expanded to list the test
> dependencies (after the build dependencies). I can sort-out these doc
> enhancements.
>
> Are you able to re-test with googletest installed?
> https://pkgsrc.se/devel/googletest

Yes.  It was just adding to the Makefile to depend on it:

  # Dependencies not documented by upstream
  #  reported 20240527
  .include "../../devel/googletest/buildlink3.mk"

and then tests start.

They all failed because the test programs were trying to link with
proj.so.25 but I have 8.2.1 and hence .22 installed.

  Start testing: May 27 21:45 EDT
  ----------------------------------------------------------
  1/61 Testing: geodesic-test
  1/61 Test: geodesic-test
  Command: "/tmp/work/wip/proj/work/proj-9.4.1/cmake-pkgsrc-build/bin/geodtest"
  Directory: /tmp/work/wip/proj/work/proj-9.4.1/cmake-pkgsrc-build/src/tests
  "geodesic-test" start time: May 27 21:45 EDT
  Output:
  ----------------------------------------------------------
  /tmp/work/wip/proj/work/proj-9.4.1/cmake-pkgsrc-build/bin/geodtest: Shared object "libproj.so.25" not found
  <end of output>
  Test time =   0.00 sec
  ----------------------------------------------------------
  Test Failed.
  "geodesic-test" end time: May 27 21:45 EDT
  "geodesic-test" time elapsed: 00:00:00

I know there are perhaps some cmake RPATH issues, where pkgsrc trying to
control the build may be at cross purposes to what a package is trying
to do.  This used to work well with autoconf/libtool.

Do you believe that the tests link against and use the just-built
libraries in the build tree?  I think they should.  If that's generally
the case I'll sort this into a pkgsrc-cmake-support bug for now.