PostGIS 3.7.0rc3 release

[Greg Troxel]

"Regina Obe" <lr at pcorp.us> writes:

>> There is a large amount of ctype UB.  I don't know if this is new, but I think I
>> would have noticed and said something.
>>
>> Briefly, ctype functions take an int, and are defined only for values that are
>> valid for unsigned char or EOF.  Everything else is UB.
>> Calling them with a variable of type char, if char is signed, leads to negative
>> values.  I would expect at least some systems in CI to warn somehow, but
>> maybe they don't.
>>
>> libtool: compile:  gcc -I../../liblwgeom/../deps/ryu/.. -I/usr/pkg/include -
>> I/usr/pkg/include -I/usr/pkg/include -I/usr/pkg/include/libxml2 -
>> I/usr/pkg/include -I/usr/pkg/include -I/usr/pkg/include/json-c -DNDEBUG -I.
>> -I../../liblwgeom -std=gnu11 -g -O2 -fno-math-errno -fno-signed-zeros -Wall -
>> O2 -fPIC -DPIC -c ../../liblwgeom/optionlist.c  -fPIC -DPIC -o .libs/optionlist.o
>> In file included from /usr/include/ctype.h:100,
>>                  from ../../liblwgeom/optionlist.c:28:
>> ../../liblwgeom/optionlist.c: In function 'option_list_string_to_lower':
>> ../../liblwgeom/optionlist.c:36:32: warning: array subscript has type 'char' [-
>> Wchar-subscripts]
>>    36 |                 *key = tolower(*key);
>>       |                                ^
>
> I'll have to take closer look at this one.  I would have expected our
> bots that have warning flags turned on would have flagged these.
> Which version of PostgreSQL and gcc compiler are you running with?

pgsql 18, gcc 12.

There are two issues: One is that the code is wrong, and the second is
why CI didn't catch it.

The code is wrong becaues C/POSIX say that ctype functions are UB if the
argument is not in the defined domain.  See

  https://pubs.opengroup.org/onlinepubs/9799919799/functions/tolower.html

and specifically:

  The tolower() [CX] [Option Start] and tolower_l() [Option End]
  functions have as a domain a type int, the value of which is
  representable as an unsigned char or the value of EOF. If the argument
  has any other value, the behavior is undefined.

If there is a variable of type char, which happens to be signed char,
and it contains a negative number (which é is), then calling tolower on
that results in a negative integer not equal to EOF.  It would conform
to C/POSIX for the function to delete all your files, or more reasonably
crash the program.

Reading lwgeom's optionlist.c:optionlist_string_to_lower(), it's clear
that it can lead to UB.

The usual fix/workaround is to cast "char" to "unsigned char" before
passing to a ctype function.  With that é is still é but it's no longer
negative, and within the domain for which the function has defined
behavior.

This is warned about on NetBSD not because of special warning detection
on these functions, but because the ctype functions are implemented as
macros (permitted by POSIX) and look up bitfields of upper, alpha,
etc. in an array of 257 with an offset so it can index EOF (-1), and 0
to 255.  Negative values read some other memory, but that's not really
ok.  This is partly as efficiency and partly to warn about incorrect
code on purpose, so it can be fixed.  I believe this is historical
behavior going back to 4.2BSD at least, maybe even 2BSD, but I'm fuzzy
on those details.


as for my environment details, I noticed that configure doesn't print
out `gcc --version`.  Maybe it should  but that's not a bug.

    gcc (nb1 20250721) 12.5.0

configure summary

    PostGIS is now configured for x86_64-unknown-netbsd11.0

   -------------- Compiler Info -------------
    C compiler:           gcc -std=gnu11 -g -O2 -fno-math-errno -fno-signed-zeros -Wall -O2
    C++ compiler (Wagyu): gcc -std=c++11 -x c++
    C++ compiler (FlatGeobuf): gcc -std=c++11 -x c++
    CPPFLAGS:              -I/usr/pkg/include -I/usr/pkg/include -I/usr/pkg/include -I/usr/pkg/include/libxml2 -I/usr/pkg/include  -I/usr/pkg/include -I/usr/pkg/include/json-c -DNDEBUG
    LDFLAGS:               -lm
    SQL preprocessor:     /usr/bin/cpp -traditional-cpp -w -P -Upixel -Ubool
    Archiver:             ar rs

   -------------- Additional Info -------------
    Interrupt Tests:   ENABLED

   -------------- Dependencies --------------
    GEOS config:          /usr/pkg/bin/pkg-config geos
    GEOS version:         3.15.0 (31500)
    GDAL config:          /usr/pkg/bin/gdal-config
    GDAL version:         3.13.3 (31303)
    PostgreSQL config:    /usr/pkg/bin/pg_config
    PostgreSQL version:   PostgreSQL 18.6
    PostgreSQL bindir:    /usr/pkg/bin
    PostGIS utilities:    ${exec_prefix}/bin
    PROJ version:         9.9.0 (90900)
    Libxml2 config:       /usr/pkg/bin/xml2-config
    Libxml2 version:      2.15.4
    JSON-C support:       yes
    protobuf support:     yes
    protobuf-c version:   1005002
    Perl:                 /usr/pkg/bin/perl

   --------------- Extensions ---------------
    PostgreSQL EXTENSION support:       enabled
    PostGIS Raster:                     enabled
    PostGIS Topology:                   enabled
    Doc visual examples:                no
    SFCGAL support:                     disabled

   -------- Documentation Generation --------
    xsltproc:             /usr/pkg/bin/xsltproc
    docbook xsl base:
    dblatex:
    convert:              /usr/pkg/bin/convert
    python3: