[PostGIS] #5901: raster2pgsql does not work with gdal 3.7+ due to new int8 datatype

[PostGIS]

#5901: raster2pgsql does not work with gdal 3.7+ due to new int8 datatype
------------------------+---------------------------
  Reporter:  lstolcman  |      Owner:  pramsey
      Type:  defect     |     Status:  new
  Priority:  medium     |  Milestone:  PostGIS 3.5.4
 Component:  postgis    |    Version:
Resolution:             |   Keywords:
------------------------+---------------------------
Description changed by lstolcman:

Old description:

> raster2pgsql (python version and compiled version) is failing with gdal
> 3.7 and newer
>
> I rounded down the issue, it is due to new datatype added in gdal 3.7:
> https://gdal.org/en/stable/development/rfc/rfc87_signed_int8.html
>
> python version of raster2pgsql does not support GDT_Int8, e.g.:
> https://git.osgeo.org/gitea/postgis/postgis/src/commit/d027c5f62ab4f63ca1618d27edc1c01ac70c9264/raster/scripts/python/raster2pgsql.py#L213
> compiled version seems to not support it either (logs below)
>

> Issue can be reproduced using docker container of gdal and the python
> version of raster2pgsql.
>

> Working raster2pgsql.py example:
>
> 1. download docker image: `docker pull ghcr.io/osgeo/gdal:ubuntu-
> small-3.6.4`
> 2. put example.tif into current directory
> 3. run docker container from current dir (with tif file): `docker run -it
> -v .:/mnt/aaa ghcr.io/osgeo/gdal:ubuntu-small-3.6.4 /bin/bash`
> 4. install postgis in container and enter python console:
>
> {{{
> apt update -y
> apt install python3-pip postgis -y --no-install-recommends
> pip install ipython
> cd /mnt/aaa
> ipython --no-autoindent
> }}}
>
> 5. paste the code to test the issue:
>
> {{{
> from osgeo import gdal
> print(f"{gdal.__version__}")
> org = gdal.Open("example.tif")
> org_band = org.GetRasterBand(1)
> print(f"{org_band.DataType=}")
> }}}
>
> output - supported datatype
>
> {{{
> In [1]: from osgeo import gdal
>    ...: print(f"{gdal.__version__}")
>    ...: org = gdal.Open("example.tif")
>    ...: org_band = org.GetRasterBand(1)
>    ...: print(f"{org_band.DataType=}")
>    ...:
> 3.6.4
> org_band.DataType=1
> }}}
>
> GDAL 3.7 - raster2pgsql.py failing:
> 1. download docker image: `docker run -it -v .:/mnt/aaa
> ghcr.io/osgeo/gdal:ubuntu-small-3.7.0 /bin/bash`
> 2,3,4,5 - as previously
>
> output - different (new) datatype (int8):
> {{{
> In [1]: from osgeo import gdal
>    ...: print(f"{gdal.__version__}")
>    ...: org = gdal.Open("example.tif")
>    ...: org_band = org.GetRasterBand(1)
>    ...: print(f"{org_band.DataType=}")
>    ...:
> 3.7.0
> org_band.DataType=14
> }}}
>

>

> the issue in compiled version of raster2pgsql (raster2pgsql.c) can be
> tested using docker (entering into container as above) -
> - `ghcr.io/osgeo/gdal:ubuntu-small-3.8.5` - contains raster2pgsql with
> gdal 3.4
> - `ghcr.io/osgeo/gdal:ubuntu-small-3.9.3` - contains raster2pgsql with
> gdal 3.8
>
> output from raster2pgsql with gdal 3.4 works fine (converts tif to sql)
>
> output of raster2pgsql from gdal 3.9.5 docker image fails with a message:
>
> {{{
> root at 04421679a038:/mnt/aaa# raster2pgsql example.tif > example.sql
> Processing 1/1: example.tif
> ERROR: rt_pixtype_size: Unknown pixeltype
> 13####################################
> ERROR: rt_raster_from_gdal_dataset: Unknown pixel type for GDAL band
> ERROR: convert_raster: Could not convert VRT dataset to PostGIS
> raster##########
> ERROR: process_rasters: Could not process raster: example.tif
> ERROR: Unable to process rasters
> root at 04421679a038:/mnt/aaa#
> }}}

New description:

 raster2pgsql (python version and compiled version) is failing with gdal
 3.7 and newer

 I rounded down the issue, it is due to new datatype added in gdal 3.7:
 https://gdal.org/en/stable/development/rfc/rfc87_signed_int8.html

 python version of raster2pgsql does not support GDT_Int8, e.g.:
 https://git.osgeo.org/gitea/postgis/postgis/src/commit/d027c5f62ab4f63ca1618d27edc1c01ac70c9264/raster/scripts/python/raster2pgsql.py#L213

 compiled version seems to not support it either (logs below)


 Issue can be reproduced using docker container of gdal and the python
 version of raster2pgsql.


 Working raster2pgsql.py example:

 1. download docker image: `docker pull ghcr.io/osgeo/gdal:ubuntu-
 small-3.6.4`
 2. put example.tif into current directory
 3. run docker container from current dir (with tif file): `docker run -it
 -v .:/mnt/aaa ghcr.io/osgeo/gdal:ubuntu-small-3.6.4 /bin/bash`
 4. install postgis in container and enter python console:

 {{{
 apt update -y
 apt install python3-pip postgis -y --no-install-recommends
 pip install ipython
 cd /mnt/aaa
 ipython --no-autoindent
 }}}

 5. paste the code to test the issue:

 {{{
 from osgeo import gdal
 print(f"{gdal.__version__}")
 org = gdal.Open("example.tif")
 org_band = org.GetRasterBand(1)
 print(f"{org_band.DataType=}")
 }}}

 output - supported datatype

 {{{
 In [1]: from osgeo import gdal
    ...: print(f"{gdal.__version__}")
    ...: org = gdal.Open("example.tif")
    ...: org_band = org.GetRasterBand(1)
    ...: print(f"{org_band.DataType=}")
    ...:
 3.6.4
 org_band.DataType=1
 }}}

 GDAL 3.7 - raster2pgsql.py failing:
 1. download docker image: `docker run -it -v .:/mnt/aaa ghcr.io/osgeo/gdal
 :ubuntu-small-3.7.0 /bin/bash`
 2,3,4,5 - as previously

 output - different (new) datatype (int8):
 {{{
 In [1]: from osgeo import gdal
    ...: print(f"{gdal.__version__}")
    ...: org = gdal.Open("example.tif")
    ...: org_band = org.GetRasterBand(1)
    ...: print(f"{org_band.DataType=}")
    ...:
 3.7.0
 org_band.DataType=14
 }}}




 the issue in compiled version of raster2pgsql (raster2pgsql.c) can be
 tested using docker (entering into container as above) -
 - `ghcr.io/osgeo/gdal:ubuntu-small-3.8.5` - contains raster2pgsql with
 gdal 3.4
 - `ghcr.io/osgeo/gdal:ubuntu-small-3.9.3` - contains raster2pgsql with
 gdal 3.8

 output from raster2pgsql with gdal 3.4 works fine (converts tif to sql)

 output of raster2pgsql from gdal 3.9.5 docker image fails with a message:

 {{{
 root at 04421679a038:/mnt/aaa# raster2pgsql example.tif > example.sql
 Processing 1/1: example.tif
 ERROR: rt_pixtype_size: Unknown pixeltype
 13####################################
 ERROR: rt_raster_from_gdal_dataset: Unknown pixel type for GDAL band
 ERROR: convert_raster: Could not convert VRT dataset to PostGIS
 raster##########
 ERROR: process_rasters: Could not process raster: example.tif
 ERROR: Unable to process rasters
 root at 04421679a038:/mnt/aaa#
 }}}

--
-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/5901#comment:1>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.