[gdal-dev] Odd issue with parquet using gdal python opening inside Django code

[Michael Smith]

Looks like if I rewrite the file with gdal it now works in the Django code. 
PARQUET: geo = {"version":"1.1.0","primary_column":"GEOMETRY","columns":{"GEOMETRY":{"encoding":"WKB","crs":null,"bbox":[-179.001667174293,-15.0016666666675,180.0,84.001666666666694],"covering":{"bbox":{"xmin":["GEOMETRY_bbox","xmin"],"ymin":["GEOMETRY_bbox","ymin"],"xmax":["GEOMETRY_bbox","xmax"],"ymax":["GEOMETRY_bbox","ymax"]}},"orientation":"counterclockwise","geometry_types":["Polygon"]}}}
PARQUET: Bounding box column 'GEOMETRY_bbox' detected for geometry column 'GEOMETRY'


I think these files were orginally written with pyogrio and polars. 


Mike




-- 


Michael Smith 
RSGIS Center – ERDC CRREL NH 
US Army Corps 










On 1/13/26, 6:18 PM, "Michael Smith" <michael.smith.erdc at gmail.com <mailto:michael.smith.erdc at gmail.com> <mailto:michael.smith.erdc at gmail.com <mailto:michael.smith.erdc at gmail.com>>> wrote:




We use gdal with Django and build gdal from conda and install the parquet driver that way. But we have just found an odd issue when a parquet file is opened in our Django code vs in a Django shell.




In the shell we get




gf = gdal.OpenEx('/tmp/stac_48069_usgs-3dep_rasters.parquet')
GDAL: On-demand registering /home/gridusr/gridpixi/.pixi/envs/default/lib/gdalplugins/ogr_Parquet.so using RegisterOGRParquet.
GDAL: GDALOpen(/tmp/stac_48069_usgs-3dep_rasters.parquet, this=0x5626a2e395a0) succeeds as Parquet.
PARQUET: Compression (of first column): zstd
ARROW: Memory pool: bytes_allocated = 0
ARROW: Memory pool: max_memory = 0
GDAL: GDALClose(/vsis3/grid-dev-publiclidar/stac/testgrid/rasters/stac_48069_usgs-3dep_rasters.parquet, this=0x5626a2e38990)




And everything works fine




In the code we get
GDAL: On-demand registering /home/gridusr/gridpixi/.pixi/envs/default/lib/gdalplugins/ogr_Parquet.so using RegisterOGRParquet.
GDAL: GDALOpen(/tmp/stac_48069_usgs-3dep_rasters.parquet, this=0x7f43945696b0) succeeds as Parquet.
PARQUET: Dealing with field GEOMETRY of extension type geoarrow.polygon as list<rings: list<vertices: struct<x: double not null, y: double not null> not null> not null>
PARQUET: Compression (of first column): zstd
GDAL: GDALOpen(/tmp/stac_48069_usgs-3dep_rasters.parquet, this=0x7f43945d6760) succeeds as Parquet.
OGR: GetLayerCount() = 1




And this line seems to be the issue
PARQUET: Dealing with field GEOMETRY of extension type geoarrow.polygon as list<rings: list<vertices: struct<x: double not null, y: double not null> not null> not null>




In this case, the geometry is not recognized. 
Calling gdal.VectorInfo(gf) I see




INFO: Open of `/tmp/stac_48069_usgs-3dep_rasters.parquet'
using driver `Parquet' successful.




Layer name: stac_48069_usgs-3dep_rasters
Geometry: None
Feature Count: 109314
Layer SRS WKT:
(unknown)
GEOMETRY: String(JSON) (0.0)




In the django python shell we get
In [7]: print (res)
INFO: Open of `/tmp/stac_48069_usgs-3dep_rasters.parquet'
using driver `Parquet' successful.




Layer name: stac_48069_usgs-3dep_rasters
Geometry: Polygon
Feature Count: 109314
Extent: (-179.001667, -15.001667) - (180.000000, 84.001667)
Layer SRS WKT:
(unknown)
Geometry Column = GEOMETRY




Any ideas?








-- 




Michael Smith 
RSGIS Center – ERDC CRREL NH 
US Army Corps