[gdal-dev] Directories and Parquet

[Michael Smith]

Paul,

I had to use 3.7.1 to have it read the geometry properly, 3.7.0 would read the directories.

docker run --rm -it ghcr.io/osgeo/gdal:ubuntu-full-latest ogrinfo /vsis3/overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=admins   -al -so
Warning 1: Field names of unhandled type map<string, list<array_element: map<string, string ('array_element')>> ('names')> ignored
Warning 1: Field sources of unhandled type list<array_element: map<string, string ('array_element')>> ignored
INFO: Open of `/vsis3/overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=admins'
      using driver `Parquet' successful.

Layer name: theme=admins
Geometry: Unknown (any)
Feature Count: 99403
Extent: (-180.000000, -89.999900) - (180.000000, 89.999900)
Layer SRS WKT:
(unknown)
Geometry Column = geometry
id: String (0.0)
updatetime: String (0.0)
version: Integer (0.0)
adminlevel: Integer (0.0)
maritime: String (0.0)
subtype: String (0.0)
localitytype: String (0.0)
context: String (0.0)
isocountrycodealpha2: String (0.0)
isosubcountrycode: String (0.0)
defaultlanugage: String (0.0)
drivingside: String (0.0)
bbox.minx: Real (0.0)
bbox.maxx: Real (0.0)
bbox.miny: Real (0.0)
bbox.maxy: Real (0.0)
type: String (0.0)

Mike


-- 

Michael Smith 
Remote Sensing/GIS Center 
US Army Corps of Engineers 






On 7/28/23, 5:50 PM, "gdal-dev on behalf of Paul Ramsey" <gdal-dev-bounces at lists.osgeo.org <mailto:gdal-dev-bounces at lists.osgeo.org> on behalf of pramsey at cleverelephant.ca <mailto:pramsey at cleverelephant.ca>> wrote:


I am playing with accessing the Overture data dump, which is multiple
files in each S3 bucket. The doco says that as of 3.6 (check!), with
the libarrow_dataset build (check) I should be able to read directory
collections as layers, but no luck thus far:


AWS_REGION=us-west-2 AWS_NO_SIGN_REQUEST=YES ./ogrinfo
/vsis3/overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=admins/type=administrativeBoundary/
AWS_REGION=us-west-2 AWS_NO_SIGN_REQUEST=YES ./ogrinfo
/vsis3/overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=admins/type=administrativeBoundary/


ogrinfo failed - unable to open
'/vsis3/overturemaps-us-west-2/release/2023-07-26-alpha.0/theme=admins/type=administrativeBoundary/'.


If I pick one of the constituent files in that directory, ogr can open
it up just fine, and we just have the array/map/list type issue that
Scott has raised in another thread. Is this expected, a result of the
access being via S3 rather than an actual filesystem directory?
Something else?


Thx,
P
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org <mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev <https://lists.osgeo.org/mailman/listinfo/gdal-dev>