[gdal-dev] Motion: adopt RFC 75: Multidimensional array

[Joe Lee]

I found the error. It was my token key in gdal.SetConfigOption().
GDAL Multidimensional support is quite amazing and works well with AWS Lambda & HDF5 data on S3!

I can't wait to see HDF4 multidimensional support.

Thank you, GDAL development team for the new driver!


On Tue, Jul 30, 2019 at 12:17 PM Joe Lee <hyoklee at hdfgroup.org<mailto:hyoklee at hdfgroup.org>> wrote:

Hi, Even!

  1-D dataset handling in the new HDF5 multi-dimension code is fantastic!
  I could convert 1-d swath into GeoTIFF using the new Python HDF5 multidimensional API.
  Please see python code at the bottom of email.

  However, I can't run it as an AWS lambda function.
  The below is the error message:


'NoneType' object has no attribute 'GetRootGroup': AttributeError
Traceback (most recent call last):
File "/var/task/lambda.py", line 42, in handler
rg = ds.GetRootGroup()
AttributeError: 'NoneType' object has no attribute 'GetRootGroup'


  The same code worked fine under AWS lambda Docker before it is deployed.
  I even tested the Python code on Google Colab and could access data on S3 using vsis3.
  Thus, I am quite puzzled.
   Has anyone successfully tested with HDF5 multidimensional with AWS Lambda?


    # Open dataset using vsis3 interface. fname is obtained from S3 bucket trigger.
    path = '/vsis3/sdt-data/' + fname
    ds = gdal.OpenEx(path, gdal.OF_MULTIDIM_RASTER)
    rg = ds.GetRootGroup()
    hg = rg.OpenGroup('HDFEOS')
    sg = hg.OpenGroup('SWATHS')
    mg = sg.OpenGroup('MOP02')
    gg = mg.OpenGroup('Geolocation Fields')
    dg = mg.OpenGroup('Data Fields')

    ar_lat = gg.OpenMDArray('Latitude')
    lat_data = ar_lat.Read(buffer_datatype = gdal.ExtendedDataType.Create(gdal.GDT_Float32))
    lats = np.frombuffer(lat_data, dtype=np.float32)
        ...

On Fri, Jul 26, 2019 at 9:34 AM Even Rouault <even.rouault at spatialys.com<mailto:even.rouault at spatialys.com>> wrote:
On jeudi 25 juillet 2019 19:57:44 CEST Joe Lee wrote:
> Unsupported number of dimensions

Ah ok, when exporting to GeoTIFF or a "classic" 2D driver, the
multidimensional-to-classic bridge currently checks that the array is a 2D
one. Here you reduced it to a 1D-slice, hence it rejects it. The bridge could
probably be made a bit more flexible by accepting 1D arrays as well. If you
export to netCDF, this should work fine though (as this won't go through the
bridge, but use directly multidimensional netCDF output capabilities)

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20190730/0bf5993e/attachment-0001.html>