[gdal-dev] Re: HDF-EOS vs. GDAL: order of dimensions

[Ivan Shmakov]

>>>>> Lucena, Ivan <ivan.lucena at pmldnet.com> writes:

 >> I believe that the driver should be given options already read and
 >> parsed, and not the helper file name or its contents.  This way,
 >> both the options parsing and the whole options concept could be
 >> much generalized among the drivers.  And it requires introducing
 >> an ``options'' structure, which may be initialized either by GDAL
 >> reading the helper file, or by the calling application itself.

 > I am dealing with a real scenario where because of the complexity,
 > variability and quantity of HDF files I can't see how the open
 > option would play a role. For me that should be a separated thread,
 > IMHO.

 > But that is how I would like to see it working:

 > There are 350 HDF that I want to server it a webapp. First thing to
 > do is to create a catalog. I am not a big Mapserver expert but I
 > guess that you should use *gdaltindex* for that. That will happen
 > in the server machine just once, during some sort of data
 > preparation process, and once in a while when you update it with
 > new files.

 > Running gdaltindex, the HDF4 driver will look for hdf-helper file
 > on the same folder as the driver and then on <gdal>\data. If there
 > is a match between a pair {product name,dataset name} on the data
 > file and one similar pair on the helper file, the driver will use
 > the dimension order from the helper file, ex.:

 > SUBDATASET_52_NAME=HDF4_EOS:EOS_SWATH:"MYD06_L2.A2006220.hdf":mod06:Cloud_Mask_1km
 > SUBDATASET_52_DESC=[2040x1354x2] Cloud_Mask_1km mod06 (8-bit integer)

 > --> This dataset has 2 bands of 2040x1354.

 > SUBDATASET_28_NAME=HDF4_EOS:EOS_SWATH:"MYD06_L2.A2006220.hdf":mod06:Radiance_Variance
 > SUBDATASET_28_DESC=[7x408x270] Radiance_Variance mod06 (16-bit integer)

 > --> This dataset has 7 bands of 408x270

 > That is because this file is a very known problematic product that
 > doesn't have dimension order information for some datasets:

 > Product: "MODIS Level 2 Cloud Properties"
 > Dataset: "Cloud_Mask_1km"
 > Dataset: "Radiance_Variance"

 > Does it make sense?

        Could you please test your data with the proof of concept
        patch I've posted to the list recently?  I've put the
        following into my "$GDAL_OPTIONS_FILE" and it seems to make
        GDAL work correctly with the dataset:

$ cat "$GDAL_OPTIONS_FILE"
{ expr 0 } { some lines skipped }
{ match "^HDF4_EOS:EOS_SWATH:.*:mod06:Cloud_Mask_1km\$" } {
    options hdf4 { dims { iy 0 ix 1 iband 2 } }
}
$ gdalinfo 'HDF4_EOS:EOS_SWATH:"MOD06_L2.A2002234.0515.003.2002243234435.hdf":mod06:Radiance_Variance:OPTIONS:source
gdal-options.tcl'
Driver: HDF4Image/HDF4 Dataset
Files: MOD06_L2.A2002234.0515.003.2002243234435.hdf
Size is 1354, 2040
Coordinate System is `'
GCP Projection = (null)
GCP[  0]: Id=, Info=
          (2.5,2.5) -> (72.4288864135742,45.6274261474609,0)
...
GCP[143]: Id=, Info=
          (1322.5,2037.5) -> (91.8692855834961,24.5590171813965,0)
Metadata:
  HDFEOSVersion=HDFEOS_V2.4
  Number_of_Instrument_Scans=2040
  Maximum_Number_of_1km_Frames=1354
...
  Parameter_Type=MODIS Input
  Cell_Along_Swath_Sampling=1, 2030, 1
  Cell_Across_Swath_Sampling=1, 1354, 1
  Geolocation_Pointer=External MODIS geolocation product
  description=See MODIS atmosphere QA plan for details

Geolocation:
  X_DATASET=HDF4_EOS:EOS_SWATH_GEOL:"MOD06_L2.A2002234.0515.003.2002243234435.hdf":mod06:Longitude
  X_BAND=1
  Y_DATASET=HDF4_EOS:EOS_SWATH_GEOL:"MOD06_L2.A2002234.0515.003.2002243234435.hdf":mod06:Latitude
  Y_BAND=1
  PIXEL_OFFSET=2
  PIXEL_STEP=5
  LINE_OFFSET=2
  LINE_STEP=5
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0, 2040.0)
Upper Right ( 1354.0,    0.0)
Lower Right ( 1354.0, 2040.0)
Center      (  677.0, 1020.0)
Band 1 Block=1354x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
Band 2 Block=1354x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
$

        While without the helper file GDAL seems to misinterpret the
        dimensions:

$ gdalinfo 'HDF4_EOS:EOS_SWATH:"MOD06_L2.A2002234.0515.003.2002243234435.hdf":mod06:Cloud_Mask_1km'
Driver: HDF4Image/HDF4 Dataset
Files: MOD06_L2.A2002234.0515.003.2002243234435.hdf
Size is 2, 1354
Coordinate System is `'
Metadata:
  HDFEOSVersion=HDFEOS_V2.4
  Number_of_Instrument_Scans=2040
  Maximum_Number_of_1km_Frames=1354
...
  Parameter_Type=MODIS Input
  Cell_Along_Swath_Sampling=1, 2030, 1
  Cell_Across_Swath_Sampling=1, 1354, 1
  Geolocation_Pointer=External MODIS geolocation product
  description=See MODIS atmosphere QA plan for details

Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0, 1354.0)
Upper Right (    2.0,    0.0)
Lower Right (    2.0, 1354.0)
Center      (    1.0,  677.0)
Band 1 Block=2x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
Band 2 Block=2x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
...
Band 2039 Block=2x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
Band 2040 Block=2x1 Type=Byte, ColorInterp=Gray
  NoData Value=0
$