Re: [gdal-dev] Can't read HDF4 - TRMM
Ivan Lucena
ivan.lucena at pmldnet.com
Tue May 25 11:41:25 EDT 2010
It look like pyHDF know more about the HDF SD dimensions than the other software, including GDAL:
>>> from pyhdf.SD import *
>>> d = SD('D:\\tmp\\3B430501016.HDF')
>>> d.datasets()
{'precipitation': (('scan', 'longitude', 'latitude'), (1, 1440, 400), 5, 0), 'relativeError': (('scan', 'longitude', 'latitude'), (1, 1440, 400), 5, 1)}
>>> ds = d.select('precipitation')
>>> ds.dimensions()
{'latitude': 400, 'longitude': 1440, 'scan': 1}
>>>
That means 1440 columns, 400 rows, and 1 band.
But GDAL, Idrisi, ArcGIS, Erdas and Envi reads it as 400 columns, 1440 rows, and 1 band.
D:\BugsJar>gdal_translate HDF4_SDS:UNKNOWN:"3B430501016.HDF":0 out.tif
Input file size is 400, 1440
0...10...20...30...40...50...60...70...80...90...100 - done.
$ gdalinfo out.tif
Driver: GTiff/GeoTIFF
Files: out.tif
Size is 400, 1440
Coordinate System is `'
Image Structure Metadata:
INTERLEAVE=BAND
Corner Coordinates:
Upper Left ( 0.0, 0.0)
Lower Left ( 0.0, 1440.0)
Upper Right ( 400.0, 0.0)
Lower Right ( 400.0, 1440.0)
Center ( 200.0, 720.0)
Band 1 Block=400x5 Type=Float32, ColorInterp=Gray
There is something wrong with the internal documentation of that file but it seems to be possible to find the dimensions from other HDF (not EOS) API functions. I would try to include calls to SDdiminfo() or SDgetdimstr() perhaps.
Regards,
Ivan
> -------Original Message-------
> From: Ivan Lucena <ivan.lucena at pmldnet.com>
> To: Joaquim Luis <jluis at ualg.pt>
> Cc: gdal-dev at lists.osgeo.org <gdal-dev at lists.osgeo.org>
> Subject: Re: [gdal-dev] Can't read HDF4 - TRMM
> Sent: May 24 '10 13:51
>
> Hi Joaquim,
>
> It seems to me that the HDF4 driver is taking the liberty of using the EOS API to read that file that is actually in HDF format, more specifically a SD (Scientific Dataset).
>
> http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/hdf4/hdf4dataset.cpp#L762
>
> That is why it doesn't go to that call to SDfileinfo():
>
> http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/hdf4/hdf4dataset.cpp#L991
>
> If change poDS->bIsHDFEOS to 0 in debugging mode and the subdatasets showed up:
>
> Driver: HDF4/Hierarchical Data Format Release 4
> Files: 3B430501016.HDF
> Size is 512, 512
> Coordinate System is `'
> Subdatasets:
> SUBDATASET_1_NAME=HDF4_SDS:UNKNOWN:"3B430501016.HDF":0
> SUBDATASET_1_DESC=[1x1440x400] precipitation (32-bit floating-point)
> SUBDATASET_2_NAME=HDF4_SDS:UNKNOWN:"3B430501016.HDF":1
> SUBDATASET_2_DESC=[1x1440x400] relativeError (32-bit floating-point)
> Corner Coordinates:
> Upper Left ( 0.0, 0.0)
> Lower Left ( 0.0, 512.0)
> Upper Right ( 512.0, 0.0)
> Lower Right ( 512.0, 512.0)
> Center ( 256.0, 256.0)
>
> Now I can gdal_translate it to GTIFF but the image comes rotated by 90 degrees just like in other software.
>
> I think we can file a ticket to deal with the miss identification, at least.
>
> Obrigado pela dica.
>
> Thanks for the tip,
>
> Ivan
>
> > -------Original Message-------
> > From: Joaquim Luis <jluis at ualg.pt>
> > To: Ivan Lucena <ivan.lucena at pmldnet.com>
> > Cc: gdal-dev at lists.osgeo.org <gdal-dev at lists.osgeo.org>
> > Subject: Re: [gdal-dev] Can't read HDF4 - TRMM
> > Sent: May 24 '10 10:46
> >
> > Ivan,
> >
> > Since nasa is now imitating me (http://mirador....) I gave it a shot
> > with Mirone, which obviously also failed because because it relies
> > mostly in gdal to read the hdf files.
> > My experience with the nasa satellite files is that they are a big mess
> > with the metadata written in endless different ways which makes the
> > parsing very painful. There is some reference info in the files. See
> > what I got using Matlab (look at the end of this long list)
> >
> > Joaquim
> >
> >
> > >> info.Attributes(1).Value
> >
> > ans =
> >
> > OBJECT=OrbitNumber;
> > Value=-9999;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=OrbitNumber;
> >
> > OBJECT=RangeBeginningDate;
> > Value=2010/04/01;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=RangeBeginningDate;
> >
> > OBJECT=RangeBeginningTime;
> > Value=00:00:00;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=RangeBeginningTime;
> >
> > OBJECT=RangeEndingDate;
> > Value=2010/05/01;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=RangeEndingDate;
> >
> > OBJECT=RangeEndingTime;
> > Value=00:00:00;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=RangeEndingTime;
> >
> > OBJECT=GranulePointer;
> > Value="3B43.100401.6A.HDF";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=GranulePointer;
> >
> > OBJECT=ShortName;
> > Value="Surface Rain from all Satellite and Surface";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=ShortName;
> >
> > OBJECT=SizeMBECSDataGranule;
> > Value=4.394;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SizeMBECSDataGranule;
> >
> > OBJECT=LongitudeOfMaximumLatitude;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=LongitudeOfMaximumLatitude;
> >
> > OBJECT=SpatialCoverageType;
> > Value="Horizontal";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SpatialCoverageType;
> >
> > OBJECT=EllipsoidName;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=EllipsoidName;
> >
> > OBJECT=EquatorialRadius;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=EquatorialRadius;
> >
> > OBJECT=DenominatorFlatteningRatio;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=DenominatorFlatteningRatio;
> >
> > OBJECT=OrbitalModelName;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=OrbitalModelName;
> >
> > OBJECT=SemiMajorAxis;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SemiMajorAxis;
> >
> > OBJECT=MeanAnomaly;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=MeanAnomaly;
> >
> > OBJECT=RightAscensionNode;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=RightAscensionNode;
> >
> > OBJECT=ArgumentOfPerigee;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=ArgumentOfPerigee;
> >
> > OBJECT=Eccentricity;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=Eccentricity;
> >
> > OBJECT=Inclination;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=Inclination;
> >
> > OBJECT=EpochTime;
> > Value=99:99:99;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=EpochTime;
> >
> > OBJECT=EpochDate;
> > Value=9999/99/99;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=EpochDate;
> >
> > OBJECT=EpochMillisec;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=EpochMillisec;
> >
> > OBJECT=WestBoundingCoordinate;
> > Value=-180;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=WestBoundingCoordinate;
> >
> > OBJECT=EastBoundingCoordinate;
> > Value=180;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=EastBoundingCoordinate;
> >
> > OBJECT=NorthBoundingCoordinate;
> > Value=50;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=NorthBoundingCoordinate;
> >
> > OBJECT=SouthBoundingCoordinate;
> > Value=-50;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SouthBoundingCoordinate;
> >
> > OBJECT=CenterLatitude;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=CenterLatitude;
> >
> > OBJECT=CenterLongitude;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=CenterLongitude;
> >
> > OBJECT=RadiusValue;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=RadiusValue;
> >
> > OBJECT=LatitudeResolution;
> > Value=".25deg";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=LatitudeResolution;
> >
> > OBJECT=LongitudeResolution;
> > Value=".25deg";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=LongitudeResolution;
> >
> > OBJECT=GeographicCoordinateUnits;
> > Value="Decimal Degrees";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=GeographicCoordinateUnits;
> >
> > OBJECT=TemporalRangeType;
> > Value="Continuous Range";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=TemporalRangeType;
> >
> > OBJECT=QualityAssuranceParameterName;
> > Value="ScienceQualityFlag";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=QualityAssuranceParameterName;
> >
> > OBJECT=QualityAssuranceParameterValue;
> > Value="NOT BEING INVESTIGATED";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=QualityAssuranceParameterValue;
> >
> > OBJECT=ReprocessingActual;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=ReprocessingActual;
> >
> > OBJECT=BrowsePointer;
> > Value="3B43_BR.100401.6A.PNG";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=BrowsePointer;
> >
> > OBJECT=ScienceContact;
> > Value="George Huffman";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=ScienceContact;
> >
> > OBJECT=MeanMotion;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=MeanMotion;
> >
> > OBJECT=OrbitAdjustFlag;
> > Value=-9999;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=OrbitAdjustFlag;
> >
> > OBJECT=AttitudeModeFlag;
> > Value=-9999;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=AttitudeModeFlag;
> >
> > OBJECT=SolarBetaAngleAtBeginningOfGranule;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SolarBetaAngleAtBeginningOfGranule;
> >
> > OBJECT=SolarBetaAngleAtEndOfGranule;
> > Value=-9999.9;
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SolarBetaAngleAtEndOfGranule;
> >
> > OBJECT=SensorAlignment;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SensorAlignment;
> >
> > OBJECT=SensorAlignmentChannelOffsets;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=SensorAlignmentChannelOffsets;
> >
> > OBJECT=ScanPathModel;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=ScanPathModel;
> >
> > OBJECT=ScanPathModelParam;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=ScanPathModelParam;
> >
> > OBJECT=EphemerisFileID;
> > Value="NULL";
> > Data_Location=PGE;
> > Mandatory=FALSE;
> > END_OBJECT=EphemerisFileID;
> >
> >
> >
> >
> > > Hi,
> > >
> > > I ran gdalinfo on some HDF4 files and no subdataset shows up.
> > >
> > > The data is available for download at this website: http://mirador.gsfc.nasa.gov/cgi-bin/mirador/presentNavigation.pl?tree=project&project=TRMM&dataGroup=Gridded&dataset=3B43:%20Monthly%200.25%20x%200.25%20degree%20merged%20TRMM%20and%20other%20sources%20estimates&version=006
> > >
> > > I tried reading those some file on:
> > >
> > > - Idrisi
> > > - ArcGIS 9.3
> > > - ArcGIS 10 (RC)
> > > - Erdas
> > >
> > > They do identify the internal datasets but they cannot identify the reference system and the images they are flipped by 90' clockwise. But that is not due to rotation. I would guess. It is more like a misinterpretation of the dimension parameter, taking rows as columns and columns as rows. It looks like the documentation for that particular product is missing here: http://mirador.gsfc.nasa.gov/cgi-bin/mirador/servcoll.pl?helpmenuclass=inventory&SearchButton=Search%20GES-DISC
> > >
> > > Does anybody has a clue?
> > >
> > > BTW. There is a netCDF version of that data that also has some issues.
> > >
> > > Regards,
> > >
> > > Ivan
> > >
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > gdal-dev mailing list
> > > gdal-dev at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/gdal-dev
> > >
> > >
> > >
> >
> >
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
More information about the gdal-dev
mailing list