[gdal-dev] Re: HDF-EOS vs. GDAL: order of dimensions

[Ivan Shmakov]

>>>>> Lucena, Ivan <ivan.lucena at pmldnet.com> writes:

 >> How do I use HDFexplorer in, say, Shell scripts?

 > Download from www.space-research.org.

	I'd rather not to download non-free software (as in ``free
	speech'') without a serious reason.  I've taken a look at the
	site and I cannot find any claims of supporting scripting in any
	way.  The contents of the site makes me suppose that this tool
	offers only GUI, and not CLI, and has no embedded language
	either.

[...]

 >> However, it would be useful if the application could provide options
 >> to the driver without creating a helper file.  Consider running a
 >> web application (say, a kind of dataset explorer), which is, for the
 >> sake of security, restricted so that it cannot write any files.  It
 >> will be impossible for this application to

 > There is not need to write files in the client application.

	Earlier in this discussion you've admitted that it may be
	necessary for user to override the options manually.  If I
	understood you correctly, you've suggested that user may be
	allowed to write his own helper file to this end.

	However, in the case of GUI application, it may be more
	appropriate to let the user to specify the options via GUI.
	There, I see no way for the application to pass the options
	specified to GDAL, but to write a temporary helper file.

	To my mind, it's somewhat unfortunate solution, since an
	application may be ran in a some kind of a ``safe mode'', which
	denies the application to write any files.

 >> allow user to specify any options for the GDAL driver not already in
 >> some helper file.  Allowing for options to be passed directly from
 >> the application to GDAL seems to solve this problem.

 > We already have example of knowledge-base files in the <gdal>\data
 > folder. That would be the reference system. So why not have a
 > hdf-knowledge-base that tells the driver some specific issues about
 > specific HDF products.

	I agree on this.  I see /no/ reason why the database like this
	shouldn't be provided.

 > People can also add to this file and send their discoveries to be
 > included in the GDAL trunk. Just like we do with Reference System.

 > Users should only create helper files on the data folder in a very
 > specific case like in my areal photography hypothetical example.

	From a system administrator's point of view, I believe that user
	should be provided a way to use a helper file residing in either
	any directory, or at least in `$HOME' (or a some specific
	subdirectory of it.)

	Otherwise, system administrator will be forced to update the
	database each time a user will face a new (product, dataset)
	combination.  It certainly will complicate the lives of both
	system administrators and users.

 >> I believe that the driver should be given options already read and
 >> parsed, and not the helper file name or its contents.  This way,
 >> both the options parsing and the whole options concept could be much
 >> generalized among the drivers.  And it requires introducing an
 >> ``options'' structure, which may be initialized either by GDAL
 >> reading the helper file, or by the calling application itself.

 > I am dealing with a real scenario where because of the complexity,
 > variability and quantity of HDF files I can't see how the open option
 > would play a role. For me that should be a separated thread, IMHO.

 > But that is how I would like to see it working:

[...]

 > Running gdaltindex, the HDF4 driver will look for hdf-helper file on
 > the same folder as the driver and then on <gdal>\data. If there is a
 > match between a pair {product name,dataset name} on the data file and
 > one similar pair on the helper file, the driver will use the
 > dimension order from the helper file, ex.:

	... And I believe it's not just the order of dimensions that may
	be specified in the helper file.  E. g., I'd like to specify
	``DimensionMap'' as well.

[...]

 > That is because this file is a very known problematic product that
 > doesn't have dimension order information for some datasets:

 > Product: "MODIS Level 2 Cloud Properties"
 > Dataset: "Cloud_Mask_1km"
 > Dataset: "Radiance_Variance"

 > Does it make sense?

	Yes, it does.

	However, as I've stated before, different interfaces should be
	provided.  There're a plenty of GDAL users with a wide variety
	of needs.  There're tasks where encoding the options within the
	dataset name would be appropriate, and the same for helper files
	and separate options structure.

	I believe, that these needs may all be addressed at once with a
	generic options facility to be added to GDAL.

	To this end:

	* upon ::Open (), GDAL parses the options encoded within the
	  dataset name; they result is passed to the specific driver;

	* upon ::OpenOptions (), application may pass additional
	  options, in addition to those already passed as specified
	  above;

	* the driver may request additional options from a
          driver-specific helper file;

	* all these options will become available to the parts of the
	  driver requesting them.

	To my mind, the algorithm roughly outlined above covers well
	your case, as well as, I believe, many other cases.