[gdal-dev] Using GDAL with gridded XYZ that has no 'nodata' values?

[Even Rouault]

Le samedi 24 mai 2014 21:45:45, Sam Franklin a écrit :
> Greetings. This is my first post to this mailing list, so apologies if I
> commit a mailing-list faux-pas :)
> 
> I wonder if anyone can offer a suggestion, as I'm hoping I've missed
> something obvious.
> 
> I regularly receive bathymetric DEMs, acquired by acoustic multibeam
> echosounders (MBES) from offshore geophysical surveys that take the form of
> narrow corridors of survey or irregularly shaped areas.
> 
> My goal is to have a end-to-end open source/GDAL workflow that converts the
> bathymetric DEM to greyscale GEOTIFF which I can then use for further
> processing/analysis using GDAL/OGR.
> 
> The convention in the offshore sector is to deliver bathymetric data as
> single 'gridded' .XYZ ASCII file, where XY coordinate pairs are evenly
> spaced to form a grid and the textfile row order is spatially sequential.
> 
> However, crucially 'nodata' values are omitted from the XYZ.  I can only
> assume the reasoning was to reduce the file size on disc. This is an
> industry convention that will not change anytime soon.
> 
> gdal_translate obviously does not read these XYZ files as gdal throws an
> error due to the lack of true rectangular grid.  I've read the XYZ info
> http://www.gdal.org/frmt_xyz.html which is clear to me, no problem there.
> 
> I do not want to use gdal_grid on the XYZ as this will introduce a further
> interpolation step. The XYZs I receive are already cleansed and gridded by
> the surveyor which I don't wish to edit further.
> 
> So, I currently use a proprietary marine geophysical package which can
> directly read the gridded XYZ without nodata values.  Using this package, I
> can export to ESRI ASCII grid (.ASC) which I then use gdal_translate
> without issue for my further processing steps.
> 
> I'd like the freedom of a non-proprietary solution and the only thing I can
> think of is to write a python script to pre-process my XYZ into a format
> that gdal will accept.  I was thinking of calculating the XY bounding box
> of the data and output an XYZ that blends the original values and nodata
> values in the correct row order.
> 
> If anyone is interested, see below link to a zip of three separate
> examples, containing:
> -- 'gridded' XYZ (without nodata values)
> -- processed greyscale GEOTIFF (using proprietary package)
> -- processed color-relief RGB GEOTIFF (using proprietary package)
> https://dl.dropboxusercontent.com/u/4086367/gdal_translate_xyz_examples.zip
> All data in these examples are referenced to SRS EPSG:27700.
> 
> If I've missed anything obvious, or if anyone has any thoughts of how I can
> achieve my goal using GDAL or other, or comment on my proposed solution,
> that would be greatly appreciated.

What would be interesting is a link to a XYZ file that fails to be opened. In 
GDAL 1.11, I've improved the driver to be able to handle lines with missing 
values at their start and end. It might be possible to improve it again to 
deal with nodata values that would be at any place within lines. If the whole 
file is regularly gridded of course.

> 
> Thanks in advance.
> Sam

-- 
Geospatial professional services
http://even.rouault.free.fr/services.html