[GRASS-user] XTF reader neede - triton format for sonar

[Hamish]

Hamish:
> > I had considered a few alternatives:
> > 
> > - create a generic libxtf (LGPL?)
> > - GRASS support via a new C module (without a libxtf)
> > - postgis import tool
> > - sqlite import tool
> > - stand alone command line converter to csv or xml ascii format
> >   (then shell script or python script importer to GIS)
> > - volunteer to hack support for it into MBSystems (see libxtf above)
> >   (then work on MBSystems -> GRASS workflow code)

Brent:
> One option missing in your list [above] is a GDAL/OGR driver for XTF?

yes, of course in hindsight that is a glaring omission and an obv choice.
however... I wonder how useful the raw data is in that form, prior to
accelerometer/RTK GPS roll/pitch/yaw/heave, sound velocity profile, etc., corrections being applied?

My current thinking is to work on extending MBSystem's existing MBIO
XTF code, and then play with output options for the cleaned dataset
from the MBIO structures.

A question I have is would people prefer to have access to the raw data
and apply their own corrections by hand in Matlab (or whatever)?
The engineer in me says reuse existing tools, the academic in me says
apply and explore these corrections manually.


another problem with straight to GDAL/OGR is that XTF is an "extensible
format", and so a real pain to nail down. I'm thinking it would to have
prior knowledge or rules for each of the dozens of hardware/software
configs out there. (but perhaps it is more self-describing than I think)
I fear I'd get it working for our specific setup but the code would be
of limited use for others (making a COTS solution more attractive).


Eric wrote:
> An XTF to ASCII conversion tool would be most welcome; we use
> Knudsen 3.5 kHz sounders quite a bit for sub-bottom profiling, and
> you're pretty much stuck working in Knudsen GUI to convert/process
> the data.
>
> I work with MBSystems quite a bit;

could you explain more the workflow there? those two statements seem at
odds.

I took a punt, unfortunately google has zero "xtfdump" hits. If XTF is
self-describing enough I suppose the natural way would be to output to
a (gzip'd!) XML ascii file using such a program.
Google did find a xtf2xml, but it's a Java TeX class of some sort.

> I think any sort of integration with Grass would be a great idea, and
> would be willing to test/bugcheck/document any new modules developed
> for it.

MBSystems already outputs mosaics to GeoTIFF, Arc ascii grids, and GMT
NetCDF grd, so raster export to GRASS is trivial.

After nav corrections, if we output the vector xyz point data
(+attributes), the binning and mosaicking steps could be done in GRASS.

I think a typical scan line could contain about 500k sidescan hits,
and 100k bathy + amplitude hits. Half a dozen lines later and you start
to cause severe memory problems for GRASS's vector engine assuming you
wish to retain links to the attributes. Which then leads back to working
on better links between GRASS & PostGIS via v.external...
I wonder what could be done with a v.external.simple [features] link
(or -s flag to existing v.external) which didn't try to build/clean
topology?  Which, for better or worse, isn't too far from going back to
GRASS 5's sites formats for vector points.


Currently I'm too much a of a beginner to know what parts of MBSystems
could be complimented by GRASS tools, and vice versa. Hopefully between
the two a complete and mature toolkit could be assembled. As both
softwares + GMT are all command line module based there is lots of room
for mingled scripting and e.g. wxPython frontend GUI wizards. And for
the hell of it we could throw R into the mix to aid with fuzzy cleaning
tasks. Many options. Any guidance Eric?

I am also interested in datalogging software options, as that is another
expensive dongle which could be traded in to fund an additional grad
student, or at least give us a backup & get us out of the situation of
depending on a single supplier while out in remote locations + expensive
ship time.


brainstorming & learning,
Hamish