[Qgis-developer] Delimited text enchancements

Sat May 25 16:39:55 PDT 2013

Hi Bob and Deb

Field in the VRT file is fine .. and VRT covers most of the other requirements (needs a bit more in terms of delimiters etc).  CSVT is at the moment completely inadequate as a metadata format.

The issue isn't about the format or content though, it is about the association between the data file and the metadata.  The premise here is that from a users point of view it is the data file they are interested in.  So when they select the data file the metadata needs to be associated with it by default.  This isn't the case with a VRT file, which defines the data source independently of where it is. 

All this could perhaps be handled by expanding the VRT format  to allow it to sit alongside the datafile readily (say an option like besideVRT instead of relativeToVRT)  and adding the additional parameters OGR CSV driver, as well as making the CSV driver handle files with extensions other than csv.  For data files the besideVRT option would also make it much easier to create VRT files, as they could just be renamed to be used for a different file.  besideVRT would mean use the file with the same name as the VRT, minus the VRT extension.

Cheers
Chris
________________________________________
From: Bob and Deb [bobdebm at gmail.com]
Sent: 26 May 2013 10:13
To: Chris Crook
Cc: QGIS Developer Mailing List; Régis Haubourg
Subject: Re: [Qgis-developer] Delimited text enchancements

How about using Field in the VRT instead of using a CSVT?

On May 25, 2013 12:16 PM, "Chris Crook" <ccrook at linz.govt.nz<mailto:ccrook at linz.govt.nz>> wrote:
Ok, I hadn't really thought about the browser much .. so far I've not used it much myself and nor have most of the people I work with.

It certainly creates a challenge with delimited text files generally, as they don't have any inherent spatial intelligence.  So just dragging them to the map without a intermediate dlalog is going to be limiting.   Which I guess is where you are coming from with the CSVT/VRT file idea.  Seeing it in the browser makes it much clearer to me.

I think it may be better to extend the CSVT idea than use VRT.  The advantage of CSVT is that it is implicitly associated with the data file.  The VRT file could be anywhere, and have any name - the user would be selecting the VRT file rather than the CSV file with the actual data.  It seems more intuitive to me to be selecting the data file.

The CSVT file is also analagous to metadata files that are alongside images (eg world files), so there is a good precedent.

Perhaps the way to go is to add more functionality to the CSVT file (to incorporate the features of the  VRT/DLT driver).  I wonder if the place to do the work is in GDAL/OGR, rather than in QGIS?  Once the capabilities are there, then the QGIS dialog would just be a way of writing the csvt file if you wanted to...

So we would be looking at something more like a VRT file in content (though perhaps not XML in terms of end user accessibility) but automatically associated with the file by its name, rather than the other way round as in the VRT file.

If we were doing that, then we might want to use something a bit more identifiable than just adding a 't' to the file name.  Maybe adding an extension like .gmd (geographic metadata) would be better.  Also much easier to identify with confidence in the browser, particularly since we wouldn't just be using data files with .csv extensions.

Cheers
Chris

_______________________________________
From: HAUBOURG [regis.haubourg at eau-adour-garonne.fr<mailto:regis.haubourg at eau-adour-garonne.fr>]
Sent: 25 May 2013 22:20
To: Chris Crook
Subject: RE : Delimited text enchancements

Hi Chris,
you are right, reading csvt is enough at the moment, and merging spreadsheet with ogr data source in one only dialog might be complex and need prototyping.  In the other end, we have growing toolbars with more an more datasources, and many users switch to browser panel. I'm trying to have QGIS as simple as possible for the end users I manage here, and few of them have advanced informatic skills.

I think I will first hire someone to prototype things as you suggest and submit it to community. Work is important if we want that all access (drag-drop / browser / toolbar menu) lead to same behaviour, but it's necessary.

Thanks again, I see things more clearly now.
Régis

________________________________________
De : Chris Crook [ccrook at linz.govt.nz<mailto:ccrook at linz.govt.nz>]
Date d'envoi : samedi 25 mai 2013 00:42
À : HAUBOURG
Objet : RE: Delimited text enchancements

Hi Régis

Don't get too excited, I just said I wouldn't start looking at it till then!!!

As it turns out, not true. I figured out overnight that adding reading the csvt file is a very small safe change, so I've pushed it up to master.  Hope no one minds - my first real commit since being given the privilege.

So the provider will read .csvt files (or .xxxt files, basically the data file name with a 't' appended to it).

A long way short of the ideas you were talking about, but a start.

On your question about a unified vector dialog, I'm not sure.  I guess it may be good to mock something up in python first.  If I understand correctly you are wanting to add spreadsheets etc to the dialog, and manage the OGR drivers as well.  The dlt txt dialog is already busy with just it's own stuff, so I'm not sure how it would work.  Maybe something similar to the way renderers are handled, so each file type has its own widget.  Also the OGR/dlt txt options are different, so that could be challenging.

My thinking so far is more about improving the dlt txt dialog so that it is easy to select all the options at once, or have them read from a metadata file if one is present...

Cheers
Chris
________________________________________
From: HAUBOURG [regis.haubourg at eau-adour-garonne.fr<mailto:regis.haubourg at eau-adour-garonne.fr>]
Sent: 25 May 2013 02:02
To: Chris Crook
Subject: RE: Delimited text enchancements

Hi Chris,
I wish I had some time to help you coding . That's faster than contracting!
What is your opinion about a unified vector dialog? If I start a call for offers, I must at least be sure community agrees..
Régis

> -----Message d'origine-----
> De : Chris Crook [mailto:ccrook at linz.govt.nz<mailto:ccrook at linz.govt.nz>]
> Envoyé : vendredi 24 mai 2013 08:40
> À : HAUBOURG
> Objet : RE: Delimited text enchancements
>
> Hi Régis
>
> I meant to say that I'm short of time for the next few of weeks but I'll
> probably start on a first cut of the CSVT enhancements we discussed after
> that as it will be really useful for me too.  It won't make 2.0 of course, as you
> noted ...
>
> Cheers
> Chris
>
> This message contains information, which is confidential and may be subject
> to legal privilege. If you are not the intended recipient, you must not peruse,
> use, disseminate, distribute or copy this message. If you have received this
> message in error, please notify us immediately (Phone 0800 665 463 or
> info at linz.govt.nz<mailto:info at linz.govt.nz>) and destroy the original message. LINZ accepts no
> responsibility for changes to this email, or for any attachments, after its
> transmission from LINZ. Thank You.

This message contains information, which is confidential and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info at linz.govt.nz<mailto:info at linz.govt.nz>) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.

_______________________________________________
Qgis-developer mailing list
Qgis-developer at lists.osgeo.org<mailto:Qgis-developer at lists.osgeo.org>
http://lists.osgeo.org/mailman/listinfo/qgis-developer