[QGIS-Developer] Editable CSV files

[Jeremy Palmer]

Hi Nyall,

On 20/07/2017, at 8:40 AM, qgis-developer-request at lists.osgeo.org<mailto:qgis-developer-request at lists.osgeo.org> wrote:

I still think the "proper"/"most open source ecosystem friendly"
approach here would be to port the extra features which QGIS'
delimited text provider has across to GDAL's csv driver. Then the "add
delimited text" button would become just a frontend for setting up the
parameters for GDAL's driver (and possibly auto-creation of a csvt
file too).


I fully agree with this, and Chris Crook, who made a lot of improvement to provider a few years back, also does.

I think the CSVT format is a very basic specification and only allows for attribute datatype definitions. The general GDAL CSV support is actually very complete and has good open options - but we need a way for those options to be persisted to disk in an interoperable way. Also, can’t forget the GDAL VRT format supports everything CSVT does plus WKT geometry type setting, projections, GDAL open options and FIDs. The issue with the VRT format is it wasn’t primarily designed as a auxiliary CSV file specification. Note, the GeoCSV spec (https://giswiki.hsr.ch/GeoCSV) also exists and GDAL supports it since 2.1, but it also doesn’t support a lot of the QGIS provider requirements.

IMHO if GDAL is to support the current QGIS delimited text provider requirements, then it really needs is an open and portable way to define and implement:

* Skipping lines (AFAIK not supported in GDAL/OGR)
* Header in file (Supported in GDAL as open option)
* CRS definition (Only possible in VRT)
* Custom delimiters (Supported in GDAL as open option)
* Regex delimiters (AFAIK not supported in GDAL/OGR - would likely need build time dependency)
* Manual setting of datatype (Only possible in VRT, autodetect is possible with GDAL as open option)
* Encoding (GDAL treats all CSVs as UTF-8, QGIS allows other encodings to be set)
* Trimming attribute fields (AFAIK not supported in GDAL/OGR)
* Skipping, treating empty fields as Null (Supported in GDAL as open option)
* Maximum number of fields that will be loaded (AFAIK not supported in GDAL/OGR)
* Escape character (AFAIK not supported in GDAL/OGR)
* Support DMS coordinates (AFAIK not supported in GDAL/OGR)
* Decimal separator is comma (AFAIK not supported in GDAL/OGR)

Also, to support editing or better translation to other formats:

* Definition of the Feature ID/Primary key (Only possible via GDAL VRT)

Of course users could be asked to see if all of these options are actually used. But many of them are indeed useful.

Cheers
Jeremy

________________________________

This message contains information, which may be in confidence and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info at linz.govt.nz) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20170720/8179f4ea/attachment-0001.html>