[Qgis-developer] import CSV to an existing layer

[Denis Rouzaud]

Hi Crhis,

Thanks a lot for your comments.

On 09/27/2013 10:08 PM, Chris Crook wrote:
> Hi Denis
>
> I'm not sure if you are suggesting adding this to the existing delimited text layer function (bad idea!), or adding a new function using components of the existing delimited text layer.  I say bad idea for a couple of reasons.  FIrstly the UI is already cluttered enough, and adding some additional controls to offer appending to an existing layer would I think make it even more confusing.  Secondly I think an "import data" function is very different from a "create layer" function, so they don't belong in the same place in the menu - it would be too hard to find.
Indeed, my first intention was to use the same dialog. I agree it's not 
the right place.
>
> On the other hand I can see that there is a very valid case for an import data type of function (and as Nathan points out, it isn't specific to CSV files).  In your case that is what you want, but I can easily imagine users wanting to import data from other data sources (eg databases, web services, etc).  But obviously that makes it a much bigger task :-)
Then, there is two ways to achieve import:

- as Nathan suggested, copy features from a layer to antoher
- unify the add layer dialogs and add an option in them that specify 
"append to a layer". And this could possibly open the "copy features 
dialog" from the first option.

We could use an attribute table with editable header to do the field 
mapping.

Then, would you consider this a good solution:
- add a "copy features to another layer" option
- add an option in the "add layer" dialogs for appending instead of 
creating a new layer

> You were asking if anyone is working on this.  I don't know for sure (!).  But things that may impact on it that I do know about:
>
> 1) Régis Haubourg was suggesting that we should merge the delimited text layer with the OGR CSV provider, so that in effect the UI just becomes a front end for setting up the OGR driver.  This would mean updating the OGR driver to support the functions in the delimited text driver first (such a regular expression - messy because there are many different implementations and currently we are using the Qt version).
Thanks for the info. Although, I am not sure it would change something 
to the workflow.
> 2) The dialog (and provider) needs to be improved to support explicitly selecting field types (slightly messy because the field types can come from a CSVT file).  Automatically inferring field types is generally useful, but has created issues with linking data (eg when text field is interpreted as numeric)
>
> 3) There is a suggestion that it could have a capability to support importing multiple files at once.
>
> Anyway just thought this could be useful background if you are working in this area...
>
> Cheers
> Chris
>
> ________________________________________
> From: Denis Rouzaud [denis.rouzaud at gmail.com]
> Sent: 28 September 2013 00:08
> To: Nathan Woodrow
> Cc: qgis-dev
> Subject: Re: [Qgis-developer] import CSV to an existing layer
>
> Hi Nathan,
>
> I understand your point.
>
> But this would require more steps: add the layer to the project, copy the features (with field mapping) and remove the layer.
> I see a main usage of appending to an existing layer in the import of GPS points for example. So mainly csv files.
> It's true that there could be a great interest for shape files.
>
> The point is that I can use the existing header of the table in the demilited text UI to do the field mapping. So, it should not be a great effort to do so.
>
> It's clear that there is no a great cohesion with all these add layers dialogs and I don't want to add more confusion to this.
> But is there any plan to improve this?
>
> Denis
>
>
>
>
>
> On 09/27/2013 01:43 PM, Nathan Woodrow wrote:
> I think a general higher level Append Features to Layer feature would be much better.  This way it would work with any layer opened in QGIS.
>
> Being able to supply a layer to layer mapping would be even better in case the fields don't match.
>
> - Nathan
>
>
> On Fri, Sep 27, 2013 at 9:34 PM, Denis Rouzaud <denis.rouzaud at gmail.com<mailto:denis.rouzaud at gmail.com>> wrote:
> Hi all,
>
> I have the intention to add the possibility of importing features from a CSV into an existing layer.
> My idea is to use the existing dialog "Create a layer from a delimited text file" and to add the choice "create a layer" or "append to an existing layer".
>
> First of all, is anyone already doing something similar?
>
> Second, if anyone already thought about this and has some ideas to share, they're welcome!
>
> Cheers,
>
> Denis
>
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org<mailto:Qgis-developer at lists.osgeo.org>
> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> This message contains information, which is confidential and may be subject to legal privilege. If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message. If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info at linz.govt.nz) and destroy the original message. LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ. Thank You.