[Qgis-user] ctrl-c selection issue

Tue Jan 9 15:33:37 PST 2018

Hmm, I can’t use Edit > Paste As, because qgis does not finish the Edit > Copy command.

The csv file is about 270 MB, 7,5 mln objects with 5 attributes, no x-y data. 

If you want to recreate the issue, the file can be found here:

https://www.cbs.nl/nl-nl/maatwerk/2017/38/buurt-wijk-en-gemeente-2017-voor-postcode-huisnummer

I run into this issue when selecting all objects with Attribute Gem217 = 310

355 also has this issue. 321 and 356 work without a problem.

I am using qgis here just as a means to chop the csv file into smaller selections which I can then use to edit, add attributes and data to and make maps of them.

Gr.

Jeroen

Van: Qgis-user [mailto:qgis-user-bounces at lists.osgeo.org] Namens Ramon Andinach
Verzonden: dinsdag 9 januari 2018 23:35
Aan: qgis-user <qgis-user at lists.osgeo.org>
Onderwerp: Re: [Qgis-user] ctrl-c selection issue

Just so we’ve got just about everything covered, does this happen for “Edit-> Paste As”?

Also looking at the size of your original CSV, I’d go along with Jeroen, and try and import that csv into some other format first. 

If you can find a way of making the selection without going near the attribute table that will help too.

Regards,

Ramon.

On 10 Jan 2018, at 06:26, Jeroen Hovens <mail at groenebij.nl <mailto:mail at groenebij.nl> > wrote:

Thanks for the ideas.

The option to save the selection as new csv file as Ramon suggested, doesn’t work; qgis goes into ‘not responding’ and no csv file is added to the layer list. However, a csv file is made, lacking a decent number of objects (first test 15095 of 18582 objects and second test is 15179 of 18582 objects, so missing about 3,5k objects). This csv file can be opened manually in qgis.

And I checked: this options does work for selections where the ctrl-c option also worked. Then the new cvs layer is added to the layer list in no time.

Is there some sort of log file in qgis that might give clues to the ‘not responding?

@Kirk

I did close and reopen qgis multiple times, even shut down the pc entirely, but to no effect. Selections that did allow for copying earlier, still work: I can make that selection again and ctrl-c works fine. Seems to me this rules out memory-issues.

Gr.

Jeroen

Van: Qgis-user [mailto:qgis-user-bounces at lists.osgeo.org] Namens Kirk Schmidt
Verzonden: dinsdag 9 januari 2018 22:20
Aan: qgis-user at lists.osgeo.org <mailto:qgis-user at lists.osgeo.org> 
Onderwerp: Re: [Qgis-user] ctrl-c selection issue

After a few selections, you may have to close and reopen qgis to purge the memory as well. Importing a csv is memory intensive so another approach is to try saving as a spatial database (SQLite) or even a shapefile.  Once written, you can load the spatial file and you should have lots of memory to keep selecting points. 

Kirk

On 1/9/2018 4:58 PM, Ramon Andinach wrote:

I’m not sure what the problem is, but if I were doing that repeatedly, I’d probably use Layer -> Save As, tick the Save Only Selected Features box and save it to CSV.  Then muck about with excel later. 

This way ought to be a bit lighter on the memory. 

Ramon.

On 10 Jan 2018, at 04:25, Jeroen Hovens < <mailto:mail at groenebij.nl> mail at groenebij.nl> wrote:

Hi,

Using qgis 2.14.18 I have been working today with a large csv file (7.5 mln objects). I just need selections of this database which I can then modify in excel. So I opened the attribute table and made a selection based on a specific value for an attribute, and copied that selection (about 30.000 objects) via the clipboard (ctrl-c) to a new Microsoft Excel file, then saved the excel file and repeated the process for another selection. This worked perfect for some time. However, some selections won’t copy: when I press the copy button the programs turn to ‘not responding’ and won’t snap out of it anymore.  Selection size when this happens is about the same (about 20k to 35k objects).

What could be the problem? Could the data be corrupted? How can I check what is wrong?

Thank you

Gr.

Jeroen 

_______________________________________________
Qgis-user mailing list
 <mailto:Qgis-user at lists.osgeo.org> Qgis-user at lists.osgeo.org
List info:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user

_______________________________________________
Qgis-user mailing list
 <mailto:Qgis-user at lists.osgeo.org> Qgis-user at lists.osgeo.org
List info:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe:  <https://lists.osgeo.org/mailman/listinfo/qgis-user> https://lists.osgeo.org/mailman/listinfo/qgis-user

-- 
Kirk Schmidt, MScF, BScF, RPF
General Manager
Nortek Resource Solutions Inc.
RR # 1
Thorburn, NS
B0K 1W0
Tel (902) 922.3607
Email:  <mailto:kirk at nortekresources.com> kirk at nortekresources.com
Web:  <http://www.nortekreources.com/> www.nortekreources.com

_______________________________________________
Qgis-user mailing list
Qgis-user at lists.osgeo.org <mailto:Qgis-user at lists.osgeo.org> 
List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20180110/2a03f7c8/attachment.html>