[Qgis-user] editing and saving very large table

[Frank Broniewski]

Hi,

you can use the DB Manager for custom layer queries. You can create SQL queries and add those to the layer tree, more information can be found here https://docs.qgis.org/3.34/en/docs/user_manual/plugins/core_plugins/plugins_db_manager.html

And of course, ordering queries based on custom criteria can be done there via ORDER BY clause ...

HTH
Frank
________________________________
Von: QGIS-User <qgis-user-bounces at lists.osgeo.org> im Auftrag von Gordon Wakelin-King via QGIS-User <qgis-user at lists.osgeo.org>
Gesendet: Mittwoch, 13. März 2024 00:20
An: chris hermansen <clhermansen at gmail.com>; Michael.Dodd <michael.dodd at open.ac.uk>
Cc: Frederik-Matti Bartels (CAPSA) <bartels at capsa-building.org>; qgis-user at lists.osgeo.org <qgis-user at lists.osgeo.org>
Betreff: Re: [Qgis-user] editing and saving very large table


Surely this is something that can be done easily with an SQL Select  with Order-by, in one of the SQL variants that QGIS can use?



I’m asking for my own benefit too. Having come from the MapInfo environment I would reach for an SQL Select to do this task, but I’m less familiar with SQL in QGIS.



Gordon Wakelin-King







From: QGIS-User <qgis-user-bounces at lists.osgeo.org> On Behalf Of chris hermansen via QGIS-User
Sent: Wednesday, March 13, 2024 8:02 AM
To: Michael.Dodd <michael.dodd at open.ac.uk>
Cc: Frederik-Matti Bartels (CAPSA) <bartels at capsa-building.org>; qgis-user at lists.osgeo.org
Subject: Re: [Qgis-user] editing and saving very large table



Michael and list,



On Tue, Mar 12, 2024 at 11:48 AM Michael.Dodd <michael.dodd at open.ac.uk<mailto:michael.dodd at open.ac.uk>> wrote:

In this particular instance the data are not spatial data, at this stage at least. There is no link to location.



The data does have a hierarchy according to several columns, by sorting I can put the data into a suitable order for editing in further information into an extra column. Without sorting the places where the extra info and which specific extra info are impossible to find and check as they are scattered all over.



I tried putting the information in using the field calculator and even though the simple expression seemed correct and there was no error message it did not put the information in. So I was even thinking of putting the information in by hand once sorted although suspect there may be issues with this too.



I was wondering about sorting the whole file then copying out/exporting chunks of a million rows at a time to go into a normal spreadsheet. If the copying out/export would not work directly then saving the sorted version of the data and chopping up using e.g. notepad++ and editing with spreadsheet. It is important to do the sorting before chopping up into the 5 separate files.



It is only later that spatial information is attached, that is not relevant at this stage as the spatial data will not be attached to all rows.



Ok so you are sorting a text file, presumably something like a .CSV.  Perhaps you are working on Windows.



What are your options to sort the file before you put it in QGIS?  It seems your spreadsheet won't accommodate the number of lines in your file; what about the "sort" utility in say Windows PowerShell?



Here's an article that talks about sorting .CSV files that way.



https://stackoverflow.com/questions/23603397/sorting-csv-using-windows-sort which in turn refers to this article



https://devblogs.microsoft.com/scripting/hey-scripting-guy-how-can-i-use-windows-powershell-to-sort-a-csv-file/



Alternatively, if you're using Linux or (I would imagine) OS/X, you can use the Posix "sort" utility.

--

Chris Hermansen · clhermansen "at" gmail "dot" com

C'est ma façon de parler.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20240313/61d87e2d/attachment.htm>