[Qgis-user] editing and saving very large table

[chris hermansen]

Michael and list,

On Tue, Mar 12, 2024 at 11:48 AM Michael.Dodd <michael.dodd at open.ac.uk>
wrote:

> In this particular instance the data are not spatial data, at this stage
> at least. There is no link to location.
>
>
>
> The data does have a hierarchy according to several columns, by sorting I
> can put the data into a suitable order for editing in further information
> into an extra column. Without sorting the places where the extra info and
> which specific extra info are impossible to find and check as they are
> scattered all over.
>
>
>
> I tried putting the information in using the field calculator and even
> though the simple expression seemed correct and there was no error message
> it did not put the information in. So I was even thinking of putting the
> information in by hand once sorted although suspect there may be issues
> with this too.
>
>
>
> I was wondering about sorting the whole file then copying out/exporting
> chunks of a million rows at a time to go into a normal spreadsheet. If the
> copying out/export would not work directly then saving the sorted version
> of the data and chopping up using e.g. notepad++ and editing with
> spreadsheet. It is important to do the sorting before chopping up into the
> 5 separate files.
>
>
>
> It is only later that spatial information is attached, that is not
> relevant at this stage as the spatial data will not be attached to all rows.
>
>
Ok so you are sorting a text file, presumably something like a .CSV.
Perhaps you are working on Windows.

What are your options to sort the file before you put it in QGIS?  It seems
your spreadsheet won't accommodate the number of lines in your file; what
about the "sort" utility in say Windows PowerShell?

Here's an article that talks about sorting .CSV files that way.

https://stackoverflow.com/questions/23603397/sorting-csv-using-windows-sort
which in turn refers to this article

https://devblogs.microsoft.com/scripting/hey-scripting-guy-how-can-i-use-windows-powershell-to-sort-a-csv-file/

Alternatively, if you're using Linux or (I would imagine) OS/X, you can use
the Posix "sort" utility.
-- 
Chris Hermansen · clhermansen "at" gmail "dot" com

C'est ma façon de parler.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20240312/142c5f3d/attachment.htm>