[Qgis-user] Very Slow Table Operations in QGIS 3.2.3

Wed Oct 3 14:54:34 PDT 2018

Hi,

In SQL queries on databases like Postgresql it is common to use the Limit
operand to ask just a sample of the output to check if all looks OK without
reading the all table.

This situation made me wonder if this couldn't be a nice feature request to
the layers attribute table when working with very large tables: to only
show/load the first X features.

Obvious operations like selection or field calculator on the table should
affect the all table, and not just the visible layers.

I am not sure if it would have an impact on how fast working with the table
looks. What Kelley case seems to show, is that loading and showing the
table is where the problem is, not in the operations on the attributes.

What do u guy think?

Best regards,

Alex Neto

A qua, 3/10/2018, 21:54, Luigi Pirelli <luipir at gmail.com> escreveu:

> tnx Kelly, this is your issue and it's ok:
> https://issues.qgis.org/issues/20008
>
> to everyone, please add any comment or info that can help to
> replicate/solve
>
> Luigi Pirelli
>
>
> **************************************************************************************************
> * LinkedIn: https://www.linkedin.com/in/luigipirelli
> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
> * GitHub: https://github.com/luipir
> * Mastering QGIS 2nd Edition:
> *
> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
> * Hire me: http://goo.gl/BYRQKg
>
> **************************************************************************************************
>
>
> On Wed, 3 Oct 2018 at 22:40, Kelley Jabr <kjabr at indeed.com> wrote:
>
>> Okay, I've created an issue for this situation. Hopefully I've done it
>> satisfactorily. Thanks again!
>>
>> On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann <a.neumann at carto.net>
>> wrote:
>>
>>> Hi,
>>>
>>> Glad to hear that it helped to run the operations on a closed table.
>>> Nevertheless, as Luigi said, it would make sense to submit an issue report
>>> about this slowness.
>>>
>>> The other thing one has to keep on mind:
>>>
>>> The SQLite API is kind of limited. Only until recently, if you wanted to
>>> rename a field, one had to copy the whole table with the renamed field
>>> changed in the new table - then the old table had to be removed (all in the
>>> background), because there was API to rename columns in SQLite. I think
>>> with the very recent version, which is not rolled out into most OS, this
>>> has changed. You'd also need a very recent GDAL/OGR.
>>> Greetings,
>>> Andreas
>>>
>>> Am 03.10.2018 um 21:20 schrieb Kelley Jabr:
>>>
>>> Luigi, I'm pretty sure you just solved my problem. I tried a few
>>> operations with the table closed and it was like night and day. That 400k
>>> field calculator finished in mere moments! Thank you!!
>>>
>>> On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli <luipir at gmail.com> wrote:
>>>
>>>> can you test doing operations with field calculator without having the
>>>> attribute table opened? just a workaround could be to use a virtual layer
>>>> then save the entire layer later.
>>>>
>>>> Luigi Pirelli
>>>>
>>>>
>>>> **************************************************************************************************
>>>> * LinkedIn: https://www.linkedin.com/in/luigipirelli
>>>> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
>>>> * GitHub: https://github.com/luipir
>>>> * Mastering QGIS 2nd Edition:
>>>> *
>>>> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
>>>> * Hire me: http://goo.gl/BYRQKg
>>>>
>>>> **************************************************************************************************
>>>>
>>>>
>>>> On Wed, 3 Oct 2018 at 18:11, Kelley Jabr <kjabr at indeed.com> wrote:
>>>>
>>>>> Hello,
>>>>>
>>>>> I'm working with a geopackage on ~530k features, and just about
>>>>> everything I try to do in the table takes way longer than expected.
>>>>>
>>>>>    - Adding a field took 5 minutes (table now has 22 total fields, so
>>>>>    not massive)
>>>>>    - Field Calculator on ~32k selected records took 8 minutes
>>>>>    (literally all I was putting in the cells was: N)
>>>>>    - Field calculator on ~480k records took so many hours that I had
>>>>>    to leave it running over the weekend and when I came back, the Field
>>>>>    Calculator had "closed unexpectedly"
>>>>>
>>>>> (I'm running on a linux machine with an Intel Core i7 and 32GB of
>>>>> memory.)
>>>>>
>>>>> Has anyone else experience similar issues? Any advice or
>>>>> recommendations for how to improve performance here?
>>>>>
>>>>> Thank you,
>>>>>
>>>>> --
>>>>>
>>>>> Kelley Jabr
>>>>>
>>>>> Analyst, Geographic Data Operations
>>>>>
>>>>> kjabr at indeed.com
>>>>>
>>>>> _______________________________________________
>>>>> Qgis-user mailing list
>>>>> Qgis-user at lists.osgeo.org
>>>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>>>
>>>>
>>>
>>> --
>>>
>>> Kelley Jabr
>>>
>>> Analyst, Geographic Data Operations
>>>
>>> kjabr at indeed.com
>>>
>>> Indeed - We help people get jobs.
>>>
>>> Indeed.com <http://www.indeed.com>
>>>
>>>
>>>
>>> _______________________________________________
>>> Qgis-user mailing listQgis-user at lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>>
>>>
>>> _______________________________________________
>>> Qgis-user mailing list
>>> Qgis-user at lists.osgeo.org
>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
>>
>>
>> --
>>
>> Kelley Jabr
>>
>> Analyst, Geographic Data Operations
>>
>> kjabr at indeed.com
>>
>> Indeed - We help people get jobs.
>>
>> Indeed.com <http://www.indeed.com>
>>
>> _______________________________________________
>> Qgis-user mailing list
>> Qgis-user at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user

-- 
Alexandre Neto
---------------------
@AlexNetoGeo
http://sigsemgrilhetas.wordpress.com
http://gisunchained.wordpress.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20181003/f321bf31/attachment.html>