[Qgis-user] Very Slow Table Operations in QGIS 3.2.3

[Luigi Pirelli]

tnx Kelly, this is your issue and it's ok:
https://issues.qgis.org/issues/20008

to everyone, please add any comment or info that can help to replicate/solve

Luigi Pirelli

**************************************************************************************************
* LinkedIn: https://www.linkedin.com/in/luigipirelli
* Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
* GitHub: https://github.com/luipir
* Mastering QGIS 2nd Edition:
*
https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
* Hire me: http://goo.gl/BYRQKg
**************************************************************************************************


On Wed, 3 Oct 2018 at 22:40, Kelley Jabr <kjabr at indeed.com> wrote:

> Okay, I've created an issue for this situation. Hopefully I've done it
> satisfactorily. Thanks again!
>
> On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann <a.neumann at carto.net>
> wrote:
>
>> Hi,
>>
>> Glad to hear that it helped to run the operations on a closed table.
>> Nevertheless, as Luigi said, it would make sense to submit an issue report
>> about this slowness.
>>
>> The other thing one has to keep on mind:
>>
>> The SQLite API is kind of limited. Only until recently, if you wanted to
>> rename a field, one had to copy the whole table with the renamed field
>> changed in the new table - then the old table had to be removed (all in the
>> background), because there was API to rename columns in SQLite. I think
>> with the very recent version, which is not rolled out into most OS, this
>> has changed. You'd also need a very recent GDAL/OGR.
>> Greetings,
>> Andreas
>>
>> Am 03.10.2018 um 21:20 schrieb Kelley Jabr:
>>
>> Luigi, I'm pretty sure you just solved my problem. I tried a few
>> operations with the table closed and it was like night and day. That 400k
>> field calculator finished in mere moments! Thank you!!
>>
>> On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli <luipir at gmail.com> wrote:
>>
>>> can you test doing operations with field calculator without having the
>>> attribute table opened? just a workaround could be to use a virtual layer
>>> then save the entire layer later.
>>>
>>> Luigi Pirelli
>>>
>>>
>>> **************************************************************************************************
>>> * LinkedIn: https://www.linkedin.com/in/luigipirelli
>>> * Stackexchange: http://gis.stackexchange.com/users/19667/luigi-pirelli
>>> * GitHub: https://github.com/luipir
>>> * Mastering QGIS 2nd Edition:
>>> *
>>> https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition
>>> * Hire me: http://goo.gl/BYRQKg
>>>
>>> **************************************************************************************************
>>>
>>>
>>> On Wed, 3 Oct 2018 at 18:11, Kelley Jabr <kjabr at indeed.com> wrote:
>>>
>>>> Hello,
>>>>
>>>> I'm working with a geopackage on ~530k features, and just about
>>>> everything I try to do in the table takes way longer than expected.
>>>>
>>>>    - Adding a field took 5 minutes (table now has 22 total fields, so
>>>>    not massive)
>>>>    - Field Calculator on ~32k selected records took 8 minutes
>>>>    (literally all I was putting in the cells was: N)
>>>>    - Field calculator on ~480k records took so many hours that I had
>>>>    to leave it running over the weekend and when I came back, the Field
>>>>    Calculator had "closed unexpectedly"
>>>>
>>>> (I'm running on a linux machine with an Intel Core i7 and 32GB of
>>>> memory.)
>>>>
>>>> Has anyone else experience similar issues? Any advice or
>>>> recommendations for how to improve performance here?
>>>>
>>>> Thank you,
>>>>
>>>> --
>>>>
>>>> Kelley Jabr
>>>>
>>>> Analyst, Geographic Data Operations
>>>>
>>>> kjabr at indeed.com
>>>>
>>>> _______________________________________________
>>>> Qgis-user mailing list
>>>> Qgis-user at lists.osgeo.org
>>>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>>
>>>
>>
>> --
>>
>> Kelley Jabr
>>
>> Analyst, Geographic Data Operations
>>
>> kjabr at indeed.com
>>
>> Indeed - We help people get jobs.
>>
>> Indeed.com <http://www.indeed.com>
>>
>>
>>
>> _______________________________________________
>> Qgis-user mailing listQgis-user at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>>
>>
>> _______________________________________________
>> Qgis-user mailing list
>> Qgis-user at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
>
>
>
> --
>
> Kelley Jabr
>
> Analyst, Geographic Data Operations
>
> kjabr at indeed.com
>
> Indeed - We help people get jobs.
>
> Indeed.com <http://www.indeed.com>
>
> _______________________________________________
> Qgis-user mailing list
> Qgis-user at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-user
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20181003/0dc52ec0/attachment.html>