Hi, <div><br></div><div>In SQL queries on databases like Postgresql it is common to use the Limit operand to ask just a sample of the output to check if all looks OK without reading the all table. </div><div><br></div><div>This situation made me wonder if this couldn't be a nice feature request to the layers attribute table when working with very large tables: to only show/load the first X features. </div><div><br></div><div>Obvious operations like selection or field calculator on the table should affect the all table, and not just the visible layers. </div><div><br></div><div>I am not sure if it would have an impact on how fast working with the table looks. What Kelley case seems to show, is that loading and showing the table is where the problem is, not in the operations on the attributes. </div><div><br></div><div>What do u guy think? </div><div><br></div><div>Best regards, </div><div><br></div><div>Alex Neto</div><div><br></div><div><br><br><div class="gmail_quote"><div dir="ltr">A qua, 3/10/2018, 21:54, Luigi Pirelli <<a href="mailto:luipir@gmail.com">luipir@gmail.com</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex" id="gmail_block_quote0"><div dir="ltr"><div dir="ltr">tnx Kelly, this is your issue and it's ok: <a href="https://issues.qgis.org/issues/20008" target="_blank">https://issues.qgis.org/issues/20008</a></div><div dir="ltr"><br></div><div>to everyone, please add any comment or info that can help to replicate/solve</div></div><div dir="ltr"><div dir="ltr"><br clear="all"><div><div dir="ltr" class="m_2593813617731198861gmail_signature">Luigi Pirelli<br><br>**************************************************************************************************<br>* LinkedIn: <a href="https://www.linkedin.com/in/luigipirelli" target="_blank">https://www.linkedin.com/in/luigipirelli</a><br>* Stackexchange: <a href="http://gis.stackexchange.com/users/19667/luigi-pirelli" target="_blank">http://gis.stackexchange.com/users/19667/luigi-pirelli</a><br>* GitHub: <a href="https://github.com/luipir" target="_blank">https://github.com/luipir</a><br>* Mastering QGIS 2nd Edition:<br>* <a href="https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition" target="_blank">https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition</a><br>* Hire me: <a href="http://goo.gl/BYRQKg" target="_blank">http://goo.gl/BYRQKg</a><br>**************************************************************************************************</div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Wed, 3 Oct 2018 at 22:40, Kelley Jabr <<a href="mailto:kjabr@indeed.com" target="_blank">kjabr@indeed.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Okay, I've created an issue for this situation. Hopefully I've done it satisfactorily. Thanks again!</div><br><div class="gmail_quote"><div dir="ltr">On Wed, Oct 3, 2018 at 2:31 PM Andreas Neumann <<a href="mailto:a.neumann@carto.net" target="_blank">a.neumann@carto.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF">
<p>Hi,</p>
<p>Glad to hear that it helped to run the operations on a closed
table. Nevertheless, as Luigi said, it would make sense to submit
an issue report about this slowness.<br>
</p>
<p>The other thing one has to keep on mind:</p>
<p>The SQLite API is kind of limited. Only until recently, if you
wanted to rename a field, one had to copy the whole table with the
renamed field changed in the new table - then the old table had to
be removed (all in the background), because there was API to
rename columns in SQLite. I think with the very recent version,
which is not rolled out into most OS, this has changed. You'd also
need a very recent GDAL/OGR.<br>
</p>
Greetings,<br>
Andreas<br>
<br>
<div class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723moz-cite-prefix">Am 03.10.2018 um 21:20 schrieb Kelley
Jabr:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Luigi, I'm pretty sure you just solved my problem.
I tried a few operations with the table closed and it was like
night and day. That 400k field calculator finished in mere
moments! Thank you!!</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, Oct 3, 2018 at 1:42 PM Luigi Pirelli <<a href="mailto:luipir@gmail.com" target="_blank">luipir@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">can you test doing operations with field
calculator without having the attribute table opened? just a
workaround could be to use a virtual layer then save the
entire layer later.
<div>
<div><br clear="all">
<div>
<div dir="ltr" class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723m_-237128707789312989gmail_signature" data-smartmail="gmail_signature">Luigi Pirelli<br>
<br>
**************************************************************************************************<br>
* LinkedIn: <a href="https://www.linkedin.com/in/luigipirelli" target="_blank">https://www.linkedin.com/in/luigipirelli</a><br>
* Stackexchange: <a href="http://gis.stackexchange.com/users/19667/luigi-pirelli" target="_blank">http://gis.stackexchange.com/users/19667/luigi-pirelli</a><br>
* GitHub: <a href="https://github.com/luipir" target="_blank">https://github.com/luipir</a><br>
* Mastering QGIS 2nd Edition:<br>
* <a href="https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition" target="_blank">https://www.packtpub.com/big-data-and-business-intelligence/mastering-qgis-second-edition</a><br>
* Hire me: <a href="http://goo.gl/BYRQKg" target="_blank">http://goo.gl/BYRQKg</a><br>
**************************************************************************************************</div>
</div>
<br>
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">On Wed, 3 Oct 2018 at 18:11, Kelley Jabr <<a href="mailto:kjabr@indeed.com" target="_blank">kjabr@indeed.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">Hello,
<div><br>
</div>
<div>I'm working with a geopackage on ~530k features,
and just about everything I try to do in the table
takes way longer than expected. </div>
<div>
<ul>
<li>Adding a field took 5 minutes (table now has 22
total fields, so not massive)<br>
</li>
<li>Field Calculator on ~32k selected records took 8
minutes (literally all I was putting in the cells
was: N)</li>
<li>Field calculator on ~480k records took so many
hours that I had to leave it running over the
weekend and when I came back, the Field Calculator
had "closed unexpectedly"</li>
</ul>
<div>(I'm running on a linux machine with an Intel
Core i7 and 32GB of memory.)</div>
<div><br>
</div>
<div>Has anyone else experience similar issues? Any
advice or recommendations for how to improve
performance here? </div>
<div><br>
</div>
<div>Thank you,</div>
<div><br>
</div>
-- <br>
<div dir="ltr" class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723m_-237128707789312989m_7860919606024074194gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div dir="ltr"><span>
<p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">Kelley Jabr</span></p>
<p style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><font color="#000000" face="Helvetica Neue"><span style="font-size:13.3333px;white-space:pre-wrap">Analyst, Geographic Data Operations</span></font></p>
<p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,255);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><a href="mailto:kjabr@indeed.com" target="_blank">kjabr@indeed.com</a></span></p>
<div><br>
</div>
</span></div>
</div>
</div>
</div>
</div>
_______________________________________________<br>
Qgis-user mailing list<br>
<a href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a></blockquote>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723gmail_signature" data-smartmail="gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr"><span>
<p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">Kelley Jabr</span></p>
<p style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><font color="#000000" face="Helvetica Neue"><span style="font-size:13.3333px;white-space:pre-wrap">Analyst, Geographic Data Operations</span></font></p>
<p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,255);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><a href="mailto:kjabr@indeed.com" target="_blank">kjabr@indeed.com</a></span></p>
<br>
<p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,255);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Indeed</span><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(255,153,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"> - We help people get jobs.</span></p>
<p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,255);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><a href="http://www.indeed.com" target="_blank">Indeed.com</a></span></p>
<div><br>
</div>
</span></div>
</div>
</div>
</div>
<br>
<fieldset class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723mimeAttachmentHeader"></fieldset>
<br>
<pre>_______________________________________________
Qgis-user mailing list
<a class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723moz-txt-link-abbreviated" href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a>
List info: <a class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-user" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a>
Unsubscribe: <a class="m_2593813617731198861m_7671550230968980835m_-5823976002201767723moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-user" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a></pre>
</blockquote>
<br>
</div>
_______________________________________________<br>
Qgis-user mailing list<br>
<a href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="m_2593813617731198861m_7671550230968980835gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><span><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,0);background-color:transparent;font-weight:700;vertical-align:baseline;white-space:pre-wrap">Kelley Jabr</span></p><p style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><font color="#000000" face="Helvetica Neue"><span style="font-size:13.3333px;white-space:pre-wrap">Analyst, Geographic Data Operations</span></font></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,255);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><a href="mailto:kjabr@indeed.com" target="_blank">kjabr@indeed.com</a></span></p><br><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,255);background-color:transparent;vertical-align:baseline;white-space:pre-wrap">Indeed</span><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(255,153,0);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"> - We help people get jobs.</span></p><p dir="ltr" style="line-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style="font-size:10pt;font-family:"Helvetica Neue";color:rgb(0,0,255);background-color:transparent;vertical-align:baseline;white-space:pre-wrap"><a href="http://www.indeed.com" target="_blank">Indeed.com</a></span></p><div><br></div></span></div></div></div></div>
_______________________________________________<br>
Qgis-user mailing list<br>
<a href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a></blockquote></div>
_______________________________________________<br>
Qgis-user mailing list<br>
<a href="mailto:Qgis-user@lists.osgeo.org" target="_blank">Qgis-user@lists.osgeo.org</a><br>
List info: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a><br>
Unsubscribe: <a href="https://lists.osgeo.org/mailman/listinfo/qgis-user" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/qgis-user</a></blockquote></div></div>-- <br><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div>Alexandre Neto</div><div>---------------------</div><div>@AlexNetoGeo</div><div><a href="http://sigsemgrilhetas.wordpress.com">http://sigsemgrilhetas.wordpress.com</a></div><a href="http://gisunchained.wordpress.com">http://gisunchained.wordpress.com</a><br></div></div>