<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body text="#FFFFFF" bgcolor="#000000">
<p>Hi Calvin,</p>
<p>This definitely sounds like the sort of thing you want to be
doing in the data provider and not in QGIS!</p>
<p>Maybe Fast Text Search would be of use. Problem is that I'm not
sure if it's included in the GeoPackage spec or QGIS' SQLite
install, but it's in the default pre-compiled SQLite.
<a class="moz-txt-link-freetext" href="https://www.sqlite.org/fts5.html">https://www.sqlite.org/fts5.html</a></p>
<p>Using that for textual searches and more conventional Operators
for numerical should get you the speed of SQLite with the
flexibility you desire.</p>
<p>Cheers,</p>
<p>Jonathan<br>
</p>
<div class="moz-cite-prefix">On 29/07/2021 14:23, C Hamilton wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CABPxTTobOhS-rvkGkqyKzMN1X_0Z8tAqyhri5J6H_P9umsnZjw@mail.gmail.com">
<div dir="ltr">Ultimately what I am trying to do is to revamp the
Search Layers plugin (<a
href="https://plugins.qgis.org/plugins/searchlayers/"
moz-do-not-send="true">https://plugins.qgis.org/plugins/searchlayers/</a>)
to allow searching using two different strings with NOT, AND, OR
operators, case dependent or independent, with qualifiers where
the strings are an exact match, within the string, begins with,
and ends with. The plugin allows for searching across all layers
and all attributes, or optionally a particular layer and
attribute. The two strings may be required to match within each
attribute or match in any of the attributes within the feature.
I am trying to make it very flexible and easy to use. I know
that QGIS has the Locator interface, but the users like the
extra interactivity that Search Layers provides.
<div><br>
</div>
<div>Here is the issue: For some of the searches I am creating a
regular expression that matches the search parameters and
iterating through each feature and then each attribute using <b>re.search</b>
to do the comparison. Unfortunately, this is very slow. On a
2.2 million data set it can take 5-10 minutes to do the
search. This is way too slow. The SQL search is less than 5
seconds. I am using a <b>QgsFeatureRequest </b>and then
setting <b>request.setFilterExpression</b>. I am only using
this when searching on a single layer and single attribute and
I don't think there is the flexibility that there is with the
regular expressions, but I don't really use SQL and don't know
what is possible. It sounds like the SQL searching may be
limited by the vector file type source.</div>
<div><br>
</div>
<div>Are there any other ways for searching that I am unaware
of? The results of the search need to identify the feature in
the layer and what attribute or attributes matched for each of
the two strings and then I can apply the NOT, AND, OR
operators. I am trying to satisfy user's requests to make
Search Layers into what they want it to be and am finding it
difficult to satisfy speed, vector file types, and
flexibility. I welcome any suggestions. Ultimately I might
just have to constrain the file types supported and say to
convert the data first.</div>
<div><br>
</div>
<div>Thanks,</div>
<div><br>
</div>
<div>Calvin</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Jul 29, 2021 at 1:42
AM Nyall Dawson <<a href="mailto:nyall.dawson@gmail.com"
moz-do-not-send="true">nyall.dawson@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote">On Thu, 29 Jul 2021 at 15:31,
Matthias Kuhn <<a href="mailto:matthias@opengis.ch"
target="_blank" moz-do-not-send="true">matthias@opengis.ch</a>>
wrote:<br>
><br>
> Can you fill a bug for this? The expression compiler in
QGIS should not forward it to ogr/gpkg/sqlite in this case.<br>
<br>
My interpretation is that Calvin is setting a layer filter, so
QGIS<br>
expressions aren't involved (i.e. no bug).<br>
<br>
Nyall<br>
<br>
><br>
> Matthias<br>
><br>
> On Wed, Jul 28, 2021 at 10:08 PM C Hamilton <<a
href="mailto:adenaculture@gmail.com" target="_blank"
moz-do-not-send="true">adenaculture@gmail.com</a>> wrote:<br>
>><br>
>> Thanks everyone for that explanation. This does
complicate the ability to set an expression filter in a plugin
when it depends on the data source type. I'm not sure what to
do about that.<br>
>><br>
>> Regards,<br>
>> Calvin<br>
>><br>
>> On Wed, Jul 28, 2021 at 8:00 PM Anita Graser <<a
href="mailto:anitagraser@gmx.at" target="_blank"
moz-do-not-send="true">anitagraser@gmx.at</a>> wrote:<br>
>>><br>
>>> Hi Calvin,<br>
>>><br>
>>> 28 Jul 2021 21:43:40 C Hamilton <<a
href="mailto:adenaculture@gmail.com" target="_blank"
moz-do-not-send="true">adenaculture@gmail.com</a>>:<br>
>>> > If I use LIKE instead of ILIKE then it
works. It is like ILIKE was not implemented for GeoPackage.
Can anyone verify this?<br>
>>><br>
>>> Yes, there seems to be no iLike in sqlite:<br>
>>><br>
>>> <a
href="https://stackoverflow.com/questions/15480319/case-sensitive-and-insensitive-like-in-sqlite"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://stackoverflow.com/questions/15480319/case-sensitive-and-insensitive-like-in-sqlite</a><br>
>>><br>
>>> And therefore neither in geopackage.<br>
>>><br>
>>> Regards<br>
>>> Anita<br>
>><br>
>> _______________________________________________<br>
>> QGIS-Developer mailing list<br>
>> <a href="mailto:QGIS-Developer@lists.osgeo.org"
target="_blank" moz-do-not-send="true">QGIS-Developer@lists.osgeo.org</a><br>
>> List info: <a
href="https://lists.osgeo.org/mailman/listinfo/qgis-developer"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br>
>> Unsubscribe: <a
href="https://lists.osgeo.org/mailman/listinfo/qgis-developer"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br>
><br>
> _______________________________________________<br>
> QGIS-Developer mailing list<br>
> <a href="mailto:QGIS-Developer@lists.osgeo.org"
target="_blank" moz-do-not-send="true">QGIS-Developer@lists.osgeo.org</a><br>
> List info: <a
href="https://lists.osgeo.org/mailman/listinfo/qgis-developer"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br>
> Unsubscribe: <a
href="https://lists.osgeo.org/mailman/listinfo/qgis-developer"
rel="noreferrer" target="_blank" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/qgis-developer</a><br>
</blockquote>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
Qgis-user mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Qgis-user@lists.osgeo.org">Qgis-user@lists.osgeo.org</a>
List info: <a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-user">https://lists.osgeo.org/mailman/listinfo/qgis-user</a>
Unsubscribe: <a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/qgis-user">https://lists.osgeo.org/mailman/listinfo/qgis-user</a>
</pre>
</blockquote>
</body>
</html>