[Qgis-user] Adding feature to PostGIS layer that has joins is really slow

[Nuno Lopes]

Hi Chris, thanks for the suggestions.
I have opened a bug with QGIS that was accepted
https://github.com/qgis/QGIS/issues/36167 .
In the meantime I'l have to go with one of the alternatives (still figuring
out which one is best for my case)

Thanks again,
Nuno Lopes

On Sun, May 3, 2020 at 5:02 PM chris hermansen <clhermansen at gmail.com>
wrote:

> Nuno and list
>
> On Sun, May 3, 2020, 04:50 Nuno Lopes <nlopes at gmail.com> wrote:
>
>> Hi Chris and list,
>>
>> This is the minimal setup where I could reproduce my performance issue.
>> The final idea is to have multiple 1:0-1 tables that have different
>> mandatory and optional fields. So putting all this in a single table would
>> be very difficult (I can see it done using triggers and checks on the
>> database but UX would be bad)
>>
>
> Normally one would allow nulls for optional attributes rather than
> separate them into different tables.
>
> "UX would be bad" - how so? If you are building CRUD forms they can view a
> semantic type field on the table and respond to that.
>
> Another possibility with PostgreSQL if the nature of your problem is the
> kind of thing where the semantic type of point is different (retail stores,
> clinics, churches, service stations, post offices, etc) resulting in a
> different collection of attributes in each case is to use table inheritance
> (composition). Then the spatial table with the semantic type indicator is
> the base "class" and all the different types are built as extensions from
> that.
>
>
>
>> Unfortunately the index doesn't solve my problem, because in this minimal
>> setup the table is empty and I get this performance issue. This problem
>> exists with a database on my own computer, if I move the database to a
>> server (even a close one) QGIS will freeze for so much time that I have to
>> kill it.
>>
>
> Ok, so you have to get rid of the separate query for each point then.
>
> Which to me means letting PostgreSQL manage the table joins, using a view
> or using composition or just putting the attributes on your table since
> they are 1:0-1 which can be managed by null values.
>
>>
>> Thanks a lot,
>> Nuno Lopes
>>
>> On Sun, May 3, 2020 at 12:43 AM chris hermansen <clhermansen at gmail.com>
>> wrote:
>>
>>> Nuno and list
>>>
>>> On Sat, May 2, 2020 at 11:58 AM Nuno Lopes <nlopes at gmail.com> wrote:
>>>
>>>> Hello, I have already posted this on
>>>> https://gis.stackexchange.com/questions/360171/adding-feature-to-postgis-layer-that-has-joins-is-really-slow
>>>> , just posting here to see if someone else has ideas.
>>>>
>>>> I have a large project that I've been setting up to work with QGIS 3.12
>>>> and I have now hit a problem with using "joins" to join several attribute
>>>> tables with a geometric one.
>>>>
>>>> I have reduced the problem to a minimal setup that has only two layers
>>>> with a one to one (or zero) relation:
>>>>
>>>>  1. `point_layer` - a layer with a geometry (points) that has 64k
>>>> entries
>>>>  2. `attribute_layer` - a layer with one text field and a foreign key
>>>> to the `point_layer`
>>>>
>>>> On the `point_layer` I added a join to the `attribute_layer` so that we
>>>> can edit the attributes directly in the feature form.
>>>>
>>>> After doing this whenever I try to create a point on the `point_layer`
>>>> **it takes more than 1 min to create the feature**.
>>>>
>>>> By examining the PostgreSQL log I found out that QGIS is querying each
>>>> feature on the `point_layer` for it's relation on the `attribute_layer`.
>>>> These queries are made one by one, so even though I'm running the database
>>>> on my own computer, the sheer number of queries leads to very poor
>>>> performance.
>>>>
>>>> I've already tried tweaking the cache config on the joins, hiding the
>>>> form on add feature and enabling the "automatically create transaction
>>>> groups where possible" to no avail.
>>>>
>>>> I've also tested older QGIS versions 3.4 and 3.8 without success.
>>>>
>>>> Am I doing anything wrong? Is there any way to work around this?
>>>>
>>>> The schema used is:
>>>>
>>>>     CREATE TABLE point_layer (
>>>>         ogc_fid integer PRIMARY KEY,
>>>>         geom public.geometry(GeometryZ,3763),
>>>>         height double precision
>>>>     );
>>>>
>>>>     CREATE INDEX point_layer_geom_idx ON point_layer USING gist (geom);
>>>>
>>>>     CREATE TABLE attribute_layer (
>>>>         id SERIAL PRIMARY KEY,
>>>>         fid integer REFERENCES point_layer(ogc_fid),
>>>>         some_value text
>>>>     );
>>>>
>>>> The query that is made repeatedly is (the fid changes):
>>>>
>>>>     SELECT "id","some_value"::text,"fid"::text
>>>>     FROM "public"."attribute_layer"
>>>>     WHERE ("fid" = ?) LIMIT 1
>>>>
>>>> The explain analyze is (the table is empty):
>>>>
>>>>                                                       QUERY PLAN
>>>>
>>>>
>>>> ---------------------------------------------------------------------------------------------------------------
>>>>      Limit  (cost=0.00..4.17 rows=1 width=68) (actual time=0.014..0.014
>>>> rows=0 loops=1)
>>>>        ->  Seq Scan on test_relation  (cost=0.00..25.03 rows=6
>>>> width=68) (actual time=0.011..0.011 rows=0 loops=1)
>>>>              Filter: (fid = 29853)
>>>>      Planning Time: 0.506 ms
>>>>      Execution Time: 0.086 ms
>>>>
>>>>
>>>>
>>>> Why not just put your attributes on the same table as the spatial one?
>>>
>>> If you don't like that idea you could consider putting an index on the
>>> fid on the attribute_layer.  But it's odd to me to see the select on the
>>> attribute_layer table given that your fid in your attribute_layer is a
>>> foreign key referencing the ogc_fid field in point_layer.
>>>
>>>
>>> --
>>> Chris Hermansen · clhermansen "at" gmail "dot" com
>>>
>>> C'est ma façon de parler.
>>>
>>
>>
>> --
>> --
>> Nuno Lopes
>>
>

-- 
--
Nuno Lopes
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20200505/adfa557d/attachment.html>