[QGIS-Developer] QGIS project maintainability, forms and styles, xml verbosity

[Nyall Dawson]

On Tue, 30 Jul 2024 at 20:58, Régis Haubourg via QGIS-Developer <
qgis-developer at lists.osgeo.org> wrote:
> In short, with the growing qgis features, but also datasets becoming
larger (more columns) too, QGIS projects are becoming heavier and heavier,
and require a lot of effort to maintain when trying to ship them as part of
applications or with datasets.

Thanks for raising this discussion -- I agree that it's an important issue,
and one we should address.

> ## First issue, forms are considered as being a part of each style.
>
> This make the qgs xml replicate 6 times the same form, in my case. It
makes it very hard to maintain a single source of truth for this form also.
The idea of having on form per style could be elegant, but it is a path
full of caveats and I suspect very few of us are using this as a feature.

+1. I think that per-style form configuration would be very rarely used,
and should definitely be opt-in rather than something which happens by
default. It's an absolute PAIN to synchronize form configuration across
styles!

A solution could be to move the existing "Show form on ..." combo box from
the form tab into a new "form settings" group, and then add a new checkbox
in there for "Allow different form configuration for different styles"
(which would be off by default for all newly added layers).

I'm sure a lot of users would appreciate this!

>
> ## Second issue, verbosity of the xml
>
> The properties of those forms have grown in time, with many new features
(default values, data defined fonts etc.. ). This is cool.
>
> However in average day use, most of those settings are left as default
values, still QGIS will write every property for each column in the xml.
Ex: default value, hidden status, LabelonTop, etc.  This is highly
denormalized and verbose, mostly for unset properties.  It is so much
denormalized that a simple QGIS xml can weight 18MB and be zipped down to
977 KB.

Agreed.


> Regarding the xml verbosity, suppose we probably could find solutions:
>
>
>  - don't write properties that are left untouched or default (which may
lead to strange behavior when defaults change)

I'd lean toward -0 for this, just because we do occasionally change the
default and the impact of a setting changing in a user's existing project
can be extreme. Rather I'd go with a "don't write properties for properties
which are unset" approach. Eg if a layer doesn't have a vertical CRS set
for, we don't write ANYTHING to the XML (as opposed to verticalCrs="" or
similar). It's a trivial change in code (i.e. checking for an empty string
before writing the attribute) but one which has to be explicitly made, so
we'd need to educate all our developers/reviewers that this is the desired
approach.

>  - don't repeat unuseful informations. ie, layer tree block repeats
provider and datasource even if it is not used. Confusion appears from time
to time when user try to save or modify the xml file because of this
>  - find a less verbose xml serialization if QT allows this. For instance,
instead of on xml block per property, why not have on block per field. For
instance :

*snip*

This one is rather tricky, as it's completely independent pieces of code
which are creating these elements. We'd have to be very careful not to
muddy the QGIS API by tightly linking objects which should remain
independent. I think if we're looking at fixing this properly then a better
approach would be to remove our hard dependency on XML and the Qt DOM
classes, and swap out to some custom class which allows us to freely change
the background format entirely (JSON!). We could even do this without
breaking the API, by "cheating" and making a QgsObjectStorage (?) class
which exposes the same public API as QDomDocument/QDomElement and then
using this throughout our API. (There'd be no break for PyQGIS, because
these objects expose the same API! 😂). And then we'd be free to make this
custom class write/read to/from XML, JSON, YML, .... whatever we want.

Even with no other changes in QGIS code a switch to a JSON format would
save a LOT of wasted characters, because we won't have any of those
"duplicate" end tags anymore. We could also avoid all these type="QString",
type="Map" strings because the type would be discernable without having to
explicitly write it out.

Nyall

>
> ```xml
>
>         <editable>
>             [here dozen of fields]
>             <field editable="1"
name="adedpe202006_l_ch_gen_princ"></field>
>         </editable>
>         <labelOnTop>
>             [here dozen of fields, again]
>             <field labelOnTop="0"
name="adedpe202006_l_ch_gen_princ"></field>
>         <labelOnTop>
> ```
>
> could be
>
> ```xml
> ```
> <form>
>   ...
>    <fields>
>             <field name="adedpe202006_l_ch_gen_princ" labelOnTop="0"
editable="1" ></field>
>    </fields>
>
> <form>
> ```
>
>
> To be fair, I have no funding at the moment, but I can try to make it
happen, if we reach a technical consensus here. Any cofunding or grant
proposal would obviously help here. I'm pretty sure companies editing
desktop apps using QGIS could find an interest here.  This thread will
probably raise the question of a new generation project file, but that is
not my intent ;)
>
> Let me know what you think
>
> Cheers
>
> Régis
>
>
> [0] qgs available in those open data sets here with a GPKG database
https://open-data.s3.fr-par.scw.cloud/bdnb_millesime_2023-01-a/millesime_2023-01-a_dep19/open_data_millesime_2023-01-a_dep19_gpkg.zip
>
> _______________________________________________
> QGIS-Developer mailing list
> QGIS-Developer at lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20240731/98ead637/attachment.htm>