[QGIS-Developer] QGIS project maintainability, forms and styles, xml verbosity

[Régis Haubourg]

Hi devs,


I'm trying to maintain a QGIS project
I am hit by some issues that I was already aware of, but this time it is 
way more painful than I faced before.

In short, with the growing qgis features, but also datasets becoming 
larger (more columns) too, QGIS projects are becoming heavier and 
heavier, and require a lot of effort to maintain when trying to ship 
them as part of applications or with datasets.

My dataset has around a dozen of tables, and one has a lot of columns, 
around 200 columns.  I know this is an not optimal data normalization to 
have so many columns. But as datascience grows, shipping gpkg or 
geoParquet files with lots of columns tends to be more frequent.

My main layer has 6 styles and each style has the same form definition 
to make it easy to read those attributes, and offer some 1-n relation 
nested forms (buildings and postal addresses).

## First issue, forms are considered as being a part of each style.

This make the qgs xml replicate 6 times the same form, in my case. It 
makes it very hard to maintain a single source of truth for this form 
also. The idea of having on form per style could be elegant, but it is a 
path full of caveats and I suspect very few of us are using this as a 
feature.

## Second issue, verbosity of the xml

The properties of those forms have grown in time, with many new features 
(default values, data defined fonts etc.. ). This is cool.

However in average day use, most of those settings are left as default 
values, still QGIS will write every property for each column in the 
xml.  Ex: default value, hidden status, LabelonTop, etc.  This is highly 
denormalized and verbose, mostly for unset properties.  It is so much 
denormalized that a simple QGIS xml can weight 18MB and be zipped down 
to 977 KB.


## So I would like to open a debate here about how to handle this.

As for the form being replicated in each style, I would propose to plan 
moving form definition up in the layer definition, not in style 
definition. This will probably imply an API break and retrocompatibility 
issues, so my best guess is that the only time frame to make it happen 
is QGIS 4.


Regarding the xml verbosity, suppose we probably could find solutions:


  - don't write properties that are left untouched or default (which may 
lead to strange behavior when defaults change)
  - don't repeat unuseful informations. ie, layer tree block repeats 
provider and datasource even if it is not used. Confusion appears from 
time to time when user try to save or modify the xml file because of this
  - find a less verbose xml serialization if QT allows this. For 
instance, instead of on xml block per property, why not have on block 
per field. For instance :

```xml

         <editable>
             [here dozen of fields]
             <field editable="1" name="adedpe202006_l_ch_gen_princ"></field>
         </editable>
         <labelOnTop>
             [here dozen of fields, again]
             <field labelOnTop="0" 
name="adedpe202006_l_ch_gen_princ"></field>
         <labelOnTop>
```

could be

```xml
```
<form>
   ...
    <fields>
             <field name="adedpe202006_l_ch_gen_princ" labelOnTop="0" 
editable="1" ></field>
    </fields>

<form>
```


To be fair, I have no funding at the moment, but I can try to make it 
happen, if we reach a technical consensus here. Any cofunding or grant 
proposal would obviously help here. I'm pretty sure companies editing 
desktop apps using QGIS could find an interest here.  This thread will 
probably raise the question of a new generation project file, but that 
is not my intent ;)

Let me know what you think

Cheers

Régis


[0] qgs available in those open data sets here with a GPKG database 
https://open-data.s3.fr-par.scw.cloud/bdnb_millesime_2023-01-a/millesime_2023-01-a_dep19/open_data_millesime_2023-01-a_dep19_gpkg.zip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-developer/attachments/20240730/c4dd8f9a/attachment-0001.htm>