[Qgis-developer] Vector attribute types compatibility

[Marco Hugentobler]

Hi Radim

Thanks for raising this important point.

>but the same typeName may have different meaning
>for different drivers


How about passing a string that tells the origin provider string? The 
data provider could then recognise if it comes from the same provider 
and use the native types. If not, it should use the Qt type and 
information from QgsField (e.g. length, precision. Probably more in 
future? ).

Regards,
Marco


Am 05.07.2013 14:42, schrieb Radim Blazek:
> I would like to ask how vector attribute types conversion between
> providers is supposed to work. Currently:
>
> QgsMemoryProvider:
> * mNativeTypes: int(10), double(20,5), string(255)
> * addAttributes() silently skips the field if type is not Int, Double
> or String, does not test length and precision, always returns true.
>
> QgsOgrProvider:
> * mNativeTypes: int(10), double(20,15), string(255), date, datetime
> (the types should be taken from OGR because depend on OGR driver, but
> I don't see any function in OGR to get supported types)
> * addAttributes(): if type is not supported, the field is not added,
> error is set and returns false, supported fields are added however.
> * createEmptyLayer() is using QgsVectorFileWriter which has its own
> field types handling (different from QgsOgrProvider::addAttributes())
> and calls directly OGR_L_CreateField()
> * convertField(): doing the same as QgsVectorFileWriter (copy pasted
> code) but not used at all
>
> QgsPostgresProvider:
> * mNativeTypes: int(0), longlong(0), double (20,20), string(255), date
> (there are more types defined but these are those with max
> lengh/precision per QVariant type)
> * addAttributes(): is using field typeName() to add fields, but
> typeName is always provider specific so it may become problem if
> fields come from another provider. If single field fails, no fields
> are added (transaction) and false is returned
> * createEmptyLayer(): is using convertField() (that is why d'n'd does
> not fail) on all fields and then addAttributes()
> * convertField(): sets field typeName based on type
>
> The problems I see:
> * impossible to take simply fields from one provider and add them to
> another one, for example:
>      * ogr -> memory: numbers are often double(24,15) or date
>      * ogr -> postgres: ogr is using String as typeName which fails as
> type in postgres, postgres mNativeTypes are too short
> * postgres provider addAttribute() takes typeName (which may be
> whatever) as type for add column
> * no method to convert not supported field to a supported one + get
> message (for example, "string cut to 255 characters" or "date
> converted to string"
>
> I believe that, from user point of view, if data are being written to
> a provider which does not supported all data types, then:
> * user should be warned before the data are written, getting a list of
> necessary type conversions and length/precision cuts
> * if confirmed by user, all the data should be written converted to
> supported types
> but there is no support for that in API.
>
> My suggestion:
> * add
>       virtual QgsField QgsVectorDataProvider::supportedField(field, ok, message)
>    which would try to convert unsupported field to a supported field
> and set message about conversion, it should almost never fail
> * all methods adding fields (createEmptyLayer, QgsVectorFileWriter)
> should use the same provider addAttributes()
> * provider addAttributes() should first check if all fields are
> supported using supportedField() and add nothing if at least one is
> unsupported (if some fields are not added successive addFeature()
> fails because of different number of attributes)
> * add support for date and datetime to memory provider and set
> benevolent length/precision limits (however it is not correct to set
> size/precision > 15 for double, that is the only way to make possible
> to copy data from OGR)
>
> I am not sure how QgsPostgresProvide::addAttributes() should work,
> because using typeName() may fail for fields from different provider
> while using type() (i.e. convertField()) would disable the possibility
> to specify explicitly postgres type by typeName. This is maybe
> crucial. In general, what defines field type - "type" or "typeName"?
> We could say that if typeName is unknown to postgres, it should use a
> type based on "type" but the same typeName may have different meaning
> for different drivers.
>
> I am talking about 2.1.
>
> Radim
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer


-- 
Dr. Marco Hugentobler
Sourcepole -  Linux & Open Source Solutions
Weberstrasse 5, CH-8004 Zürich, Switzerland
marco.hugentobler at sourcepole.ch http://www.sourcepole.ch
Technical Advisor QGIS Project Steering Committee