[Qgis-developer] Vector attribute types compatibility

[Radim Blazek]

On Sun, Jul 7, 2013 at 12:26 PM, Marco Hugentobler
<marco.hugentobler at sourcepole.ch> wrote:
> Hi Radim
>
> Thanks for raising this important point.
>
>> but the same typeName may have different meaning
>> for different drivers
>
> How about passing a string that tells the origin provider string? The data
> provider could then recognise if it comes from the same provider and use the
> native types.

It sounds too hacky. Provider key should not be used above providers
abstraction level, I believe.

> If not, it should use the Qt type and information from
> QgsField (e.g. length, precision. Probably more in future? ).

So the same what Matthias is suggesting, right? It sounds better.

Radim

> Regards,
> Marco
>
>
> Am 05.07.2013 14:42, schrieb Radim Blazek:
>>
>> I would like to ask how vector attribute types conversion between
>> providers is supposed to work. Currently:
>>
>> QgsMemoryProvider:
>> * mNativeTypes: int(10), double(20,5), string(255)
>> * addAttributes() silently skips the field if type is not Int, Double
>> or String, does not test length and precision, always returns true.
>>
>> QgsOgrProvider:
>> * mNativeTypes: int(10), double(20,15), string(255), date, datetime
>> (the types should be taken from OGR because depend on OGR driver, but
>> I don't see any function in OGR to get supported types)
>> * addAttributes(): if type is not supported, the field is not added,
>> error is set and returns false, supported fields are added however.
>> * createEmptyLayer() is using QgsVectorFileWriter which has its own
>> field types handling (different from QgsOgrProvider::addAttributes())
>> and calls directly OGR_L_CreateField()
>> * convertField(): doing the same as QgsVectorFileWriter (copy pasted
>> code) but not used at all
>>
>> QgsPostgresProvider:
>> * mNativeTypes: int(0), longlong(0), double (20,20), string(255), date
>> (there are more types defined but these are those with max
>> lengh/precision per QVariant type)
>> * addAttributes(): is using field typeName() to add fields, but
>> typeName is always provider specific so it may become problem if
>> fields come from another provider. If single field fails, no fields
>> are added (transaction) and false is returned
>> * createEmptyLayer(): is using convertField() (that is why d'n'd does
>> not fail) on all fields and then addAttributes()
>> * convertField(): sets field typeName based on type
>>
>> The problems I see:
>> * impossible to take simply fields from one provider and add them to
>> another one, for example:
>>      * ogr -> memory: numbers are often double(24,15) or date
>>      * ogr -> postgres: ogr is using String as typeName which fails as
>> type in postgres, postgres mNativeTypes are too short
>> * postgres provider addAttribute() takes typeName (which may be
>> whatever) as type for add column
>> * no method to convert not supported field to a supported one + get
>> message (for example, "string cut to 255 characters" or "date
>> converted to string"
>>
>> I believe that, from user point of view, if data are being written to
>> a provider which does not supported all data types, then:
>> * user should be warned before the data are written, getting a list of
>> necessary type conversions and length/precision cuts
>> * if confirmed by user, all the data should be written converted to
>> supported types
>> but there is no support for that in API.
>>
>> My suggestion:
>> * add
>>       virtual QgsField QgsVectorDataProvider::supportedField(field, ok,
>> message)
>>    which would try to convert unsupported field to a supported field
>> and set message about conversion, it should almost never fail
>> * all methods adding fields (createEmptyLayer, QgsVectorFileWriter)
>> should use the same provider addAttributes()
>> * provider addAttributes() should first check if all fields are
>> supported using supportedField() and add nothing if at least one is
>> unsupported (if some fields are not added successive addFeature()
>> fails because of different number of attributes)
>> * add support for date and datetime to memory provider and set
>> benevolent length/precision limits (however it is not correct to set
>> size/precision > 15 for double, that is the only way to make possible
>> to copy data from OGR)
>>
>> I am not sure how QgsPostgresProvide::addAttributes() should work,
>> because using typeName() may fail for fields from different provider
>> while using type() (i.e. convertField()) would disable the possibility
>> to specify explicitly postgres type by typeName. This is maybe
>> crucial. In general, what defines field type - "type" or "typeName"?
>> We could say that if typeName is unknown to postgres, it should use a
>> type based on "type" but the same typeName may have different meaning
>> for different drivers.
>>
>> I am talking about 2.1.
>>
>> Radim
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>
>
> --
> Dr. Marco Hugentobler
> Sourcepole -  Linux & Open Source Solutions
> Weberstrasse 5, CH-8004 Zürich, Switzerland
> marco.hugentobler at sourcepole.ch http://www.sourcepole.ch
> Technical Advisor QGIS Project Steering Committee
>
>
> _______________________________________________
> Qgis-developer mailing list
> Qgis-developer at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/qgis-developer