[Qgis-developer] Vector attribute types compatibility

[Matthias Kuhn]

What circumstances do exist, where we are not be able to determine the 
native type based upon QgsField metainformation, but without using 
typeName?

I think we should focus on the development on an unambiguous type 
schema (QgsFields) and only use the typeName (with provider 
information?) as a very last resort.

We could then introduce a method QgsField::canConvertTo( QgsField& 
other ) and let this method return yes / yes with constraints (as you 
described, truncate, round etc) / no.
Right now, the QgsFields::operator== is abused for this operation, but 
it's not really accurate, like a number of bugs (FieldCalculator leaves 
you with a column full of NULL's after saving) have shown already. I 
already introduced this patch [1] to deal with the current state.
I'm not sure if in addition to precision and length another field 
number of bits or bytes should be introduced? (I'm not familiar enought 
with the providers to answer this question, but I remember having seen 
INT8, INT16 etc. datatypes)

Additionally, these fields (precision, length, bytes) should also be 
able to carry information like "not applicable" or "unlimited", so 
canConvertTo knows, if it needs to compare these fields also or compare 
them in a special way.
E.g. String types won't need "precision"
E.g. String types can have length unlimited

Regards,
Matthias

[1] 
https://github.com/qgis/Quantum-GIS/commit/0b780b0a5df8817415873ef2a55286566d53b293

On Son 07 Jul 2013 12:26:55 CEST, Marco Hugentobler wrote:
> Hi Radim
>
> Thanks for raising this important point.
>
>> but the same typeName may have different meaning
>> for different drivers
>
>
> How about passing a string that tells the origin provider string? The
> data provider could then recognise if it comes from the same provider
> and use the native types. If not, it should use the Qt type and
> information from QgsField (e.g. length, precision. Probably more in
> future? ).
>
> Regards,
> Marco
>
>
> Am 05.07.2013 14:42, schrieb Radim Blazek:
>> I would like to ask how vector attribute types conversion between
>> providers is supposed to work. Currently:
>>
>> QgsMemoryProvider:
>> * mNativeTypes: int(10), double(20,5), string(255)
>> * addAttributes() silently skips the field if type is not Int, Double
>> or String, does not test length and precision, always returns true.
>>
>> QgsOgrProvider:
>> * mNativeTypes: int(10), double(20,15), string(255), date, datetime
>> (the types should be taken from OGR because depend on OGR driver, but
>> I don't see any function in OGR to get supported types)
>> * addAttributes(): if type is not supported, the field is not added,
>> error is set and returns false, supported fields are added however.
>> * createEmptyLayer() is using QgsVectorFileWriter which has its own
>> field types handling (different from QgsOgrProvider::addAttributes())
>> and calls directly OGR_L_CreateField()
>> * convertField(): doing the same as QgsVectorFileWriter (copy pasted
>> code) but not used at all
>>
>> QgsPostgresProvider:
>> * mNativeTypes: int(0), longlong(0), double (20,20), string(255), date
>> (there are more types defined but these are those with max
>> lengh/precision per QVariant type)
>> * addAttributes(): is using field typeName() to add fields, but
>> typeName is always provider specific so it may become problem if
>> fields come from another provider. If single field fails, no fields
>> are added (transaction) and false is returned
>> * createEmptyLayer(): is using convertField() (that is why d'n'd does
>> not fail) on all fields and then addAttributes()
>> * convertField(): sets field typeName based on type
>>
>> The problems I see:
>> * impossible to take simply fields from one provider and add them to
>> another one, for example:
>>      * ogr -> memory: numbers are often double(24,15) or date
>>      * ogr -> postgres: ogr is using String as typeName which fails as
>> type in postgres, postgres mNativeTypes are too short
>> * postgres provider addAttribute() takes typeName (which may be
>> whatever) as type for add column
>> * no method to convert not supported field to a supported one + get
>> message (for example, "string cut to 255 characters" or "date
>> converted to string"
>>
>> I believe that, from user point of view, if data are being written to
>> a provider which does not supported all data types, then:
>> * user should be warned before the data are written, getting a list of
>> necessary type conversions and length/precision cuts
>> * if confirmed by user, all the data should be written converted to
>> supported types
>> but there is no support for that in API.
>>
>> My suggestion:
>> * add
>>       virtual QgsField QgsVectorDataProvider::supportedField(field,
>> ok, message)
>>    which would try to convert unsupported field to a supported field
>> and set message about conversion, it should almost never fail
>> * all methods adding fields (createEmptyLayer, QgsVectorFileWriter)
>> should use the same provider addAttributes()
>> * provider addAttributes() should first check if all fields are
>> supported using supportedField() and add nothing if at least one is
>> unsupported (if some fields are not added successive addFeature()
>> fails because of different number of attributes)
>> * add support for date and datetime to memory provider and set
>> benevolent length/precision limits (however it is not correct to set
>> size/precision > 15 for double, that is the only way to make possible
>> to copy data from OGR)
>>
>> I am not sure how QgsPostgresProvide::addAttributes() should work,
>> because using typeName() may fail for fields from different provider
>> while using type() (i.e. convertField()) would disable the possibility
>> to specify explicitly postgres type by typeName. This is maybe
>> crucial. In general, what defines field type - "type" or "typeName"?
>> We could say that if typeName is unknown to postgres, it should use a
>> type based on "type" but the same typeName may have different meaning
>> for different drivers.
>>
>> I am talking about 2.1.
>>
>> Radim
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer
>
>