[Qgis-developer] Vector attribute types compatibility

[Radim Blazek]

I would like to ask how vector attribute types conversion between
providers is supposed to work. Currently:

QgsMemoryProvider:
* mNativeTypes: int(10), double(20,5), string(255)
* addAttributes() silently skips the field if type is not Int, Double
or String, does not test length and precision, always returns true.

QgsOgrProvider:
* mNativeTypes: int(10), double(20,15), string(255), date, datetime
(the types should be taken from OGR because depend on OGR driver, but
I don't see any function in OGR to get supported types)
* addAttributes(): if type is not supported, the field is not added,
error is set and returns false, supported fields are added however.
* createEmptyLayer() is using QgsVectorFileWriter which has its own
field types handling (different from QgsOgrProvider::addAttributes())
and calls directly OGR_L_CreateField()
* convertField(): doing the same as QgsVectorFileWriter (copy pasted
code) but not used at all

QgsPostgresProvider:
* mNativeTypes: int(0), longlong(0), double (20,20), string(255), date
(there are more types defined but these are those with max
lengh/precision per QVariant type)
* addAttributes(): is using field typeName() to add fields, but
typeName is always provider specific so it may become problem if
fields come from another provider. If single field fails, no fields
are added (transaction) and false is returned
* createEmptyLayer(): is using convertField() (that is why d'n'd does
not fail) on all fields and then addAttributes()
* convertField(): sets field typeName based on type

The problems I see:
* impossible to take simply fields from one provider and add them to
another one, for example:
    * ogr -> memory: numbers are often double(24,15) or date
    * ogr -> postgres: ogr is using String as typeName which fails as
type in postgres, postgres mNativeTypes are too short
* postgres provider addAttribute() takes typeName (which may be
whatever) as type for add column
* no method to convert not supported field to a supported one + get
message (for example, "string cut to 255 characters" or "date
converted to string"

I believe that, from user point of view, if data are being written to
a provider which does not supported all data types, then:
* user should be warned before the data are written, getting a list of
necessary type conversions and length/precision cuts
* if confirmed by user, all the data should be written converted to
supported types
but there is no support for that in API.

My suggestion:
* add
     virtual QgsField QgsVectorDataProvider::supportedField(field, ok, message)
  which would try to convert unsupported field to a supported field
and set message about conversion, it should almost never fail
* all methods adding fields (createEmptyLayer, QgsVectorFileWriter)
should use the same provider addAttributes()
* provider addAttributes() should first check if all fields are
supported using supportedField() and add nothing if at least one is
unsupported (if some fields are not added successive addFeature()
fails because of different number of attributes)
* add support for date and datetime to memory provider and set
benevolent length/precision limits (however it is not correct to set
size/precision > 15 for double, that is the only way to make possible
to copy data from OGR)

I am not sure how QgsPostgresProvide::addAttributes() should work,
because using typeName() may fail for fields from different provider
while using type() (i.e. convertField()) would disable the possibility
to specify explicitly postgres type by typeName. This is maybe
crucial. In general, what defines field type - "type" or "typeName"?
We could say that if typeName is unknown to postgres, it should use a
type based on "type" but the same typeName may have different meaning
for different drivers.

I am talking about 2.1.

Radim