[fdo-internals] New RFC 30 Posted

Mon Nov 24 16:15:21 EST 2008

Yes, a wrapper around either FdoFeatureReader or a PropertyValueCollection - same API for both. However, it would not necessarily copy the data from the FeatureReader - in the case where it wraps a FeatureReader it could simply do a passthrough, for example:

virtual FdoString* FdoFRFeature::GetString(FdoString* propName) { return m_reader->GetString(propName); }

Where m_reader is a private member variable pointing to the FeatureReader.

In the case where it wraps a PropertyValueCollection, it would be *roughly*:

virtual FdoString* FdoPVCFeature::GetString(FdoString* propName) { return ((FdoStringValue*)m_propValCol->FindItem(propName))->GetString(); }

The actual implementation is not the point though, as long as the outfacing API allows for implementing it efficiently. I don't think an API which uses PropertyValue objects has the maximum potential for efficiency because you would be incurring at least two virtual calls to refcounting functions on top of already expensive virtual calls to get the value.

Traian

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Monday, November 24, 2008 4:19 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

I guess there is lot of discus on that topic.
You are proposing an wrapper over feature reader which would expose data of reader ?
Than that wrapper would need to read rows (or one row ) from reader and copy data to new data buffer before it can be used in another command?
If it would fetch data to new data buffers than it is same performance problem of copying data. Same performance as to copy data to property values as we do already eventually if we want to do something else with that data.
Data type conversions can be implemented on existing reader too.

What I see as way to improve performance is to be able to set data buffers for select command so returned reader would used those buffers. So this same data without copying can be reused.

Haris

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: Monday, November 24, 2008 9:34 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

Think of it as a data row - a feature reader lets you iterate through many such data rows via ReadNext(). It is also very similar to OGRFeature for example (http://www.gdal.org/ogr/classOGRFeature.html).

So, for example say you want to transfer a feature from one connection to another - you would do a select from one connection, and that will return a FeatureReader. You can wrap that in the data row object, which has essentially the same API as the feature reader, except that it does not allow ReadNext(). Then you can pass that wrapper to an FdoInsert which will take the data row and insert it into the destination connection.

Internally you could also wrap a PropertyValue collection in such a data row object - no matter whether it is backed by a FeatureReader or a PropertyValue collection, it would offer the same API. Also, you can have the data row convert all its data into a PropertyValueCollection which you can use in case you don't need performance, thus enabling interop with existing code, which uses such PropertyValue collections.

You can also for example define this object so that it can do automatic type conversion for you - say you call GetInt32 on an Int16 property - it can automatically do the cast for you. Which would provide equivalent functionality to RFC 30 by the way, or it can use whatever code RFC30 adds to FDO. :)

Traian

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Monday, November 24, 2008 3:36 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

I am affraid I don't understand now. That equivalent of reader would be returned by command (than it is new reader) or it would be created from returned current Reader ?

Haris

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: Monday, November 24, 2008 9:01 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

Yes that's a problem and that's why I said "FdoFeatureReader (or equivalent)". What I meant by "equivalent" was something that has the almost the same API as FdoIFeatureReader, without the ReadNext/Close functions. Essentially it would represent a single row of data, with an API that allows fast access in a way that is the same as reading features coming from a query.

Traian

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Monday, November 24, 2008 3:21 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

Yes I understood that. I am afraid that it can be dangerous to use readers as input to another command.
One of thing would be that reader would not be closed before another command would be executed.
Not necessarily but I am worried that it could create problems with current readers/commands.

If we look further as you suggested to improve performance of readers, than it would be better to have them to returned shared pointers to data or to be able to set data buffers to be used by readers.
But even before that I would prefer to have index access to columns of readers as we speak of performance, we discussed it few times :)

Haris

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: Monday, November 24, 2008 8:41 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

Yes, but creating PropertyValues would be a performance overhead. That's why I'm suggesting the alternative approach of allowing commands to pull the feature's data directly from a FeatureReader (or equivalent).

Traian

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Monday, November 24, 2008 2:25 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

Right now from Reader we can't get PropertyValues only raw data.
To get Property value and property value collection which then could be passed to another command would be of great help and remove some of unnecessary  data conversions.

Haris

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: Monday, November 24, 2008 7:39 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

An alternative to this suggestion is to make the destination command accept an FdoIFeatureReader (or something equivalent that provides access to property values in a procedural way) as argument, rather than a collection of PropertyValues. I was thinking of this when I was implementing the SDF->SQLite converter, which requires two translations from PropertyValue to raw data - once when initializing the inputs of the insert command and a second time inside the destination provider when matching the input property values to the destination schema.

Traian

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Monday, November 24, 2008 8:27 AM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] New RFC 30 Posted

Hi Brent,
not directly related to this RFC but still it is important issue regarding FDO data values.

I find a problem working with FDO Readers and commands is that I need to get value as basic type from reader and then convert to FdoDataValue etc...
I would like to be able to use data values and properties with values from readers directly and make those as input value for another FDO command.
Right now I need to check data type returned from property get that data as native convert back to fdo data value and put as input to FDO command.

Haris

From: fdo-internals-bounces at lists.osgeo.org [mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Brent Robinson
Sent: Friday, November 21, 2008 7:32 PM
To: FDO Internals Mail List
Subject: [fdo-internals] New RFC 30 Posted

Hi,

RFC 30 (http://trac.osgeo.org/fdo/wiki/FDORfc30) is now ready for review. The main purpose is add FDO API functions to encapsulate conversions between different FDO data types. Please review and let me know if you have any questions or comments.

If approved, this one will likely be implemented 2 versions after FDO 3.3.

Thanks,
Brent.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo-internals/attachments/20081124/7d7db97d/attachment-0001.html