[fdo-users] RE: Property2ColNameChar

Traian Stanev traian.stanev at autodesk.com
Wed Jun 11 14:15:50 EDT 2008 

It can be optimized if you assume certain order of access, but the cost will never be a true zero, and therefore it is a matter of performance, not just convenience. However, it would only really matter for the performance of providers which work with local data and are otherwise fast, i.e. cases where the conversion from string to index is a significant portion of time spent.

Traian


> -----Original Message-----
> From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> bounces at lists.osgeo.org] On Behalf Of Dan Stoica
> Sent: Wednesday, June 11, 2008 1:50 PM
> To: FDO Users Mail List
> Subject: RE: [fdo-users] RE: Property2ColNameChar
>
> I'm not arguing at all those two facts.
>
> What I'm advocating here is the fact that accessing by name has to be
> optimized anyways. And once this is done, having 'by index' is just a
> matter of convenience rather than performance. Do you agree?
>
> And even with an RFC (as Orest suggested) approved it will take a while
> to implement 'by index' for ALL providers, otherwise provider agnostic
> applications will break (a new capability needed?).
>
> Dan.
>
> -----Original Message-----
> From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> bounces at lists.osgeo.org] On Behalf Of Traian Stanev
> Sent: Wednesday, June 11, 2008 12:55 PM
> To: FDO Users Mail List
> Subject: RE: [fdo-users] RE: Property2ColNameChar
>
>
> The FDO feature reader API requires you to pass a string property name.
> Do you agree?
>
> Somewhere this string needs to be converted to an integer somehow, in
> order to access a column by index. Do you agree?
>
> I don't see how you can keep arguing those two simple facts.
>
>
> Traian
>
>
> > -----Original Message-----
> > From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> > bounces at lists.osgeo.org] On Behalf Of Dan Stoica
> > Sent: Wednesday, June 11, 2008 12:50 PM
> > To: FDO Users Mail List
> > Subject: RE: [fdo-users] RE: Property2ColNameChar
> >
> > " This happens every time you need to access the value of any column
> > via the FDO feature reader."
> >
> > No. The internal caching is based on the assumption that the
> properties
> > are fetched in the same order. So the mapping occurs only once, at
> the
> > 1st ReadNext().
> >
> > Dan.
> >
> > -----Original Message-----
> > From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> > bounces at lists.osgeo.org] On Behalf Of Traian Stanev
> > Sent: Wednesday, June 11, 2008 12:19 PM
> > To: FDO Users Mail List
> > Subject: RE: [fdo-users] RE: Property2ColNameChar
> >
> >
> > > And how would it help in the most common use case, "Select *" ?
> >
> > OK suppose you do a "select *".
> >
> > Then you get a feature reader back and you need to read property
> > values, for example for geometry. You call
> >
> >    Blah = reader->GetGeometry(L"GEOM");
> >
> > Internally the provider has to map the L"GEOM" to a column index.
> This
> > happens every time you need to access the value of any column via the
> > FDO feature reader.
> >
> >
> > Traian
> >
> >
> >
> > > -----Original Message-----
> > > From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> > > bounces at lists.osgeo.org] On Behalf Of Dan Stoica
> > > Sent: Wednesday, June 11, 2008 12:15 PM
> > > To: FDO Users Mail List
> > > Subject: RE: [fdo-users] RE: Property2ColNameChar
> > >
> > > Yes, but it also true that the providers already enhanced will be
> > > penalized without a real benefit because they have to implement
> "Get
> > by
> > > index" :-)
> > >
> > > And how would it help in the most common use case, "Select *" ?
> > >
> > > Dan.
> > >
> > > -----Original Message-----
> > > From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> > > bounces at lists.osgeo.org] On Behalf Of Traian Stanev
> > > Sent: Wednesday, June 11, 2008 11:53 AM
> > > To: FDO Users Mail List
> > > Subject: RE: [fdo-users] RE: Property2ColNameChar
> > >
> > >
> > > True, but this only applies to some providers. Other providers,
> which
> > > can deterministically map property name to index, are penalized
> > > unnecessarily.
> > >
> > > Traian
> > >
> > >
> > > > -----Original Message-----
> > > > From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> > > > bounces at lists.osgeo.org] On Behalf Of Dan Stoica
> > > > Sent: Wednesday, June 11, 2008 11:51 AM
> > > > To: FDO Users Mail List
> > > > Subject: RE: [fdo-users] RE: Property2ColNameChar
> > > >
> > > > > we are missing "stanadard" index access to properties and than
> > > > application can decide which one to use.
> > > >
> > > > Even so, the provider has to keep an internal mapping because the
> > > index
> > > > of the property doesn't necessary match the index of the
> > > corresponding
> > > > column (e.g. RevisionNumber column).
> > > >
> > > > Besides, doing everything under the hood has another advantage:
> it
> > > > saves the conversion from wchar to char (usually needed when
> going
> > > from
> > > > logical to physical) by caching the char value as well.
> > > >
> > > > Dan.
> > > >
> > > > -----Original Message-----
> > > > From: fdo-users-bounces at lists.osgeo.org [mailto:fdo-users-
> > > > bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
> > > > Sent: Wednesday, June 11, 2008 11:15 AM
> > > > To: FDO Users Mail List
> > > > Subject: [fdo-users] RE: Property2ColNameChar
> > > >
> > > > I have looked at the code of Property2ColNameChar and few
> questions
> > > pop
> > > > up.
> > > >
> > > > I believe when I was developing King.Oracle I was posting that we
> > > would
> > > > need to have index access to properties, simillar as database
> > client
> > > > APIs have. I am not sure but I would be suprised if other
> database
> > > > access software have implemented such mechanisems on data access
> > > level.
> > > >
> > > > To me , It looks like that we are missing "stanadard" index
> access
> > to
> > > > properties and than application can decide which one to use.
> > > >
> > > > Haris
> > > >
> > > > -----Original Message-----
> > > > From: fdo-users-bounces at lists.osgeo.org
> > > > [mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Dan
> Stoica
> > > > Sent: Wednesday, June 11, 2008 4:23 PM
> > > > To: FDO Users Mail List
> > > > Subject: RE: [fdo-users] PostGIS FDO Provider performance issues
> > > >
> > > > My bet is that "something else" is the look up for the property
> in
> > > the
> > > > logical/physical collections of properties.
> > > > E.g. for 20 properties in the class the provider will do aprox.
> > > > 2,000,000 wide string comparisons for 200,000 rows.
> > > > Actually it is 4 million because of IsNull() calls. This is the
> > > > performance killer.
> > > >
> > > > The problem has been solved for rdbms providers by caching the
> > > > properties in the order they are fetched (at the first fetch). So
> > the
> > > > property will be found either in the same position (after a
> IsNull)
> > > or
> > > > in the next slot. This is equivalent to fetching a property by
> > index
> > > > rather by name.
> > > >
> > > > The implementation is in GenericRdbms:
> > > > FdoRdbmsFeatureReader::Property2ColNameChar()
> > > >
> > > > Hope this helps,
> > > > Dan.
> > > >
> > > > -----Original Message-----
> > > > From: fdo-users-bounces at lists.osgeo.org
> > > > [mailto:fdo-users-bounces at lists.osgeo.org] On Behalf Of Zac
> Spitzer
> > > > Sent: Wednesday, June 11, 2008 3:03 AM
> > > > To: FDO Users Mail List
> > > > Subject: Re: [fdo-users] PostGIS FDO Provider performance issues
> > > >
> > > > I would agree it's too much overhead, as this happening with
> Oracle
> > > as
> > > > well ( anyone else seen
> > > > this happen with another db?) perhaps there is something else in
> > > play.
> > > >
> > > > How many rows are fetched by the client? Fetching one row at a
> time
> > > is
> > > > slower than returning blocks of 100 rows..
> > > >
> > > > Using mapguide, it's always the mapguide process (ie FDO) and not
> > the
> > > > database which is showing the high cpu
> > > >
> > > > z
> > > >
> > > >
> > > >
> > > > On Wed, Jun 11, 2008 at 4:56 PM, Carl Jokl
> <carl.jokl at keynetix.com>
> > > > wrote:
> > > > >
> > > > > It is true that there is extra overhead in performing the
> > > conversion
> > > > or going
> > > > > through FDO but I think it is fair to say that having PostGIS
> 16
> > to
> > > > 20
> > > > times
> > > > > slower on the same data is more of a performance hit than would
> > > have
> > > > been
> > > > > expected.
> > > > > --
> > > > > View this message in context:
> > > > http://www.nabble.com/PostGIS-FDO-Provider-performance-issues-
> > > > tp17756475
> > > > p17771210.html
> > > > > Sent from the FDO Users mailing list archive at Nabble.com.
> > > > >
> > > > > _______________________________________________
> > > > > fdo-users mailing list
> > > > > fdo-users at lists.osgeo.org
> > > > > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Zac Spitzer -
> > > > http://zacster.blogspot.com (My Blog)
> > > > +61 405 847 168
> > > > _______________________________________________
> > > > fdo-users mailing list
> > > > fdo-users at lists.osgeo.org
> > > > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > > > _______________________________________________
> > > > fdo-users mailing list
> > > > fdo-users at lists.osgeo.org
> > > > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > > > _______________________________________________
> > > > fdo-users mailing list
> > > > fdo-users at lists.osgeo.org
> > > > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > > > _______________________________________________
> > > > fdo-users mailing list
> > > > fdo-users at lists.osgeo.org
> > > > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > > _______________________________________________
> > > fdo-users mailing list
> > > fdo-users at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > > _______________________________________________
> > > fdo-users mailing list
> > > fdo-users at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > _______________________________________________
> > fdo-users mailing list
> > fdo-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/fdo-users
> > _______________________________________________
> > fdo-users mailing list
> > fdo-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/fdo-users
> _______________________________________________
> fdo-users mailing list
> fdo-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-users
> _______________________________________________
> fdo-users mailing list
> fdo-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/fdo-users

More information about the fdo-users mailing list