[fdo-internals] RE: SDF feature reader

Wed Aug 13 17:15:54 EDT 2008

if i interpret code correctly there was allready attempt to solve issue
with  multiple readers with "Tag" in DbData and call to function
RefreshData before every Get.. in feature reader.

I would like to ask is why only one sqlite cursor is used for one sqlite
table for all readers. That looks like potential source of problem ?

Create new sqlite cursor for every reader ?

Haris

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
Karoui
Sent: Wednesday, August 13, 2008 10:47 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

There is probably a way to fix this issue and only make a copy if 2
readers are used. 

Badreddine

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian
Stanev
Sent: Wednesday, August 13, 2008 4:39 PM
To: 'FDO Internals Mail List'
Subject: RE: [fdo-internals] RE: SDF feature reader 

Yes, this is an intentional limitation, for the same reason
(performance).

Traian

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris
Kurtagic
Sent: Wednesday, August 13, 2008 4:53 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

Hm,  but problem is not thread related.

Having 2 feature readers for same SDF connection inside one thread will
create invalid data.

Haris

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Traian
Stanev
Sent: Wednesday, August 13, 2008 10:32 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

This behavior is really a tradeoff between performance and full thread
safety. SDF is near the performance end of the spectrum. I figure, if
someone desires full 

thread safety, they can use a real database.

Traian

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Gavin Cramer
Sent: Wednesday, August 13, 2008 4:27 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

GetGeometry (the one returning an FdoByte *) has documentation that says
the following:

"The array's memory area is only guaranteed to be valid until a call to
ReadNext() or Close(), or the disposal of this reader object."

So, the code is behaving according to the specification.  There should
be a similar statement for GetString, as well as any other accessor that
returns a pointer to non-reference-counted arrays, but I do not see one.
Anyway, an FDO client should expect the above behaviour from any FDO
provider, even if it actually does some internal copying.

Regardless, it looks like the behaviour that Haris sees is a defect.
Different readers should not be sharing an internal cursor.

Gavin

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
Karoui
Sent: Wednesday, August 13, 2008 4:14 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

The free threaded provider would use FdoThreadCapability_MultiThreaded
defined in
https://svn.osgeo.org/fdo/trunk/Fdo/Unmanaged/Inc/Fdo/Connections/Capabi
lities/ThreadCapability.h

The SDF provider is returning FdoThreadCapability_PerConnectionThreaded.

I had a quick look and I can see that the
SDFSimpleFeatureReader::GetGeometry(FdoString*, FdoInt32*) is returing
the pointer from the internal buffer. This may have been done to avoid
paying the penalty of copying the geometry. The second oveload,
GetGeometry(FdoString*) is returing a copy and should not have the same
issue. 

We may have to be extra carefull in case we decide to fix this issue to
make sure the reader performance is not impacted. 

Badreddine

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris
Kurtagic
Sent: Wednesday, August 13, 2008 3:12 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

it looks that feature readers are sharing same sqlite cursor with one
data buffer.   (BTW: Application is sending this readers to be used by
another app so it is not able to create new buffers)

Hm, you have lost me in this threading namings :)

Same SDF connection can't be shared across threads, right ? ( I thought
that as thread safe )

I would like to understand reasons for that behavior and if there are
rooms for improvements.

Sqlite connection can be shared between threads.

Thank you,

Haris

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
Karoui
Sent: Wednesday, August 13, 2008 8:36 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

It was not obvious that the readers were on the same thread. In that
case it's a defect. Make sure to copy the geometry before ReadNext is
called. It's possible the second reader(ReadNext) is invalidating the
geometry.

The SDF provider is thread safe but not free threaded, it's
per-connection threaded.

Badreddine

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris
Kurtagic
Sent: Wednesday, August 13, 2008 2:39 PM
To: FDO Internals Mail List
Subject: RE: [fdo-internals] RE: SDF feature reader 

I was saying  for  two feature readers inside same thread will result in
error.

Also, i would like to understand why SDF provider is not thread safe.

Haris

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Badreddine
Karoui
Sent: Wednesday, August 13, 2008 8:18 PM
To: FDO Internals Mail List
Subject: [fdo-internals] RE: SDF feature reader 

The SDF provider is per connection threaded which means only one thread
can access a reader at a time. You can have many threads using their own
connections/readers at the same time.

Badreddine

From: fdo-internals-bounces at lists.osgeo.org
[mailto:fdo-internals-bounces at lists.osgeo.org] On Behalf Of Haris
Kurtagic
Sent: Wednesday, August 13, 2008 2:29 PM
To: FDO Internals Mail List
Subject: [fdo-internals] SDF feature reader 

Hi,

I have issue when when more than one feature reader is created against
same SDF connection

It looks like same buffer ( created by sqlite cursor ) is shared betwean
feature readers.

This will cause that data returned from feature reader with function
GetGeometry will be corrupted when another reader is used.

Is this a bug or I am not looking into it correctly ?

I would like to be able to use multi thread access to sqlite database.

If I am right, SDF provider is not "thread safe" ? if so, what are main
reasons for  it ?

Thank you,

Haris

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/fdo-internals/attachments/20080813/d61ccb39/attachment-0001.html