[mapguide-internals] RE: PATCH: Raster stability fixes, ticket#462

[Bruce Dechant]

Haris,

Please see my comments below.

Thanks,
Bruce

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Monday, March 30, 2009 11:59 AM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: PATCH: Raster stability fixes, ticket#462

I think we are talking about several different things which are and are
not related.

- missing refcount in FDO GDAL provider is even perhaps not a bug. I did
suggested that provider increase ref count,
 but also we could say that it would be acceptable to rely on client to
not call reader after closing connections.
What is most important is that only adding ref count and leaving
everything else as it is simply will not work. 
[Bruce Dechant] I'm not suggesting that we just fix the reference count and call it done. The FDO connection manager still needs to be fixed to ideally not use the reference count.

- this bug is not related to FDO GDAL provider alone, it was just a
place where it was most obvious to spot because of no conn. pool and
long duration of operations.

- real mistake is in MG which allows that objects which are retrieved
from connection are used even after MG would close that connection.
[Bruce Dechant] agreed this is a serious bug.

- If GDAL and FDO gdal provider are thread safe and in which size we
should look at this separately and preferably after we solve MG
problems. Mixing uncertainty of GDAL multi threading will make things
harder.

- neither adding ref count or thread lock in MG are solution (and I
didn't proposed any of those).

- I don't think we need any extra API, MG needs to keep count of
connection he opens and closes. It doesn't sound like a too big task.
What I think could be most time consuming is finding out if MG is really
closing connections it opens. 
[Bruce Dechant] I was making the FDO API as a suggestion only - I haven't looked at exactly what needs to be done to properly address this problem.

Haris



-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Bruce
Dechant
Sent: Monday, March 30, 2009 5:47 PM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: PATCH: Raster stability
fixes,ticket#462

Traian - Thanks for investigating this over the weekend, much
appreciated!

Anyways, for the 2.1 release of MapGuide I think we should incorporate
the reference count fix to the GDAL provider - this should be done
regardless as Traian has already pointed out the weak reference to the
FDO connection in the reader is bad. I don't think we should make
significant changes to the FDO connection manager this close to release.

Post 2.1 release the FDO connection manager should be more closely
looked at to see what can be done to more easily handle single threaded
providers and to perhaps get rid of the reference count dependency for
determining if a FDO connection is still in use. This may require an
additional FDO API so the provider could just tell the server if the
connection is in use. 

Thanks,
Bruce


-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Traian
Stanev
Sent: Monday, March 30, 2009 8:50 AM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] RE: PATCH: Raster stability fixes,
ticket#462


I was merely offering a lock-less alternative to the patch that was
posted to the ticket, which was just a big lock around raster
stylization. If there are other, cleaner, solutions we can certainly
discuss them once we know what they are. 

Before shutting up, let me point out that the big lock inside the
provider is not necessary, since the provider already advertises itself
as non-threadsafe in its ThreadCapability. If anything it's the caller's
responsibility to lock. So if you want to lock, do it from MapGuide.

The refcounting problem needs to be fixed, regardless of any connection
pooling fixes -- it's a bug that can be triggered by code other than the
connection pooling.


Traian


-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org
[mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Jason
Birch
Sent: Sunday, March 29, 2009 8:24 PM
To: MapGuide Internals Mail List
Subject: RE: [mapguide-internals] RE: PATCH: Raster stability fixes,
ticket#462

I think the work to test out GDAL as a multithreaded provider is
cool, and am personally really interested in trying it out myself
against some of my data sets to see how stable it is.

I have a couple concerns though.  First, given that GDAL is not
guaranteed to be thread safe, I'm worried that this may just
make the problems we are seeing harder to separate from inherent
multithreading problems in the GDAL provider. Second, I worry
that if this is implemented, it will be seen as a solution,
rather than what it really is: a way of masking the underlying
problem in the MapGuide connection manager, similar to the
temporary "big lock" workaround.

Given that Haris has already spent a lot of days on the analysis
of this problem, I think it would be a good idea to determine
the best solution and implement it rather than putting something
that looks like it works (like the previous fix of using a single
pooled connection) in place.

I know that Haris has been experimenting with at least a couple
alternative permanent solutions to this problem, and will be
interested to hear some discussion around them.

Jason _______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals
_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals