[mapguide-users] RE: Changing custom PoolSize for GDAL has no effect

Sun May 29 13:08:45 EDT 2011

Hi Haris,

One addition that I remembered while reading what I wrote in the ticket -- I also removed another global raster mutex on the MapGuide side, which was no longer necessary after the fix. The mutex was *probably* working around the refcounting problem, but I don't know if that mutex is still there -- it's been two years since I last looked at that code.

Traian

-----Original Message-----
From: mapguide-users-bounces at lists.osgeo.org [mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Traian Stanev
Sent: Sunday, May 29, 2011 12:47 PM
To: MapGuide Users Mail List
Subject: RE: [mapguide-users] RE: Changing custom PoolSize for GDAL has no effect

Hi Haris,

I fixed the refcount and removed the global mutex on the GDAL side (there is still one on MapGuide connection pool side). 

The changes I had made to MapGuide were in order to allow opening of multiple GDAL connections at the same time. However, they are not strictly necessary or related to the other fix -- one can choose how to use GDAL by setting the pool size in the config file. My change to the provider simply allowed to set the pool size to more than 1 for GDAL on the MapGuide side, but did not require it.

The schema thing was an independent optimization which is necessary if one increases the number of pooled connections and allows MapGuide to open and close them like for other providers.

Traian

-----Original Message-----
From: mapguide-users-bounces at lists.osgeo.org [mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Haris Kurtagic
Sent: Sunday, May 29, 2011 5:25 AM
To: MapGuide Users Mail List
Subject: Re: [mapguide-users] RE: Changing custom PoolSize for GDAL has no effect

Hi Traian

We are trying to use MG OS 2.2 with GDAL and it is not stable.

Right now at customer sites we are using 2.1 with GDAL with patches we made some time ago. I remember we had lot of discussion around gdal/mg but can't remember whole story.

I remember the issue was with concurrent access to gdal provider ( and also others ).

If I am right at the end, there were some bigger changes made in MG connection manager to solve those issues ?
In two patches you submitted, there are basically two things : added ref count to connection and creating copy of the schema ?
The refcount change is related or not to changes made in MG connection manager?

Thanks,
Haris

On Sat, May 28, 2011 at 6:32 PM, Traian Stanev <traian.stanev at autodesk.com> wrote:
>
>
> Hi Zack,
>
>
>
> The MapGuide PSC was against this patch back then. It is up to them to 
> decide if they want it now. It's been too long and I no longer have a 
> MapGuide environment set up to test it with the latest code, so it 
> would be up to someone who needs this to check it in and test it, or I 
> can merge it in without testing.
>
>
>
> Traian
>
>
>
>
>
> From: mapguide-users-bounces at lists.osgeo.org
> [mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Zac 
> Spitzer
> Sent: Saturday, May 28, 2011 2:38 AM
>
> To: MapGuide Users Mail List
> Subject: Re: [mapguide-users] RE: Changing custom PoolSize for GDAL 
> has no effect
>
>
>
> That patch sounds pretty important, can we commit it?
>
> On Sun, Apr 17, 2011 at 3:18 AM, Traian Stanev 
> <traian.stanev at autodesk.com>
> wrote:
>
> Not sure if that would help, but a patch I did a while back to fix 
> refcounting issues with the GDAL provider also included an 
> optimization about config file parsing which sped up opening of new 
> GDAL connections by a factor of 10 at least.
>
> The patch is attached to this ticket:
> http://trac.osgeo.org/mapguide/ticket/462 called 
> gdal_nomutex_cachedschema.patch.
>
> It was a long time ago, but as far as I remember this optimization was 
> key in case one wants to enable pooling of GDAL connections. It also 
> removed the internal global mutex in the provider which made it work 
> single-threaded regardless of the setting in the MapGuide 
> configuration. That patch requires my addref patch attached to the same ticket.
>
> Traian
>
> -----Original Message-----
> From: mapguide-users-bounces at lists.osgeo.org
> [mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Trevor 
> Wekel
> Sent: Friday, April 15, 2011 12:26 AM
> To: MapGuide Users Mail List
> Subject: RE: [mapguide-users] RE: Changing custom PoolSize for GDAL 
> has no effect
>
> Hi Gunter,
>
> Since only one raster feature source can be loaded at any given time, 
> you may up with some additional load overhead if your config.xml 
> (raster
> catalog) documents are fairly large.  As far as I know, the config.xml 
> information will only remain cached for the lifetime of the connection 
> to the raster feature source.  With 35 raster feature sources, it may 
> be getting parsed for every raster request.
>
> Regards,
> Trevor
>
>
> -----Original Message-----
> From: mapguide-users-bounces at lists.osgeo.org
> [mailto:mapguide-users-bounces at lists.osgeo.org] On Behalf Of Gunter 
> Becker
> Sent: April 13, 2011 2:05 PM
> To: mapguide-users at lists.osgeo.org
> Subject: [mapguide-users] RE: Changing custom PoolSize for GDAL has no 
> effect
>
> Thanks Trevor,
>
> can I suppose that my configuration (35 raster featuresources, 50 
> concurrent
> user) could work with
> the single-threaded GDAL Provider or does the provider means a 
> bottleneck for my application?
>
> The repository error: I will try final release as soon it is available 
> and if the error exists I will post it here. I supposed it has to do 
> something with the rasterprovider since I've not found any hint in 
> logfiles on what's going on. Even with full error report enabled.
>
> Funding the work on the GDAL Provider: I have to discuss this with my 
> company (personally I'd like to fund but it is not my decision)
>
> Thanks, Gunter
>
> --
> View this message in context:
> http://osgeo-org.1803224.n2.nabble.com/Changing-custom-PoolSize-for-GD
> AL-has-no-effect-tp6268276p6270448.html
> Sent from the MapGuide Users mailing list archive at Nabble.com.
> _______________________________________________
> mapguide-users mailing list
> mapguide-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-users
>
> _______________________________________________
> mapguide-users mailing list
> mapguide-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-users
>
>
> --
> Zac Spitzer
> Solution Architect / Director
> Ennoble Consultancy Australia
> http://www.ennoble.com.au
> http://zacster.blogspot.com
> +61 405 847 168
>
>
>
> _______________________________________________
> mapguide-users mailing list
> mapguide-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-users
>
>
_______________________________________________
mapguide-users mailing list
mapguide-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-users
_______________________________________________
mapguide-users mailing list
mapguide-users at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-users