[mapguide-internals] Strange multithreading issue / slow dbxml access

[Bruce Dechant]

Andreas,

There was a recent submission to trunk that did reduce the Session repository memory consumption significantly.
I was running out of memory with 50 active, but 200 works for me now with the update here: https://trac.osgeo.org/mapguide/changeset/5779

3% load - something is definitely blocking :)

Thanks,
Bruce

-----Original Message-----
From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org] On Behalf Of Andreas Morf
Sent: Thursday, May 26, 2011 2:35 AM
To: 'MapGuide Internals Mail List'
Subject: RE: [mapguide-internals] Strange multithreading issue / slow dbxml access

Bruce
I'm already on latest trunk version...
Playing around with dbxml env parameters lead less problems:

- m_dbEnv.set_tread_count(10)

-> big reduction of DB_BUSY occurencies (still some)
-> no MgUnclassifiedException anymore (mysterious for me - maybe a db lock due to concurrent dbxml access in
layer.SetLayerDefinition an just 1 dbxml thread?)

- using in-memory dbxml for Session Repository
-> quite a speedup (but not as much as I expected)
-> memory consumption significant (as expected - 50 active sessions lead to around 1.5G for mgserver)

So there is still no ultimate solution for DB_BUSY error issues.

Performance remark:
Writing down 50 Layers to Sessionrepository (with 8 java threads) now takes 1.5 seconds (still long on a dual X5680) but processor
load never exceeds 3% - why ?
At map rendering more than 70% is achieved on 24 vcores - performance quite ok like that ;-)

Andreas

> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org]
> On Behalf Of Bruce Dechant
> Sent: Wednesday, May 25, 2011 6:45 PM
> To: MapGuide Internals Mail List
> Subject: RE: [mapguide-internals] Strange multithreading issue / slow dbxml access
> 
> Andreas,
> 
> Are you using the latest code from trunk?
> Please see these tickets as they contain a fix for the DB_BUSY issue.
> http://trac.osgeo.org/mapguide/ticket/1552
> http://trac.osgeo.org/mapguide/ticket/1687
> 
> Thanks,
> Bruce
> 
> -----Original Message-----
> From: mapguide-internals-bounces at lists.osgeo.org [mailto:mapguide-internals-bounces at lists.osgeo.org]
> On Behalf Of Andreas Morf
> Sent: Friday, May 20, 2011 3:44 AM
> To: mapguide-internals at lists.osgeo.org
> Subject: [mapguide-internals] Strange multithreading issue / slow dbxml access
> 
> Hi all,
> 
> While customizing my runtime map within java-webtier I am using following
> code seqence:
> ---
> resourceService.SetResource(newLayerResId, layerDefContent, null);
> layer.SetLayerDefinition(newLayerResId, resourceService);
> layer.ForceRefresh();
> ---
> 
> Using this code in a sequential manner there is no problem but it is quite
> slow because the map building process allways has to wait for SetResource
> (see below).
> Trying to parallelize and outsourcing execution of the code sequence to
> different threads (ThreadPoolExecutor) leads to following problem:
> 
> resourceService.SetResource(newLayerResId, layerDefContent, null); -->
> execution ok in different threads
> layer.SetLayerDefinition(newLayerResId, resourceService); --> allways throws
> MgUnclassifiedException
> 
> I was able to debug in Layerbase.cpp until the last line of
> MgLayerBase::SetLayerDefinition
> --
> m_layers->GetMap()->OnLayerDefinitionChanged(this);
> --
> which always throwed that Exception but i was not able to determine why...
> 
> 
> ..and again, with parallel resourceService.SetResource there are these
> annoying
> 
> > (8096) An exception occurred in DB XML component.
> > Please try your operation later as the repository was busy.
> 
> coming up again..
> 
> Besides, storing about 40 layer-xml to dbxml always takes around 3 seconds -
> my opinion: this is really slow  (on a quite fast XEON machine)
> Any hints for optimisation places?
> 
> So I'm stuck with tuning mgserver on a multi-processor machine again.
> 
> Andreas
> 
> 
> 
> 
> 
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals
> _______________________________________________
> mapguide-internals mailing list
> mapguide-internals at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapguide-internals


_______________________________________________
mapguide-internals mailing list
mapguide-internals at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/mapguide-internals