[Mapguide-trac] [mapguide-trac] #2292: Bad memory allocation in MapGuide 2.4 on centos

[MapGuide Open Source]

#2292: Bad memory allocation in MapGuide 2.4 on centos
--------------------------+-------------------------------------------------
   Reporter:  gabrimonfa  |       Owner:       
       Type:  defect      |      Status:  new  
   Priority:  low         |   Milestone:       
  Component:  General     |     Version:  2.4.0
   Severity:  major       |    Keywords:       
External_id:              |  
--------------------------+-------------------------------------------------
 I'm experiencing some sporadic problems with MapGuide 2.4 on centos.
 When the problem happens (sporadically) MapGuide stops to serve completely
 until manually restarted.

 This issue happens around three or four times a week in a couple of
 MapGuide with moderate load.

 Log does not show always the same error.
 This is one of the most common log entry

 {{{
 <2013-04-16T18:16:04>   2115980144      Ajax Viewer     xx.xx.xx.xx
 Anonymous
  Error: Out of memory.
         std::bad_alloc
  StackTrace:
   - MgRenderingServiceHandler.ProcessOperation() line 83 file
 RenderingServiceHandler.cpp
   - MgOpRenderDynamicOverlay.Execute() line 162 file
 OpRenderDynamicOverlay.cpp
   - MgServerRenderingService.RenderDynamicOverlay() line 407 file
 ServerRenderingService.cpp
 }}}

 often followed by

 {{{
 <2013-04-16T18:16:18>   1822264176                      Anonymous
  Error: An exception occurred in DB XML component.
         Error: DB_RUNRECOVERY: Fatal error, run database recovery
  StackTrace:
   - MgSiteServiceHandler.ProcessOperation() line 83 file
 SiteServiceHandler.cpp
   - MgOpCreateSession.Execute() line 109 file OpCreateSession.cpp
   - MgServerSiteService.CreateSession() line 762 file
 ServerSiteService.cpp
   - MgServerResourceService.CreateRepository() line 541 file
 ServerResourceService.cpp
   - MgSessionRepository.MgSessionRepository() line 76 file
 SessionRepository.cpp
 }}}


 Sometimes the log is the following:

 {{{
 <2013-04-18T11:33:26>   2063526768      Ajax Viewer     xx.xx.xx.xx
 Anonymous
  Error: An exception occurred in DB XML component.
         Error: Cannot allocate memory
  StackTrace:
   - MgRenderingServiceHandler.ProcessOperation() line 83 file
 RenderingServiceHandler.cpp
   - MgOpQueryFeatures.Execute() line 125 file OpQueryFeatures.cpp
   - MgServerRenderingService.QueryFeatures() line 848 file
 ServerRenderingService.cpp
   - MgServerRenderingService.RenderForSelection() line 1520 file
 ServerRenderingService.cpp
   -
 MgServerResourceService.GetResourceContent(Id=Library://vhproduction/xxxx/00_layer/xxxx/xxxxxxx_readonly.LayerDefinition
 Tags=) line 1222 file ServerResourceService.cpp
   - MgApplicationRepositoryManager.GetResourceContent() line 385 file
 ApplicationRepositoryManager.cpp
   - MgResourceDefinitionManager.GetResource
 (MgLibraryResourceContents.dbxml)() line 320 file
 ResourceDefinitionManager.cpp
   - MgResourceContentManager.GetDocument() line 590 file
 ResourceContentManager.cpp
   - MgResourceDefinitionManager.GetDocument
 (MgLibraryResourceContents.dbxml)() line 476 file
 ResourceDefinitionManager.cpp
 }}}

 Performance log shows a curious fact when the problem happens:
 TOTALPROCESSEDOP remained one operation behind TOTALRECEIVEDOP, as if one
 operation was stuck.

 The server is a virtual machine with plenty of RAM.
 When the issue happens there is no memory pressure at all, usually
 MapGuide is using under 25% of available RAM.
 I've also modified kernel.shmmax to 1GB, without success.

 I've increased in serverconfig.ini
   * DBMaxTransactions from 1000 to 2000
   * DBMaxLockers from 1000 to 2000

 This appears to make the problem a little bit less likely (but I cannot be
 sure) but does not solve it.

 This is the complete DB ENVIRONMENT stanza of serverconfig.ini

 {{{
 LibraryCacheSize                   = 256
 SessionCacheSize                   = 8
 DBPageSize                         = 32
 DBXMLPageSize                      = 32
 LibraryLogBufferSize               = 12
 SessionLogBufferSize               = 1
 #DBMaxTransactions                  = 1000
 DBMaxTransactions                  = 2000
 SessionDBPageSize                  = 2
 SessionDBXMLPageSize               = 0.5
 DBTimeout                          = 0.2
 #DBMaxLockers                       = 1000
 DBMaxLockers                       = 2000
 }}}

 Restarting MapGuide always solves the problem but I needed to make a
 script that detects this kind of problem and signal me to restart the
 service.
 Obviously this is far from acceptable.
 I've not tried with MapGuide 2.5.0

 Any workaround would be appreciated

-- 
Ticket URL: <https://trac.osgeo.org/mapguide/ticket/2292>
MapGuide Open Source <http://mapguide.osgeo.org/>
MapGuide Open Source Internals