[mapguide-trac] #1779: Load MGP file failed with huge template maps

[MapGuide Open Source]

#1779: Load MGP file failed with huge template maps
--------------------------------+-------------------------------------------
   Reporter:  christinebao      |       Owner:  ChristineBao
       Type:  defect            |      Status:  new         
   Priority:  medium            |   Milestone:  2.3         
  Component:  Resource Service  |     Version:  2.1.0       
   Severity:  major             |    Keywords:              
External_id:  1397129.01        |  
--------------------------------+-------------------------------------------

Comment(by christinebao):

 == Tech assessment: ==

 '''Cause of the defect:'''[[BR]]
 There are 6502 resources in the package, that means it requires 6502 pairs
 of DeleteResource/SetResource to finish the upload package process, and it
 must be run in a transaction. However the DBEnvironment set max
 transaction number to 1000, and the default lock number is 1000, which can
 not support 6502 pairs of operations, thus cause the defect. [[BR]]
 To fix the defect, the number of transaction and lock should be increased.
 By testing the following number works:[[BR]]
 max lock: 20000[[BR]]
 Every coin has two sides. Increasing the number of transaction and lock
 will cause the pre-alloc memory for lock table to increase. It's the
 disadvantage of this fixing.[[BR]]

 Fixing:[[BR]]
 Increasing the number of transaction and locks.[[BR]]
 DbEnvironment.h[[BR]]
 static const u_int32_t MG_MAX_LOCKS = 20000;[[BR]]

 DbEnvironment.cpp[[BR]]
     m_dbEnv.set_tx_max(MG_MAX_TRANSACTIONS);[[BR]]
     m_dbEnv.set_lk_max_lockers(MG_MAX_LOCKS);[[BR]]
     m_dbEnv.set_lk_max_locks(MG_MAX_LOCKS);[[BR]]
     m_dbEnv.set_lk_max_objects(MG_MAX_LOCKS);[[BR]]



 Better fixing:[[BR]]
 20000 is too much for normal operation. It's better to move the setting to
 server.ini to make it configurable. So user can increase it (in this
 case), or decrease it when not need to. [[BR]]
 There are many configuration in DbEnvironment.h should move to server.ini.
 It's suggested to do them together.[[BR]]


 Reference document:[[BR]]
 a. A similiar issue:
 http://forums.oracle.com/forums/thread.jspa?threadID=1017325&tstart=150[[BR]]
     BDB developer replies that if "Error: DB_BUFFER_SMALL: User memory too
 small for return value" returns, it should be a bug in BDB. However 2.5.16
 (our current version) still have this issue in certain cases, and the BDB
 developer's suggestion is to increase cache size, locks etc.[[BR]]
 b. set_lk_max_lockers API: [[BR]]
 http://pybsddb.sourceforge.net/api_c/env_set_lk_max_lockers.html[[BR]]
     Quoted: Set the maximum number of simultaneous locking entities
 supported by the Berkeley DB lock subsystem. This value is used by
 DB_ENV->open to estimate how much space to allocate for various lock-table
 data structures. The default value is 1000 lockers.[[BR]]

 c. set_lk_max_locks API: [[BR]]
 http://pybsddb.sourceforge.net/api_c/env_set_lk_max_locks.html[[BR]]
   Quoted: Set the maximum number of locks supported by the Berkeley DB
 lock subsystem. This value is used by DB_ENV->open to estimate how much
 space to allocate for various lock-table data structures. The default
 value is 1000 locks.[[BR]]

 d. set_lk_max_objects API: [[BR]]
 http://pybsddb.sourceforge.net/api_c/env_set_lk_max_objects.html[[BR]]
    Quoted: Set the maximum number of simultaneously locked objects
 supported by the Berkeley DB lock subsystem. This value is used by
 DB_ENV->open to estimate how much space to allocate for various lock-table
 data structures.

-- 
Ticket URL: <http://trac.osgeo.org/mapguide/ticket/1779#comment:2>
MapGuide Open Source <http://mapguide.osgeo.org/>
MapGuide Open Source Internals