[mapguide][MG252][Modified] Server hung/crashed after running load tests

[Walt Welton-Lair]

You can view the artifact detail at the following URL:

    https://mapguide.osgeo.org/servlets/Scarab/id/MG252

Type:
Defect

Artifact ID:
MG252 (Server hung/crashed after running load tests)

Modified by:
Walt Welton-Lair
waltweltonlair (walt.welton-lair at autodesk.com)

The following modifications were made to this artifact:
---------------------------------------------------------------------

-- Developer Notes set to new value:
Old value:
[WCW] I have a fixed a threading problem under Linux.  The problem is in OperationThread.cpp.
1/ we grab a message block from the queue (around line 58)
2/ we then get the MgServerStreamData (line 78) and MgClientHandler (line 95), with the latter being addref'ed and stored in a smart pointer.
3/ once all the data from the stream has been processed (CheckStream returns mssDone, lines 127 & 160) we do the following:
  - set the MgClientHandler status to queued
  - create a new MgServerStreamData from the old
    => the referenced MgClientHandler is *transferred* from the old MgServerStreamData to the new
  - create a new message block containing the new MgServerStreamData
  - push the new message block back on the stack
4/ this is where things get interesting
  - the original thread continues, and the smart pointer for the MgClientHandler eventually goes out of scope and calls Release.
  - at the same time the newly added message block gets removed the queue by the next available OperationThread and starts to get processed
    => this involves accessing the MgClientHandler
  - both threads are now simultaneously accessing the client handler (specifically the ref count), and under Linux I'm seeing the ref count go bad, which sometimes leads to multiple attempts to delete the object
  - surprisingly this shouldn't fail since MgClientHandler derives from MgGuardDisposable, nevertheless it does
5/ the fix is to simply ensure the MgClientHandler smart pointer is cleared *before* adding the new message block to the queue
New value:
[WCW] I have a fixed a threading problem under Linux.  The problem is in OperationThread.cpp.
1/ we grab a message block from the queue (around line 58)
2/ we then get the MgServerStreamData (line 78) and MgClientHandler (line 95), with the latter being addref'ed and stored in a smart pointer.
3/ once all the data from the stream has been processed (CheckStream returns mssDone, lines 127 & 160) we do the following:
  - set the MgClientHandler status to queued
  - create a new MgServerStreamData from the old
    => the referenced MgClientHandler is *transferred* from the old MgServerStreamData to the new
  - create a new message block containing the new MgServerStreamData
  - push the new message block back on the stack
4/ this is where things get interesting
  - the original thread continues, and the smart pointer for the MgClientHandler eventually goes out of scope and calls Release.
  - at the same time the newly added message block gets removed the queue by the next available OperationThread and starts to get processed
    => this involves accessing the MgClientHandler
  - both threads are now simultaneously accessing the client handler (specifically the ref count), and under Linux I'm seeing the ref count go bad, which sometimes leads to multiple attempts to delete the object
  - surprisingly this shouldn't fail since MgClientHandler derives from MgGuardDisposable, nevertheless it does
5/ the fix is to simply ensure the MgClientHandler smart pointer is cleared *before* adding the new message block to the queue

[WCW] Also updated Operationthread.cpp to release message blocks immediately after they have been processed.




---------------------------------------------------------------------
This message was automatically generated by Project Tracker.