[Mapserver-dev] Optimizing for ArcSDE

[Brock Anderson]

Hello Developers,

I am trying to speed up the ArcSDE code.  Now that Mapserver can run in 
fast-cgi mode we are able to pool SDE connections, which greatly reduces 
response time for getMap requests.  We're still not as fast as ArcIMS 
(with the WMS connector) for similar getMap requests.  Some timed 
debugging statements indicate that there is still some slowdown in 
msSDELayerOpen() and msSDELayerClose(). 

Howard Butler made some improvements to this code yesterday that helped 
reduce the time cost for requesting default layer versions, but we are 
still incurring about 20ms delay in opening and again in freeing 
streams.  For each layer.  SE_stream_create() and SE_stream_free() 
appear to communicate with the SDE server.  Why else would there be any 
time cost to these calls?

We could pool the streams to save this cost.  Since we only ever use one 
stream per connection, would it make sense to pool streams together with 
connections?  I was thinking we could create a new structure with both 
the sde->connection, and the stream, and we could pass this to the 
connection pooling API.  The msSDECloseConnection() callback function 
could be modified to close the stream, then the connection.

Does anyone have any suggestions?  Would this approach work to save us 
from the stream overhead?  Do we even need a solution this complex, or 
is there something simple that could alleviate this overhead?

Independent of streams, we are incurring about a 20ms cost in getting 
layer info.  Would it make sense to cache the layer infos?  The layer 
info probably shouldn't be cached with the connections.

I would appreciate your comments.

Brock Anderson