[Mapserver-dev] Optimizing for ArcSDE

[Paul Ramsey]

Frank Warmerdam wrote:

> 
>> We could pool the streams to save this cost.  Since we only ever
>> use one stream per connection, would it make sense to pool streams
>>  together with connections?  I was thinking we could create a new 
>> structure with both the sde->connection, and the stream, and we
>> could pass this to the connection pooling API.  The
>> msSDECloseConnection() callback function could be modified to close
>> the stream, then the connection.
>> 
>> Does anyone have any suggestions?  Would this approach work to save
>> us from the stream overhead?  Do we even need a solution this
>> complex, or is there something simple that could alleviate this
>> overhead?
>> 
>> Independent of streams, we are incurring about a 20ms cost in
>> getting layer info.  Would it make sense to cache the layer infos?
>> The layer info probably shouldn't be cached with the connections.
> 
> 
> Brock,
> 
> My take is that for SDE the "connection" we pool should include the
> stream and possibly the layer info.  To make this work we will need
> to do some fiddling to ensure the identifier (normally the CONNECTION
> string) is unique for different versions, streams and/for layers.

We cannot cache layerinfo together with stream/connection, because that 
would imply one cached connection per layer (lots!).  We will have to 
cache layerinfo separately (connectionstring + version + layername as key?)

We can cache stream/connection together, and the "key" should be the 
connection string + the version string.  That way we can retain Howard's 
version access ideas while still doing stream caching.

> Of course, I don't know much about SDE, so keep that in mind.

Yeah, reading the function API, this looks doable, but I don't know 
about the potential dangers of reusing streams over and over.  We'll 
have to read up more on the stream management functions.

> Best regards,