[MapProxy] MapProxy: S3 performance

[Oliver Tonnhofer]

Hi,

> On 25.04.2017, at 12:01, giovanni.masarin at libero.it wrote:
> From the tests I am doing, I have verified that reading and writing performance are considerably lower than the traditional file system in the case of many simultaneous requests.

Well, that's expected. Reading a local file should always be a lot faster than reading on object from S3.


> I noticed that a http / s connection pool is not used but a connection to s3 is instantiated for each request. Maybe this is a cause of slowing down. I know that boto3 can use connection pool, but MapProxy implementation seems not to use it. Can I force a connection pool? 

I think this is handled by boto3 sessions, but I might be wrong. If you know how to enable the pooling: patches are welcomed! 


> Are there configurations to improve s3 cache performance? 


MapProxy's S3 cache is not meant to be a replacement for the regular file cache. It's a building block for more complex architectures. For example, you can use it to store tiles on S3 and let the client request tiles directly from there or via a caching proxy.


Regards,
Oliver

-- 
Oliver Tonnhofer  | Omniscale GmbH & Co KG  | https://omniscale.com
OpenStreetMap WMS and tile services         | https://maps.omniscale.com