[MapProxy] MapProxy: S3 performance
olt at omniscale.de
Tue Apr 25 10:12:08 PDT 2017
> On 25.04.2017, at 12:01, giovanni.masarin at libero.it wrote:
> From the tests I am doing, I have verified that reading and writing performance are considerably lower than the traditional file system in the case of many simultaneous requests.
Well, that's expected. Reading a local file should always be a lot faster than reading on object from S3.
> I noticed that a http / s connection pool is not used but a connection to s3 is instantiated for each request. Maybe this is a cause of slowing down. I know that boto3 can use connection pool, but MapProxy implementation seems not to use it. Can I force a connection pool?
I think this is handled by boto3 sessions, but I might be wrong. If you know how to enable the pooling: patches are welcomed!
> Are there configurations to improve s3 cache performance?
MapProxy's S3 cache is not meant to be a replacement for the regular file cache. It's a building block for more complex architectures. For example, you can use it to store tiles on S3 and let the client request tiles directly from there or via a caching proxy.
Oliver Tonnhofer | Omniscale GmbH & Co KG | https://omniscale.com
OpenStreetMap WMS and tile services | https://maps.omniscale.com
More information about the MapProxy