[MapProxy] MapProxy: S3 performance

[Marc Monnerat]

Hello Giovanni,

How big are your source files ?

We have source files in a WMTS pyramid (tiles of 256x256) and reproject
them with mapproxy in other scheme (webmercator for instance). Mapproxy is
autoscaled, with Varnish trying to serve the tiles directly from the S3
bucket, if it does exist. Works pretty well.

Here is a small demo, in Webmercator (all tiles are reprojected on the fly
and stored in S3, or served directly by varnish if already existing)

http://codepen.io/procrastinatio/full/vmyGPG/


Regards,


Marc

2017-04-25 12:57 GMT+02:00 BERÉNYI Attila <aberenyi at gislab.hu>:

> Hi Giovanni - please have a look at this other thread
> http://lists.osgeo.org/pipermail/mapproxy/2017-March/002543.html.
>
> I'm still interested in this - unfortunately, I still couldn't carve out
> enough time to test this properly...
>
> Cheers
>
> giovanni.masarin at libero.it <giovanni.masarin at libero.it> ezt írta
> (időpont: 2017. ápr. 25., K, 12:09):
>
>> Hi, I'm testing the new s3 cache functionality.
>>
>> From the tests I am doing, I have verified that reading and writing
>> performance are considerably lower than the traditional file system in the
>> case of many simultaneous requests.
>>
>> I noticed that a http / s connection pool is not used but a connection to
>> s3 is instantiated for each request. Maybe this is a cause of slowing down.
>> I know that boto3 can use connection pool, but MapProxy implementation
>> seems not to use it. Can I force a connection pool?
>>
>> Are there configurations to improve s3 cache performance?
>>
>> Thank you
>> _______________________________________________
>> MapProxy mailing list
>> MapProxy at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapproxy
>
> --
> BERÉNYI Attila, PhD
> ügyvezető
>
> GISLab Consulting Kft.
> H-1031 Budapest, Kadosa utca 59.
>
> M: +36 20 457 1800 <+36%2020%20457%201800>
> W: http://gislab.hu
>
> _______________________________________________
> MapProxy mailing list
> MapProxy at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapproxy
>



-- 
-- ἡ φύσις οὐδὲν ποιεῖ ἅλματα.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapproxy/attachments/20170425/62042e46/attachment.html>