[MapProxy] Performance tuning
shr3k at typo3-hosting.com
Tue Nov 19 04:58:04 PST 2013
What concerns first case with real machine, was that seeding to mbtiles too? I experienced same problems with mbtiles locking with multiple seeding processes in case of seeding from much slower remote WMSes than any local renderer and definitely given up - now I'm using TileCache directory structure.
From: Oliver Tonnhofer [mailto:olt at omniscale.de]
To: BERÉNYI Attila [mailto:aberenyi at gislab.hu]
Cc: mapproxy at lists.osgeo.org
Sent: Mon, 18 Nov 2013 17:57:06 +0100
Subject: Re: [MapProxy] Performance tuning
On 18.11.2013, at 16:54, BERÉNYI Attila wrote:
> I have googled quite a lot to find the information I am looking for - no luck so far. I am well aware this is the hundredth time that someone ask about performance tuning - please bear with me.
> First things first, MapProxy is awesome, well done guys!
> So, I have two separate test cases:
> • "Real" machine
> • Xeon 5355, 24GB RAM, PostgreSQL 9.3.1-3, PostGIS 2.1.0-2 running on Arch.
> • Tile generation chain: PostGIS -> Geoserver -> MapProxy.
> • Geographic extent: [-10, 46, 4, 65] (EPSG:4326)
> • Zoom levels: 19.
> • I have tried several -c (up to 160) parameters (i.e. number of parallel seed processes) but I have never reached the optimal 8-ish re the 15 minutes average load, even though I have adjusted the parameters in Geoserver and PostgreSQL as well.
The source services are the bottleneck in most cases. Have you checked your IO load(e.g. %wa in top)? How many tiles/seconds do you seed?
> • My tests show that even 2 parallel processes per vCPU (i.e. -c 32) cause issue/bottleneck (http://pastebin.com/raw.php?i=29MPZcYp). However the 15 minutes average load is only ~8.
Again, how many tiles/seconds do you seed? SQLite/MBTiles does not work well with heavy read concurrency, but this hasn't been a problem in my seeding processes. Maybe it's necessary to increase the timeout in SQLite during seeding (not configurable ATM).
> And finally a question/suggestion: have you ever considered to utilise the GPU's capabilities within the worker processes (http://gis.stackexchange.com/questions/77181/generate-tiles-using-gpu)?
You should ask on the Mapnik/Geoserver list.
Oliver Tonnhofer | Omniscale GmbH & Co KG | http://omniscale.com
http://mapproxy.org | https://github.com/olt | @oltonn
MapProxy mailing list
MapProxy at lists.osgeo.org
More information about the MapProxy