<div dir="ltr">Gents,<div><br></div><div>Thanks very much for the replies.</div><div><br></div><div>Oliver,<br></div><div><br></div><div>I have not checked the I/O rate, but I will definitely do. I am seeding to SQLite from PostGIS, but I guess if there are issues with heavy read concurrency it could be the same for heavy write concurrency.</div>
<div><br></div><div>Martin,</div><div><br></div><div>The source is a PostGIS running on the same machine so there is no network overhead or remote sources, but I'll give the file cache a go, see what comes out.</div><div>
<br></div><div>Cheers,</div><div>Attila</div><div class="gmail_extra"><br><br><div class="gmail_quote">2013/11/19 Martin Koke¹ <span dir="ltr"><<a href="mailto:shr3k@typo3-hosting.com" target="_blank">shr3k@typo3-hosting.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Attila,<br>
<br>
What concerns first case with real machine, was that seeding to mbtiles too? I experienced same problems with mbtiles locking with multiple seeding processes in case of seeding from much slower remote WMSes than any local renderer and definitely given up - now I'm using TileCache directory structure.<br>
<br>
Martin<br>
_____<br>
From: Oliver Tonnhofer [mailto:<a href="mailto:olt@omniscale.de">olt@omniscale.de</a>]<br>
To: BERÉNYI Attila [mailto:<a href="mailto:aberenyi@gislab.hu">aberenyi@gislab.hu</a>]<br>
Cc: <a href="mailto:mapproxy@lists.osgeo.org">mapproxy@lists.osgeo.org</a><br>
Sent: Mon, 18 Nov 2013 17:57:06 +0100<br>
Subject: Re: [MapProxy] Performance tuning<br>
<div><div class="h5"><br>
<br>
On 18.11.2013, at 16:54, BERÉNYI Attila wrote:<br>
> I have googled quite a lot to find the information I am looking for - no luck so far. I am well aware this is the hundredth time that someone ask about performance tuning - please bear with me.<br>
><br>
> First things first, MapProxy is awesome, well done guys!<br>
><br>
> So, I have two separate test cases:<br>
> • "Real" machine<br>
> • Xeon 5355, 24GB RAM, PostgreSQL 9.3.1-3, PostGIS 2.1.0-2 running on Arch.<br>
> • Tile generation chain: PostGIS -> Geoserver -> MapProxy.<br>
> • Geographic extent: [-10, 46, 4, 65] (EPSG:4326)<br>
> • Zoom levels: 19.<br>
> • I have tried several -c (up to 160) parameters (i.e. number of parallel seed processes) but I have never reached the optimal 8-ish re the 15 minutes average load, even though I have adjusted the parameters in Geoserver and PostgreSQL as well.<br>
<br>
The source services are the bottleneck in most cases. Have you checked your IO load(e.g. %wa in top)? How many tiles/seconds do you seed?<br>
<br>
<br>
> • My tests show that even 2 parallel processes per vCPU (i.e. -c 32) cause issue/bottleneck (<a href="http://pastebin.com/raw.php?i=29MPZcYp" target="_blank">http://pastebin.com/raw.php?i=29MPZcYp</a>). However the 15 minutes average load is only ~8.<br>
<br>
Again, how many tiles/seconds do you seed? SQLite/MBTiles does not work well with heavy read concurrency, but this hasn't been a problem in my seeding processes. Maybe it's necessary to increase the timeout in SQLite during seeding (not configurable ATM).<br>
<br>
<br>
> And finally a question/suggestion: have you ever considered to utilise the GPU's capabilities within the worker processes (<a href="http://gis.stackexchange.com/questions/77181/generate-tiles-using-gpu" target="_blank">http://gis.stackexchange.com/questions/77181/generate-tiles-using-gpu</a>)?<br>
<br>
You should ask on the Mapnik/Geoserver list.<br>
<br>
<br>
Regards,<br>
Oliver<br>
<br>
--<br>
Oliver Tonnhofer | Omniscale GmbH & Co KG | <a href="http://omniscale.com" target="_blank">http://omniscale.com</a><br>
<a href="http://mapproxy.org" target="_blank">http://mapproxy.org</a> | <a href="https://github.com/olt" target="_blank">https://github.com/olt</a> | @oltonn<br>
<br>
<br>
<br>
<br>
<br>
<br>
<br>
</div></div>_______________________________________________<br>
MapProxy mailing list<br>
<a href="mailto:MapProxy@lists.osgeo.org">MapProxy@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/mapproxy" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapproxy</a><br>
_______________________________________________<br>
MapProxy mailing list<br>
<a href="mailto:MapProxy@lists.osgeo.org">MapProxy@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/mapproxy" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapproxy</a></blockquote></div>
</div></div>