[MapProxy] mapproxy-seed very slow
Sebastian E. Ovide
sebastian.ovide at gmail.com
Fri Aug 20 07:21:52 EDT 2010
Hi Oliver,
On Fri, Aug 20, 2010 at 11:59 AM, Oliver Tonnhofer <olt at omniscale.de> wrote:
>
> On 20.08.2010, at 12:06, Sebastian E. Ovide wrote:
> > It takes a loooong time to start using 100% of only 1 CPU for creating
> the first tail... after that it creates around 10 tiles a second...
>
>
ok, changing the extent (bbox) this problem is solved
> Can you post your configuration?
> It should start fast unless you start seeding from a very low level, say
> from [13-15]. How did you calculated the 10 tiles/s? With meta-tiles you
> will get multiple tiles for each WMS request.
>
>
seed.yaml
views:
local_raster:
level: [12, 17]
srs: ['EPSG:3857']
bbox: [ -628920.885545, 6445802.356116, 194676.378521,
7804820.959957 ]
bbox_srs: EPSG:3785
local_shape:
level: [12, 17]
srs: ['EPSG:3857']
bbox: [ -628920.885545, 6445802.356116, 194676.378521,
7804820.959957 ]
bbox_srs: EPSG:3785
seeds:
local_raster:
views: ['local_raster']
local_shape:
views: ['local_shape']
proxy.yaml is empty
[12:14:11] 16 2.16% (4891.96981, 7797799.87754, 9783.93962, 7802691.84735)
(#592) ETA: 2010-08-20-12:58:52
[12:14:15] 15 2.16% (9783.93962, 7797799.87754, 19567.87924,
7804820.95996) (#616) ETA: 2010-08-20-12:58:55
[12:14:15] 16 2.16% (9783.93962, 7802691.84735, 14675.90943,
7804820.95996) (#616) ETA: 2010-08-20-12:58:55
[12:14:17] 16 2.17% (14675.90943, 7802691.84735, 19567.87924,
7804820.95996) (#626) ETA: 2010-08-20-12:58:57
[12:14:18] 16 2.17% (9783.93962, 7797799.87754, 14675.90943,
7802691.84735) (#636) ETA: 2010-08-20-12:58:59
[12:14:22] 16 2.17% (14675.90943, 7797799.87754, 19567.87924,
7802691.84735) (#656) ETA: 2010-08-20-12:59:02
[12:14:25] 15 2.18% (0.00000, 7788015.93792, 9783.93962, 7797799.87754)
(#680) ETA: 2010-08-20-12:59:06
[12:14:25] 16 2.18% (0.00000, 7792907.90773, 4891.96981, 7797799.87754)
(#680) ETA: 2010-08-20-12:59:06
[12:14:28] 16 2.18% (4891.96981, 7792907.90773, 9783.93962, 7797799.87754)
(#700) ETA: 2010-08-20-12:59:08
from 12:14:15 to 12:14:25, 680-616 64 tails in 10 seconds...
> > using top I can see between 2 mapproxy-seed script. using only 2 CPUs
> (over 4) and CPU% around 70%... I can see also others mapproxy-seed using 0%
> CPU...
>
> mapproxy-seed has a --concurrency option (or -c). Try to increase that.
>
>
with mapproxy-seed -f etc/proxy.yaml etc/seed.yaml -c 4
[12:17:32] 16 2.45% (88055.45658, 7792907.90773, 92947.42639,
7797799.87754) (#781) ETA: 2010-08-20-13:42:50
[12:17:35] 16 2.45% (92947.42639, 7792907.90773, 97839.39621,
7797799.87754) (#801) ETA: 2010-08-20-13:42:53
[12:17:38] 16 2.45% (88055.45658, 7788015.93792, 92947.42639,
7792907.90773) (#821) ETA: 2010-08-20-13:42:55
[12:17:41] 16 2.46% (92947.42639, 7788015.93792, 97839.39621,
7792907.90773) (#841) ETA: 2010-08-20-13:42:58
[12:17:45] 14 2.46% (97839.39621, 7788015.93792, 117407.27545,
7804820.95996) (#869) ETA: 2010-08-20-13:43:02
[12:17:45] 15 2.46% (97839.39621, 7797799.87754, 107623.33583,
7804820.95996) (#869) ETA: 2010-08-20-13:43:02
[12:17:45] 16 2.46% (97839.39621, 7802691.84735, 102731.36602,
7804820.95996) (#869) ETA: 2010-08-20-13:43:02
[12:17:46] 16 2.47% (102731.36602, 7802691.84735, 107623.33583,
7804820.95996) (#879) ETA: 2010-08-20-13:43:03
[12:17:48] 16 2.47% (97839.39621, 7797799.87754, 102731.36602,
7802691.84735) (#889) ETA: 2010-08-20-13:43:04
[12:17:51] 16 2.47% (102731.36602, 7797799.87754, 107623.33583,
7802691.84735) (#909) ETA: 2010-08-20-13:43:07
[12:17:53] 15 2.48% (107623.33583, 7797799.87754, 117407.27545,
7804820.95996) (#933) ETA: 2010-08-20-13:43:10
[12:17:53] 16 2.48% (107623.33583, 7802691.84735, 112515.30564,
7804820.95996) (#933) ETA: 2010-08-20-13:43:10
[12:17:54] 16 2.48% (112515.30564, 7802691.84735, 117407.27545,
7804820.95996) (#943) ETA: 2010-08-20-13:43:11
[12:17:56] 16 2.49% (107623.33583, 7797799.87754, 112515.30564,
7802691.84735) (#953) ETA: 2010-08-20-13:43:12
[12:17:58] 16 2.49% (112515.30564, 7797799.87754, 117407.27545,
7802691.84735) (#973) ETA: 2010-08-20-13:43:14
[12:18:01] 15 2.49% (97839.39621, 7788015.93792, 107623.33583,
7797799.87754) (#997) ETA: 2010-08-20-13:43:16
[12:18:01] 16 2.49% (97839.39621, 7792907.90773, 102731.36602,
7797799.87754) (#997) ETA: 2010-08-20-13:43:16
[12:18:02] 16 2.50% (102731.36602, 7792907.90773, 107623.33583,
7797799.87754) (#1017) ETA: 2010-08-20-13:43:18
[12:18:04] 16 2.50% (97839.39621, 7788015.93792, 102731.36602,
7792907.90773) (#1037) ETA: 2010-08-20-14:56:55
[12:18:06] 16 2.51% (102731.36602, 7788015.93792, 107623.33583,
7792907.90773) (#1057) ETA: 2010-08-20-14:56:57
[12:18:09] 15 2.51% (107623.33583, 7788015.93792, 117407.27545,
7797799.87754) (#1081) ETA: 2010-08-20-14:56:59
[12:18:09] 16 2.51% (107623.33583, 7792907.90773, 112515.30564,
7797799.87754) (#1081) ETA: 2010-08-20-14:56:59
[12:18:11] 16 2.51% (112515.30564, 7792907.90773, 117407.27545,
7797799.87754) (#1101) ETA: 2010-08-20-14:57:01
[12:18:13] 16 2.52% (107623.33583, 7788015.93792, 112515.30564,
7792907.90773) (#1121) ETA: 2010-08-20-14:57:02
[12:18:15] 16 2.52% (112515.30564, 7788015.93792, 117407.27545,
7792907.90773) (#1141) ETA: 2010-08-20-14:57:04
[12:18:18] 13 2.53% (117407.27545, 7788015.93792, 156543.03393,
7804820.95996) (#1171) ETA: 2010-08-20-14:57:07
[12:18:18] 14 2.53% (117407.27545, 7788015.93792, 136975.15469,
7804820.95996) (#1171) ETA: 2010-08-20-14:57:07
[12:18:18] 15 2.53% (117407.27545, 7797799.87754, 127191.21507,
7804820.95996) (#1171) ETA: 2010-08-20-14:57:07
[12:18:18] 16 2.53% (117407.27545, 7802691.84735, 122299.24526,
7804820.95996) (#1171) ETA: 2010-08-20-14:57:07
[12:18:20] 16 2.53% (122299.24526, 7802691.84735, 127191.21507,
7804820.95996) (#1181) ETA: 2010-08-20-14:57:08
[12:18:21] 16 2.53% (117407.27545, 7797799.87754, 122299.24526,
7802691.84735) (#1191) ETA: 2010-08-20-14:57:09
[12:18:23] 16 2.54% (122299.24526, 7797799.87754, 127191.21507,
7802691.84735) (#1211) ETA: 2010-08-20-14:57:11
[12:18:26] 15 2.54% (127191.21507, 7797799.87754, 136975.15469,
7804820.95996) (#1235) ETA: 2010-08-20-14:57:13
[12:18:26] 16 2.54% (127191.21507, 7802691.84735, 132083.18488,
7804820.95996) (#1235) ETA: 2010-08-20-14:57:13
[12:18:27] 16 2.54% (132083.18488, 7802691.84735, 136975.15469,
7804820.95996) (#1245) ETA: 2010-08-20-14:57:14
[12:18:28] 16 2.55% (127191.21507, 7797799.87754, 132083.18488,
7802691.84735) (#1255) ETA: 2010-08-20-14:57:15
[12:18:30] 16 2.55% (132083.18488, 7797799.87754, 136975.15469,
7802691.84735) (#1275) ETA: 2010-08-20-14:57:16
[12:18:33] 15 2.56% (117407.27545, 7788015.93792, 127191.21507,
7797799.87754) (#1299) ETA: 2010-08-20-14:57:18
[12:18:33] 16 2.56% (117407.27545, 7792907.90773, 122299.24526,
7797799.87754) (#1299) ETA: 2010-08-20-14:57:18
[12:18:35] 16 2.56% (122299.24526, 7792907.90773, 127191.21507,
7797799.87754) (#1319) ETA: 2010-08-20-14:57:20
[12:18:37] 16 2.56% (117407.27545, 7788015.93792, 122299.24526,
7792907.90773) (#1339) ETA: 2010-08-20-14:57:21
from 12:17:32 to 12:18:33 1299 - 781 around 8.6 per second...
> > Can you advice any method to speed up the pretailing ?
>
> Limit the seed area with a shapefile, use --concurrency, use image/tiff as
> request_format, use new PIL version (see mapproxy blog), and use
> link_single_color_images=true.
>
>
need to try this... Will post with the results..
> Regards,
> Oliver
>
> --
> Oliver Tonnhofer <olt at omniscale.de>
> Omniscale - Dominik Helle, Oliver Tonnhofer GbR
> Nadorster Str. 60, 26123 Oldenburg
> Tel: +49(0)441/9392774-2 (Fax: 9)
>
>
Regards
--
Sebastian E. Ovide
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mapproxy/attachments/20100820/3b50cf8c/attachment.html
More information about the MapProxy
mailing list