[gdal-dev] GDAL_retile slowdown at next level
aca at ngi.be
Tue May 11 04:14:41 EDT 2010
It looks like gdal_retile.py slows down when processing a new level of the
Source files : 8065 geotiffs orthophotos of 4000 x 4000 pixels, in total 400
Command : gdal_retile.py -targetDir pyramid/ -levels 12 -v -s_srs
/mnt/webgisdata/geoserver/config/3812.prj -ps 512 512 -r bilinear -co
"COMPRESS=JPEG" -co "INTERLEAVE=PIXEL" -tileIndex orthocol -tileIndexField
Level 0 was processed in 6 days (500000 files, 1 file is processed in 1
second, total 64 GB).
Level 1 starts processing : 1 file is processed in 5 minutes. As it'd take
me a year to finish the level, I stopped it.
Restarted with the same command and -pyramidOnly parameter
Level 1 was processed in 6 days (125000 files, 1 file is processed in 4
seconds, total 17 GB).
Level 2 starts processing : 1 file is processed in 3 minutes. It would take
2 monthes to finish this level.
I'll wait a little more to see the evolution and then stop the process
again, and restart it based on the level 1 files.
Any idea why it slows down like this ?
The source files and the output are on a CIFS fileserver, the computer
processing gdal_retile is a 4 CPU, 2.5 GB RAM. top says :
top - 10:12:57 up 7 days, 23:37, 1 user, load average: 1.00, 1.06, 1.09
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.2%us, 0.0%sy, 0.0%ni, 98.2%id, 0.0%wa, 0.0%hi, 0.0%si,
Mem: 2560220k total, 2506852k used, 53368k free, 21316k buffers
Swap: 2097144k total, 5496k used, 2091648k free, 949224k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4913 ctiadmin 20 0 243m 163m 8064 D 6 6.5 283:36.06 python
Any idea appreciated,
View this message in context: http://osgeo-org.1803224.n2.nabble.com/GDAL-retile-slowdown-at-next-level-tp5034870p5034870.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
More information about the gdal-dev