[Gdal-dev] performance issue with gdal_translate...

[Ed McNierney]

John -

In addition to Frank's note, I would point out that while you're using
0.5% of memory, you're also using 100% of the CPU.  Increasing the
amount of memory available to GDAL will avoid/minimize disk I/O, but if
disk I/O is a major performance factor then I would not expect you to
see 100% CPU usage.  Your CPU can't spend more than 100% of its time on
gdal_translate, so it is possible that your situation is CPU-bound.

It is also possible that that CPU time is partially or largely devoted
to disk buffer processing, so increasing GDAL_CACHEMAX is certainly a
very good idea.

	- Ed

Ed McNierney
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
Phone: +1 (978) 251-4242
Fax: +1 (978) 251-1396
ed at topozone.com 

-----Original Message-----
From: gdal-dev-bounces at lists.maptools.org
[mailto:gdal-dev-bounces at lists.maptools.org] On Behalf Of Frank
Warmerdam
Sent: Wednesday, May 31, 2006 3:34 PM
To: John Cole
Cc: 'gdal-dev at lists.maptools.org'
Subject: Re: [Gdal-dev] performance issue with gdal_translate...

John Cole wrote:
> I'm creating a tiled geotif on a large jpg file
> 
> gdal_translate -of GTiff -co "TILED=YES" madison_1f_01.jpg 
> madison_1f_01.tif
> 
> Input file size is 44400, 42600
> 
> Gdal_translate is working, but it's really slow and only using 0.5% of

> memory (according to top).
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  4350 root      25   0 65228  22m 6252 R  100  0.5   2:18.82
gdal_translate
> 
> Since I have a decent system for this, I was wondering what --config 
> switches I could use to speed this up?
> 
> I'm running Fedora Core 5 on a dual xeon 3ghz with 4GB of memory.

John,

I think you want to force GDAL_CACHEMAX up.  When reformatting scanline
oriented file to tiles or tiles to scanlines it is important to have a
cache large enough to hold a whole row of tiles or performance degrades
seriously.

GDAL_CACHEMAX is normally a number of megabytes, and the default is 10 I
think.

So something like:

gdal_translate -of GTIFF -co TILED=YES --config GDAL_CACHEMAX 120 \
   madison_1f_01.jpg madison_1f_01.tif

Would use a 120MB cache.

Best regards,
-- 
---------------------------------------+--------------------------------
---------------------------------------+------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF,
http://osgeo.org

_______________________________________________
Gdal-dev mailing list
Gdal-dev at lists.maptools.org
http://lists.maptools.org/mailman/listinfo/gdal-dev