[Gdal-dev] performance issue with gdal_translate...

[John Cole]

Frank,
  Thanks, that helped greatly.  BTW, If you put 1000 in, it doesn't appear
to use the value, 999 seems to be the maximum number.  I had tried 1000
earlier, and it didn't do anything so I has assumed that I was using the
wrong switch.

Looks like it will finish in <5minutes now :-)  It had been running for an
hour and hadn't hit the first 2%.

Thanks,

John

-----Original Message-----
From: Frank Warmerdam [mailto:fwarmerdam at gmail.com] On Behalf Of Frank
Warmerdam
Sent: Wednesday, May 31, 2006 2:34 PM
To: John Cole
Cc: 'gdal-dev at lists.maptools.org'
Subject: Re: [Gdal-dev] performance issue with gdal_translate...

John Cole wrote:
> I'm creating a tiled geotif on a large jpg file
> 
> gdal_translate -of GTiff -co "TILED=YES" madison_1f_01.jpg
madison_1f_01.tif
> 
> Input file size is 44400, 42600
> 
> Gdal_translate is working, but it's really slow and only using 0.5% of
> memory (according to top).
> 
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
>  4350 root      25   0 65228  22m 6252 R  100  0.5   2:18.82
gdal_translate
> 
> Since I have a decent system for this, I was wondering what --config
> switches I could use to speed this up?
> 
> I'm running Fedora Core 5 on a dual xeon 3ghz with 4GB of memory.

John,

I think you want to force GDAL_CACHEMAX up.  When reformatting scanline
oriented file to tiles or tiles to scanlines it is important to have
a cache large enough to hold a whole row of tiles or performance degrades
seriously.

GDAL_CACHEMAX is normally a number of megabytes, and the default is 10 I
think.

So something like:

gdal_translate -of GTIFF -co TILED=YES --config GDAL_CACHEMAX 120 \
   madison_1f_01.jpg madison_1f_01.tif

Would use a 120MB cache.

Best regards,
-- 
---------------------------------------+------------------------------------
--
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF, http://osgeo.org
This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the sender. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail.