[Gdal-dev] speeding up the gdalwarp process for mosaicing andreprojection

[Ed McNierney]

John -

 

Are the input and output files on the same disk subsystem, and what kind
of disk(s) is involved?

 

gdalwarp will run as a single task, so if it uses 1 processor 100% of
the time you'll see 25% utilization.  If you run two gdalwarp jobs at
the same time, using the same disks, you can very easily cause disk
thrashing and bog the whole process down, so it's not surprising to find
two jobs in parallel takes more than two jobs run serially.

 

If possible, have the input and output files on different disks, and
avoid having read/write activity on the same disk or more than one job
writing to the same disk.

 

-          Ed

 

Ed McNierney

President and Chief Mapmaker

TopoZone.com / Maps a la carte, Inc.

73 Princeton Street, Suite 305

North Chelmsford, MA  01863

Phone: 978-251-4242, Fax: 978-251-1396

ed at topozone.com

 

From: gdal-dev-bounces at lists.maptools.org
[mailto:gdal-dev-bounces at lists.maptools.org] On Behalf Of John Mitchell
Sent: Monday, May 14, 2007 5:52 PM
To: mitchelljj98 at gmail.com; Gdal-Dev; MAPSERVER-USERS at lists.umn.edu
Cc: Mark A Hollis
Subject: [Gdal-dev] speeding up the gdalwarp process for mosaicing
andreprojection

 

Hi,

I have run the following gdalwarp command

gdalwarp.exe -wo SKIP_NOSOURCE=YES -rc -s_srs EPSG:" + EPSG + " -t_srs
EPSG:" + EPSG_wgs84 + " -of HFA " + " -q " + inputFiles + " " +
outputFile 

against a group of geotiff files which both mosaics and reprojects them
to wgs84 and it works fine, but it takes 22:47 hours to generate the
52.89 GB mosaic.  
Does anyone have any suggestions on how to speed up the gdalwarp
process.  I have noticed that the 4 processor server is only being
utilized between 14-26% across all 4 processors.  Is their a way to
force a higher utilization of these processors?  I have in past tried to
run 2 gdalwarp jobs simultaneously and I have noticed that it takes them
much longer compared to the total time when running serially (meaning
one after the other). 

Thanks,


-- 
John J. Mitchell 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20070514/2fc08d7a/attachment.htm>