[gdal-dev] Why is saving an array to ENVI file format using SaveArray so slow?

[mike c]

Dear Even and GDAL community

 

Thanks for the feedback. 


 

 After receiving your
email I tried to narrow down the source of the problem.  On further investigation it seems like indeed,
like you,  writing to my local disk is quick.  I notice, however,  the problem occurs when writing to file shares
 on a local area network.  The remote file shares where the underlying
file system is UNIX  seems to cause the
problem i.e. writing an image (128,4000,128) using  SaveArray(img,remote system
,’ENVI’) from my Windows system to remote
file share(where the file system is UNIX) takes 3-4 minutes. Writing the image  between the same systems to GeoTiff format
appears to be quicker ~24 second. Using the ndarray.tofile() to write the same
image from the same system to the same remote disk takes ~20 seconds.  Copying the same size image between the two
systems using Explorer takes only a few seconds.   Using  the SaveArray(img,remote system ,’ENVI’)   operation
to write the image to  a remote Windows file
share on the same network  takes  a   few seconds. 

 

Any thoughts as to why this is happening and whether there
is a way of overcoming this problem would be welcomed.

 

Thanks 

mike   
> From: even.rouault at spatialys.com
> To: gdal-dev at lists.osgeo.org
> Subject: Re: [gdal-dev] Why is saving an array to ENVI file format using SaveArray so slow?
> Date: Fri, 24 Jul 2015 21:31:30 +0200
> CC: mikec7200 at hotmail.com
> 
> Mike,
> 
> I've tried the following snippet:
> 
> from osgeo import gdal, gdalnumeric
> import numpy
> 
> ar = numpy.zeros( [128,4000,128], dtype = numpy.float32)
> gdalnumeric.SaveArray(ar,'testenvi.bin','ENVI')
> gdalnumeric.SaveArray(ar,'testtif.tif','GTiff')
> 
> And it runs in ~ 2 seconds on both GDAL 1.11 and trunk. (Linux 64bit, but I'm 
> not sure why the OS would account for such a dramatic difference)
> 
> Even
> 
> > Dear GDAL Developers
> > 
> > 
> > 
> > I’m  finding that when
> > saving  a relatively small array (shape =
> > 128,4000,128  ie 128 band x 4000 lines x
> > 128 columns of float data = 256Mb total)  to a ENVI file format using the
> > call:
> > 
> > 
> > 
> > gdalnumeric.SaveArray(img,strf_out,'ENVI')
> > 
> > 
> > 
> > performance is extremely slow (~260 seconds).  When I save the same array
> > to Geotif  format using;
> > 
> > 
> > 
> > gdalnumeric.SaveArray(img,strf_out,'GTIFF')
> > 
> > 
> > 
> > the operation
> > takes  ~24 seconds which is much more
> > acceptable.
> > 
> > 
> > 
> > I’m currently
> > using gdal version 1.11.1 .
> > 
> > 
> > 
> > My question to
> > the forum is then  -why is there such a
> > large discrepancy between the performance of the two operations and is
> > there a way to improve the performance when writing an array to  an ENVI
> > file format?
> > 
> > 
> > 
> > I have tried changing
> > the GDAL Cache options but performance doesn’t seem to alter.  When
> > profiling the code which saves the array to GeoTIFF format I get the
> > output contained in the attached file timing_savearray_GTIFF.txt.  When
> > profiling the code which saves the array to ENVI format I get the output
> > contained in the attached file timing_savearray_ENVI.txt
> > 
> > 
> > 
> > I would like to
> > save the data to ENVI file format.  Your
> > thoughts on this matter would be appreciated.
> > 
> > 
> > 
> > Thanks
> > 
> > 
> > 
> > Regards
> > 
> > mike
> 
> -- 
> Spatialys - Geospatial professional services
> http://www.spatialys.com
 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20150727/159816a0/attachment.html>