[gdal-dev] How to make my GDAL code faster?

[Benoît Andrieu]

Hi Nicolas !

I would be interested to know what you finally did ! ^^

By "rolling mean", I think he suggests to not make all the computations for every window of 5x5 pixels.
If your mean uses a simple computation (average for example), you don't need to sum all the values for every window ; you can keep the previous results to make your computation.

About the multithreading, I don't use it in my code for the GDAL part (reading writing protected by mutexes) but I use it for the processing part.

Regards and bonne chance ! ^^

Benoît Andrieu
bea at ixsea.com
benoit.andrieu at gmail.com
  ----- Original Message ----- 
  From: Nicolas DEGARNE 
  To: Frank Warmerdam ; gdal-dev at lists.osgeo.org 
  Sent: Wednesday, April 15, 2009 9:27 PM
  Subject: Re: [gdal-dev] How to make my GDAL code faster?


  Hy,

  Thanks for your help Frank,it's allowed me to improve my program from 200s to 14s that's show a great progress.

  I didn't understand what is "rolling mean" could you please explain me?

  An other question, is it possible to make multithread process?

  thanks a lot for your help

  Best regards,

  Nicolas







  Once you overhaul things to operate on substantial swaths of data, there
  may also be some other opportunities for optimizing within your algorithm
  (rolling means, for instance) but first solve the big IO overhead bottleneck.






  2009/4/15 Frank Warmerdam <warmerdam at pobox.com>

    Nicolas DEGARNE wrote:

      I read on help and /gdal.h/ File Reference that it could exist faster way so I have some ideas like :                   Merging the code to c++
                  Using Read/Write block
                  Use Tile/Block to accelerate the process
                  Use the Warpprocess to calulate windows mean's faster

      I hope you understand my request and it was clear
             



    Nicolas,

    My advice would be to read an entire 5 line swath into a buffer, and
    then operate within that buffer.  Very small read/write operations
    are pretty expensive with GDAL and your approach of reading five pixels
    off one line as a single request is very very fine grained.

    Switching to C++ will not help noticably (just reducing one level of
    function call overhead, and a bit of extra checking the C API provides).

    At this point there is little value in going to exact TIFF tiles or using
    the block API.

    The Warp API is also going to be very high overhead to compute means.

    Once you overhaul things to operate on substantial swaths of data, there
    may also be some other opportunities for optimizing within your algorithm
    (rolling means, for instance) but first solve the big IO overhead bottleneck.

    Best regards,
    -- 
    ---------------------------------------+--------------------------------------
    I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
    light and sound - activate the windows | http://pobox.com/~warmerdam
    and watch the world go round - Rush    | Geospatial Programmer for Rent





  -- 
  Nicolas Degarne
  76 rue de Nancy
  94170 Le Perreux
  06 84 93 80 94
  nicolas.degarne at gmail.com




------------------------------------------------------------------------------


  _______________________________________________
  gdal-dev mailing list
  gdal-dev at lists.osgeo.org
  http://lists.osgeo.org/mailman/listinfo/gdal-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20090416/ce1eaafc/attachment.html