[Gdal-dev] Benchmarks about LUT colour correction and JPEG2000 drivers

[Rahkonen Jukka]

Hi,
 
I did some benchmarking while making an acceptance test for a new feature in MapServer, LUT colour correction on-the-fly by using predefined LUT-tables.  More information about this fine feature here:
http://mapserver.gis.umn.edu/docs/howto/raster_data/#special-processing-directives <http://mapserver.gis.umn.edu/docs/howto/raster_data/#special-processing-directives> 
Downloadable ready made MapServer -version for Windows can be found here:
 http://www.gdal.org/dl/fwtools/FWTools109.exe

Frank W. supposed my findings might interest the people reading mapserver-users or even gdal-dev lists, so I cross post this short kind of a test report to both.  
 
I had already checked that the new feature worked as supposed and next I was willing to see how big an effect applying LUT tables on-the-fly has on the efficiency of MapServer.  I set up a Jmeter project to send series of individual WMS requests to MapServer and recorder the server throughput with or without PROSESSING "LUT=..." argument.  Because I have the same images in GeoTIFF, JPEG2000 and ECW formats I desided to make also a quick test to see how MapServer performs with those.  There were actually four members in the format test because FWTools MapServer ships with two JPEG2000 drivers, JP2KAK and JP2ECW.
 
Here are the results in brief:
 
LUT-correction has a negligible effect on MapServer efficiency
   - server throughput without LUT-correction: 271 requests per minute
   - server throughput without LUT-correction: 268 requests per minute
 
There is a really significant difference between the two JPEG2000 drivers, JP2KAK and JP2ECW
   - server throughput with JP2ECW driver: 80 requests per minute
   - server throughput with JP2KAK driver: 24 requests per minute
 
MapServer likes ECW more than JPEG2000
  - server throughput with ECW images: 120 requests per minute
 
My tests should be otherwise quite reliable, but I am not sure if my test set is suitable for straight comparison between JPEG2000 and ECW because my JPEG2000 images were lossless and ECW images were compressed about to the ratio of 1 to 5.  However I am not aware of lossless JPEG2000 files being much harder to decompress than lossy ones even the file size is bigger if JPEG2000 codestreams are organised in a reasonable way, and that should be the case with my JPEG2000 files.
 
I believe that the results about ECW and JPEG2000 can not be generalised but they just tell what I was testing: how MapServer (or perhaps it is GDAL?) utilises them with certain drivers.  I have not found that big difference between ECW and JPEG2000 with other software, and to my experience Kakadu is about (or at least) as fast as ECW JPEG2000.   But for sure it seems to be so that MapServer performance with the current JP2KAK driver is quite bad compared to JP2ECW driver.  It should be no surprise that tiled TIFFs with overviews really suit MapServer best.
 
-Jukka Rahkonen-