[mapserver-users] processing scale=auto with float multibands rasters

Tue Aug 4 14:43:59 PDT 2015

Hi Jukka,

The value I use for test was retrieve using qgis defaults so I don't
know if your formula was the same.

So I do a quick test
these are the values for my test raster:

Band 1  Minimum=0.038, Maximum=1.584, Mean=0.135, StdDev=0.026
Band 2  Minimum=0.004, Maximum=1.567, Mean=0.115, StdDev=0.027
Band 3  Minimum=0.003, Maximum=1.636, Mean=0.096, StdDev=0.031
Band 4  Minimum=-0.016, Maximum=1.796, Mean=0.082, StdDev=0.042
Band 5  Minimum=-0.311, Maximum=4.486, Mean=0.256, StdDev=0.138
Band 6  Minimum=-0.197, Maximum=3.277, Mean=0.161, StdDev=0.098
Band 7  Minimum=-0.118, Maximum=2.295, Mean=0.088, StdDev=0.063
Band 8  Minimum=0.039, Maximum=1.366, Mean=0.090, StdDev=0.035
Band 9  Minimum=-0.039, Maximum=0.413, Mean=0.003, StdDev=0.010
Band 10 Minimum=240.580, Maximum=524.602, Mean=299.351, StdDev=6.342
Band 11 Minimum=249.037, Maximum=514.462, Mean=297.375, StdDev=5.855

Using you formula:
(I hope to understand it correctly)

PROCESSING "BANDS=4,3,2"  (the red,green,blue channels)

the values should be:

Band 4:
0.082 - (-0.016 * 0.042) = 0.082672
0.082 + (1.796 * 0.042) = 0.157432

Band 3:
0.096 - (0.003 * 0.031) = 0.095907
0.096 + (1.636 * 0.031) = 0.146716

Band 2:
0.115 - (0.004 * 0.027) = 0.114892
0.115 + (1.567 * 0.027) = 0.157309

so the setting should be:

 PROCESSING "SCALE_1= 0.082672 , 0.157432"
 PROCESSING "SCALE_2= 0.095907 , 0.146716"
 PROCESSING "SCALE_3= 0.114892 , 0.157309"

With this values the image is not correct .

I add a link to a sample images of what the mapserver will produce
with these values.

https://www.dropbox.com/s/izleegjbp18j81f/sample-mapserver-with-values.gif?dl=0

A.

2015-08-04 22:52 GMT+02:00 Rahkonen Jukka (MML)
<jukka.rahkonen at maanmittauslaitos.fi>:
> Hi,
>
> Do we talk about the same thing? If gdalinfo -stats shows
> Minimum=2.000, Maximum=255.000, Mean=96.297, StdDev=37.015
> you should put min at 96.297-(2*37.015)
> You can test the effect with QGIS as described here http://gis.stackexchange.com/questions/87914/how-to-perform-a-standard-deviation-stretch-on-a-raster-in-qgis
> However, I have not really used 64-bit images and don't know what kind of histograms they usually have.
>
> In Mapserver 2.0 the PROCESSING stuff should be possible to set via variable substitution and your users could make the adjustment on-the-fly. Perhaps they could accept that?
>
> -Jukka-
>
> Andrea Peri wrote:
>
>> Hi Jukka, I could open a feature request for this, but before I need to understand if the results with this strategy are affordable for us.
>
>> Infact I do some test with a domain min/max lesser than 2x and it results unaccepatable for our users. The raster are too poor contrasted as reported me from our users.
>> So I'm not sure that an AUTO with a STD stretch and a multiplicator 2x could be instead an acceptable solution.
>
> I should do more tests...
>
> Thx.
>
>
>
> 2015-08-04 20:29 GMT+02:00 Rahkonen Jukka (MML)
> <jukka.rahkonen at maanmittauslaitos.fi>:
>> Hi Andrea,
>>
>> For viewing purposes it is rather common to do LUT stretch by standard deviation which is not so sensible as min-max stretch. Consider making a feature request for a new enhanced PROCESSING AUTO directive which would do the STD stretch, possibly with an optional multiplication. 2 x STD might be a good default value.
>>
>> -Jukka Rahkonen-
>>
>> Andrea Peri wrote:
>>
>>> HI Jukka,
>>> unfortunatelly, I cannot use the LUT because the raster I use is update weekly and every week it could change the min/max values for every bands (it has 11 bands).
>>
>>> Hovewer if it work using the min/max from get map , probably it could be ok form me.
>>> The question s that even when the getmap request the entire bbox the image reported seem to be empty without no error log.
>>
>>
>>
>> 2015-08-04 19:50 GMT+02:00 Rahkonen Jukka (MML)
>> <jukka.rahkonen at maanmittauslaitos.fi>:
>>> Hi,
>>>
>>> I suppose that Even means that with AUTO setting Mapserver is analyzing the min/max pixel values from inside the BBOX of each GetMap and then scales pixel values to 0-256 for the output.  One totally black or white pixel within the BBOX can make the result from 64bit source very bad.
>>>
>>> I would try to use LUT instead of SCALE http://www.mapserver.org/input/raster.html#special-processing-directives.
>>>
>>> -Jukka Rahkonen-
>>>
>>>
>>>
>>>
>>> -----Alkuperäinen viesti-----l
>>> Lähettäjä: mapserver-users-bounces at lists.osgeo.org
>>> [mailto:mapserver-users-bounces at lists.osgeo.org] Puolesta Andrea Peri
>>> Lähetetty: 4. elokuuta 2015 20:07
>>> Vastaanottaja: Even Rouault
>>> Kopio: mapserver-users at lists.osgeo.org
>>> Aihe: Re: [mapserver-users] processing scale=auto with float
>>> multibands rasters
>>>
>>> Hi Even,
>>>
>>> thx for response.
>>>
>>> Please can you help me to better understand the question ?
>>>
>>> My raster is effectively a 18000 x 24000 px image with tiles.
>>>
>>> it have a tile size of 1024 x 1024 produced from gdalwarp:
>>>
>>> gdalwarp -s_srs EPSG:32632 -t_srs EPSG:32632 -overwrite --config
>>> GDAL_CACHEMAX 1000 -wm 1000 -co TILED=YES -co COMPRESS=NONE -co
>>> BLOCKXSIZE=1024 -co BLOCKYSIZE=1024 -co BIGTIFF=YES -co SPARSE_OK=TRUE .......
>>>
>>> and also have overviews from
>>>
>>> gdaladdo --config GDAL_CACHEMAX 1500 -ro -r average ..... 2 4 8 16 32
>>> 64 128 256 512 1024
>>>
>>> If I understand correctly your info.
>>> The mapserver could search the min/max using only the first tile ?
>>>
>>> This mean that it more probably found always only 0 values or near 0 values beacuse these could be the values in the first tile.
>>>
>>> And so it return a white image because the min/max is always min=0/
>>> max=0 (or something quite like this)
>>> I understand correct ?
>>>
>>> thx,
>>>
>>> A.
>>>
>>>
>>> 2015-08-04 16:01 GMT+02:00 Even Rouault <even.rouault at spatialys.com>:
>>>> On Monday 03 August 2015 11:02:43 Andrea Peri wrote:
>>>>> Hi,
>>>>> I have a geotiff multibands with float64 values to serve with a wms
>>>>> mapserver service.
>>>>>
>>>>> With this kind of rasters,
>>>>> is affordable the directive
>>>>>
>>>>>     PROCESSING "BANDS=4,3,2"
>>>>>     PROCESSING "SCALE=AUTO"
>>>>>
>>>>> (the 4,3,2 are the red,green,blue channels)
>>>>>
>>>>> Infact it seem do nothing.
>>>>>
>>>>> Instead all work well if I set:
>>>>>
>>>>>     PROCESSING "BANDS=4,3,2"
>>>>>     PROCESSING "SCALE_1=0.01,0.2"
>>>>>     PROCESSING "SCALE_2=0.01,0.2"
>>>>>     PROCESSING "SCALE_3=0.01,0.2"
>>>>>
>>>>> where ii use the min=0.01 and max=0.2 But I like to have the exact
>>>>> values of min/max for every raster.
>>>>
>>>> Andrea,
>>>>
>>>> SCALE=AUTO should work but looking at the actual implementation I
>>>> can see it computes the min/max on the pixel values intersecting the
>>>> requested area, and not on the whole raster as I'd have expected. So
>>>> in a tiling context, it will likely not produce the expected result.
>>>>
>>>> Even
>>>>
>>>> --
>>>> Spatialys - Geospatial professional services
>>>> http://www.spatialys.com
>>>
>>>
>>>
>>> --
>>> -----------------
>>> Andrea Peri
>>> . . . . . . . . .
>>> qwerty àèìòù
>>> -----------------
>>> _______________________________________________
>>> mapserver-users mailing list
>>> mapserver-users at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>
>>
>>
>> --
>> -----------------
>> Andrea Peri
>> . . . . . . . . .
>> qwerty àèìòù
>> -----------------
>
>
>
> --
> -----------------
> Andrea Peri
> . . . . . . . . .
> qwerty àèìòù
> -----------------

-- 
-----------------
Andrea Peri
. . . . . . . . .
qwerty àèìòù
-----------------