[GRASS-dev] r.in.gdal changes range of values for sentinel bands (?)

Tue Feb 6 12:33:43 PST 2018

On Tue, Feb 6, 2018 at 10:25 AM, Veronica Andreo <veroandreo at gmail.com>
wrote:
>
> Sorry, sorry... that was a different band... I realize now. So, for the
record, no issues,
> gdalinfo -mm T21KZP_20171212T134159_B02.jp2
> reports the same range as the one I see after importing in grass.
>
> Computed Min/Max=15.000,17547.000
>
> Again, sorry for the noise and thanks for the -mm hint, MM :)

Good to hear that gdalinfo -mm and GRASS stats are indeed identical.

A hint: use r.info -s instead of r.univar -g for simple stats (number of
non-null cells, min, max, mean, stddev)  on the full raster map, it's
faster.

The reason why gdalinfo with and without -mm might report different values
is that some software creating raster datasets for some obscure reason
estimates min, max, mean, stddev and for some even more obscure reason
writes these estimates to metadata. At least regarding min, max one would
expect actual min, max, not some estimate with estimated min > actual min
and estimated max < actual max.

Markus M
>
> best,
> Vero
>
> 2018-02-05 20:54 GMT+01:00 Veronica Andreo <veroandreo at gmail.com>:
>>
>> Forgot the list, sorry...
>>
>> -------
>>
>> Hi Markus,
>>
>> here the output of gdalinfo -mm for the same band reported earlier:
>>
>> gdalinfo -mm T21KZP_20171212T134159_B01.jp2
>> Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
>> Files: T21KZP_20171212T134159_B01.jp2
>> [...]
>>
>> Band 1 Block=192x192 Type=UInt16, ColorInterp=Gray
>>   Min=908.000 Max=1511.000   Computed Min/Max=934.000,6934.000
>>   Minimum=908.000, Maximum=1511.000, Mean=1138.806, StdDev=74.933
>>   Overviews: 915x915, 457x457, 228x228, 114x114
>>   Overviews: arbitrary
>>   Metadata:
>>     STATISTICS_MAXIMUM=1511
>>     STATISTICS_MEAN=1138.8064
>>     STATISTICS_MINIMUM=908
>>     STATISTICS_STDDEV=74.933459275813
>>   Image Structure Metadata:
>>     COMPRESSION=JPEG2000
>>     NBITS=15
>>
>> still differs a lot from what I get after importing into GRASS.
>>
>> System Info

>> GRASS version: 7.5.svn

>> GRASS SVN revision: r72203

>> Build date: 2018-01-01

>> Build platform: x86_64-pc-linux-gnu

>> GDAL: 2.1.3

>> PROJ.4: 4.9.3

>> GEOS: 3.6.1

>> SQLite: 3.20.1

>> Python: 2.7.14

>> wxPython: 3.0.2.0

>> Platform: Linux-4.14.14-200.fc26.x86_64-x86_64-with-fedora-26-Twenty_Six
>>
>> best,
>> Vero
>>
>>
>> 2018-02-05 19:49 GMT+01:00 Markus Metz <markus.metz.giswork at gmail.com>:
>>>
>>>
>>>
>>> On Mon, Feb 5, 2018 at 1:14 PM, Veronica Andreo <veroandreo at gmail.com>
wrote:
>>> >
>>> > I tested also with 74 release branch... same result
>>> >
>>> > 2018-02-05 13:05 GMT+01:00 Veronica Andreo <veroandreo at gmail.com>:
>>> >>
>>> >> Hi devs
>>> >>
>>> >> I'm testing Martin's new add-on, r.sentinel.import. I created an
UTM21S location for my data and imported them with:
>>> >>
>>> >> r.sentinel.import
input=Downloads/S2B_MSIL1C_20171212T134159_N0206_R124_T21KZP_20171212T201017.SAFE/GRANULE/L1C_T21KZP_A004011_20171212T134158/IMG_DATA/
>>> >>
>>> >> The problem arose when I tried to use i.color.enhance to display an
RGB: I got blank screens, so I thought there must be something with the
range of values.
>>> >>
>>> >> r.univar map=T21KZP_20171212T134159_B02 at PERMANENT

>>> >> total null and non-null cells: 120560400
>>> >> total null cells: 0
>>> >> Of the non-null cells:
>>> >> ----------------------
>>> >> n: 120560400
>>> >> minimum: 15
>>> >> maximum: 17547
>>> >> range: 17532
>>> >> mean: 913.778
>>> >> mean of absolute values: 913.778
>>> >> standard deviation: 134.322
>>> >> variance: 18042.4
>>> >> variation coefficient: 14.6996 %
>>> >> sum: 110165471238
>>> >>
>>> >> that already seemed odd...
>>> >> I checked the original band2 jp2 file... Values are different...
>>> >>
>>> >> gdalinfo T21KZP_20171212T134159_B02.jp2
>>> >> Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
>>> >> Files: T21KZP_20171212T134159_B02.jp2
>>> >> ...
>>> >> Band 1 Block=1024x1024 Type=UInt16, ColorInterp=Gray
>>> >>   Min=602.000 Max=1576.000
>>> >>   Minimum=602.000, Maximum=1576.000, Mean=916.799, StdDev=123.509
>>> >>   Overviews: 5490x5490, 2745x2745, 1372x1372, 686x686
>>> >>   Overviews: arbitrary
>>> >>   Metadata:
>>> >>     STATISTICS_MAXIMUM=1576
>>> >>     STATISTICS_MEAN=916.7992
>>> >>     STATISTICS_MINIMUM=602
>>> >>     STATISTICS_STDDEV=123.50943477872
>>> >>   Image Structure Metadata:
>>> >>     COMPRESSION=JPEG2000
>>> >>     NBITS=15
>>>
>>> Can you try again with gdalinfo -mm? This forces computation of the
actual min/max values, which can differ from what is stored in the metadata
(an approximation).
>>>
>>> There can also be something wrong with i.color.enhance
>>>
>>> Markus M
>>>
>>> >>
>>> >> Just to check, I imported with r.in.gdal and I'm getting exactly the
same as with r.sentinel.import.
>>> >>
>>> >> What am I doing wrong here? Why would r.in.gdal change values like
that? What else needs to be set? I don't remember to have this problem
before and it happens in all bands as far as I've checked.
>>> >>
>>> >> Ah, I'm using trunk r72203
>>> >>
>>> >> Thanks in advance for any help :)
>>> >>
>>> >> Cheers,
>>> >> Vero
>>> >>
>>> >>
>>> >>
>>> >
>>> >
>>> > _______________________________________________
>>> > grass-dev mailing list
>>> > grass-dev at lists.osgeo.org
>>> > https://lists.osgeo.org/mailman/listinfo/grass-dev
>>>
>>
>>
>
>
> _______________________________________________
> grass-dev mailing list
> grass-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/grass-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20180206/405d70bc/attachment.html>