[GRASS-user] error in i.sentinel.mask: nan values in xml file

Wed Jan 29 14:14:56 PST 2020

Hi Robi,

So I found that i.sentinel.import fails for latlong locations. I moved to
UTM as in the original data and all fine. I reached the cloud masking step,
and I find a different problem now.

Here's the command and an extract of the output:
GRASS 7.9.dev (dengue_cba):~/webgis > i.sentinel.mask -s \
>   blue=$blue \
>   green=$green \
>   red=$red \
>   swir11=$swir11 \
>   nir=$nir \
>   swir12=$swir12 \
>   nir8a=$nir8a \
>   cloud_raster=cloud \
>   shadow_raster=shadow \
>   mtd_file=`find $out_dir -name 'MTD_TL.xml'` \
>   scale_fac=10000
WARNING: All subsequent operations will be limited to the current
         computational region
--- Start rescaling bands ---
T21JYM_20200126T134201_B02_10m
 100%
T21JYM_20200126T134201_B03_10m
 100%
T21JYM_20200126T134201_B04_10m
 100%
T21JYM_20200126T134201_B08_10m
 100%
T21JYM_20200126T134201_B8A_20m
 100%
T21JYM_20200126T134201_B11_20m
 100%
T21JYM_20200126T134201_B12_20m
 100%

T21JYM_20200126T134201_B02_10m_double,T21JYM_20200126T134201_B03_10m_double,T21JYM_20200126T134201_B04_10m_double,T21JYM_20200126T134201_B08_10m_double,T21JYM_20200126T134201_B8A_20m_double,T21JYM_20200126T134201_B11_20m_double,T21JYM_20200126T134201_B12_20m_double
--- All bands have been rescaled ---
--- Start computing maximum values of bands ---
T21JYM_20200126T134201_B02_10m_double
T21JYM_20200126T134201_B03_10m_double
T21JYM_20200126T134201_B04_10m_double
T21JYM_20200126T134201_B08_10m_double
T21JYM_20200126T134201_B8A_20m_double
T21JYM_20200126T134201_B11_20m_double
T21JYM_20200126T134201_B12_20m_double
--- Computed maximum value: dict_values([0.9352, 1.168, 1.1056, 1.0704,
0.7023, 0.9381, 1.3597]) ---
--- Statistics have been computed! ---
--- Start clouds detection procedure ---
--- Computing cloud mask... ---
 100%
--- Converting raster cloud mask into vector map ---
Extracting areas...
 100%
Writing areas...
 100%
Building topology for vector map <cloud_v_5922 at carla>...
Registering primitives...
Building areas...
 100%
Attaching islands...
 100%
Attaching centroids...
 100%
r.to.vect complete.
--- Cleaning geometries ---
[...]
--- Start removing misclassification from the shadow mask ---
--- Data preparation... ---
WARNING: Values in column <cat> will be overwritten
WARNING: Number of centroids exceeds number of areas: 24 > 22
WARNING: Number of duplicate centroids: 2
WARNING: Values in column <cat> will be overwritten
WARNING: Values in column <cat> will be overwritten
--- Reading mean sun zenith and azimuth from metadata file to compute
clouds shift ---
--- the mean sun Zenith is: 28.732 deg ---
--- the mean sun Azimuth is: 82.028 deg ---
--- Start computing the east and north clouds shift at steps of 100m of
clouds height---
Traceback (most recent call last):
  File "/home/veroandreo/.grass7/addons/scripts/i.sentinel.mask", line 638,
in <module>
    main()
  File "/home/veroandreo/.grass7/addons/scripts/i.sentinel.mask", line 561,
in main
    quiet=True, stderr=subprocess.DEVNULL)
  File
"/home/veroandreo/software/grass79-dev/dist.x86_64-pc-linux-gnu/etc/python/grass/script/core.py",
line 499, in run_command
    return handle_errors(returncode, result=None, args=args, kwargs=kwargs)
  File
"/home/veroandreo/software/grass79-dev/dist.x86_64-pc-linux-gnu/etc/python/grass/script/core.py",
line 392, in handle_errors
    returncode=returncode)
grass.exceptions.CalledModuleError: Module run v.overlay v.overlay --o --q
ainput=addcat_5922 binput=cl_shift_5922 operator=and output=overlay_5922
ended with error
Process ended with non-zero return code 1. See errors in the (error) output.

Where is this error output to inspect?

This is the UUID in case you want to test:
541021f8-63f5-4e9d-ba28-425c4c8451df

Thanks much in advance
Vero

ps: I use grass-dev with python 3.7.4 (because of the ctypes issue)

El mié., 29 ene. 2020 a las 14:53, Veronica Andreo (<veroandreo at gmail.com>)
escribió:

> Hi Robi,
>
> My bad, region was wrong... anyway, trying to replicate from the beginning
> I see that i.sentinel.import with -r is importing empty maps. However, the
> jp2 files have data. I can visualize them in QGIS. I'll report in a
> different thread. Once I can figure it out and reach this point again, I'll
> report back to you. Thanks for the clarification!
>
> Cheers
> Vero
>
> El mié., 29 ene. 2020 a las 9:08, roberta fagandini (<
> robifagandini at hotmail.it>) escribió:
>
>> Hi Vero,
>> i.sentinel.mask doesn't retrieve the maximum from the xml file but
>> computes the value from the image using r.univar. The error could be due to
>> the computational region..has the region been set correctly?
>> Can you send the MTD_TL.xml file? I can download the image and make some
>> tests.
>>
>> Roberta
>>
>> ------------------------------
>> *Da:* grass-user <grass-user-bounces at lists.osgeo.org> per conto di
>> Veronica Andreo <veroandreo at gmail.com>
>> *Inviato:* martedì 28 gennaio 2020 19:57
>> *A:* grass-user <grass-user at lists.osgeo.org>
>> *Oggetto:* [GRASS-user] error in i.sentinel.mask: nan values in xml file
>>
>> Hello
>>
>> I try to use i.sentinel.mask but I get the following error:
>>
>> i.sentinel.mask blue=$blue green=$green red=$red swir11=$swir11 nir=$nir
>> swir12=$swir12 nir8a=$nir8a cloud_mask=cloud shadow_mask=shadow
>> mtd_file=`find $out_dir -name 'MTD_TL.xml'`
>>
>> ADVERTENCIA: All subsequent operations will be limited to the current
>>              computational region
>> ADVERTENCIA: Any rescale factor has been applied
>> --- Start computing maximum values of bands ---
>> T21JYM_20200126T134201_B02_10m
>> T21JYM_20200126T134201_B03_10m
>> T21JYM_20200126T134201_B04_10m
>> T21JYM_20200126T134201_B08_10m
>> T21JYM_20200126T134201_B8A_20m
>> T21JYM_20200126T134201_B11_20m
>> T21JYM_20200126T134201_B12_20m
>> --- Computed maximum value: dict_values([nan, nan, nan, nan, nan, nan,
>> nan]) ---
>> --- Statistics have been computed! ---
>> --- Start clouds detection procedure ---
>> --- Computing cloud mask... ---
>> Mapa no válido <nan>
>> Parse error
>> ERROR: error al parsear
>> ERROR: An error occurred while running r.mapcalc with expression:
>>        cloud_def82115 = if((((T21JYM_20200126T134201_B02_10m > (0.08*nan))
>>        && (T21JYM_20200126T134201_B03_10m > (0.08*nan)) &&
>>        (T21JYM_20200126T134201_B04_10m > (0.08*nan))) == 1) &&
>>        (((T21JYM_20200126T134201_B04_10m < ((0.08*nan)*1.5)) &&
>>        (T21JYM_20200126T134201_B04_10m >
>>        T21JYM_20200126T134201_B12_20m*1.3)) == 0) &&
>>        (((T21JYM_20200126T134201_B11_20m < (0.1*nan)) &&
>>        (T21JYM_20200126T134201_B12_20m < (0.1*nan))) == 0) &&
>>        ((if(T21JYM_20200126T134201_B8A_20m ==
>>        max(T21JYM_20200126T134201_B8A_20m, 2 *
>>        T21JYM_20200126T134201_B02_10m, 2 * T21JYM_20200126T134201_B03_10m,
>>        2 * T21JYM_20200126T134201_B04_10m))) == 0) &&
>>        ((T21JYM_20200126T134201_B02_10m > 0.2) == 1), 0, null())
>>
>> I checked the xml file and there are a lot of nan's yes, but there are
>> also real values. Why doesn't the max function in this part:
>> https://github.com/OSGeo/grass-addons/blob/master/grass7/imagery/i.sentinel/i.sentinel.mask/i.sentinel.mask.py#L321
>> recognize them? Isn't there a way to avoid nan while computing max?
>>
>> Thanks much in advance
>> Vero
>>
>> ps: I can provide the xml if needed.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20200129/42e52f5f/attachment.html>