[GRASS-user] error in i.sentinel.mask: nan values in xml file

Thu Jan 30 06:02:35 PST 2020

Ciao Robi,

Thanks for testing! So, it seems it does not work when I set a smaller
region then. I didn't think that could be the problem. If I set the region,
to the full scene, then it completes the process with no errors as in your
case. Did you try setting a smaller region? Is it a problem for this
particular scene or a more general issue with the module, that will only
work for the full scene?

I have opened an issue in GitHub earlier this morning (
https://github.com/OSGeo/grass-addons/issues/85), I'll make a comment there
regarding region setting.

Thanks again!
Vero

El jue., 30 ene. 2020 a las 9:49, roberta fagandini (<
robifagandini at hotmail.it>) escribió:

> Hi Vero,
> I have just tested i.sentinel.mask using your image and the command ended
> without errors (computational region set to the extension fo the image -
> location wgs84/utm21).
>
> i.sentinel.mask -s
> blue=T21JYM_20200126T134201_B02_10m at test_vero
> green=T21JYM_20200126T134201_B03_10m at test_vero
> red=T21JYM_20200126T134201_B04_10m at test_vero
> nir=T21JYM_20200126T134201_B08_10m at test_vero
> nir8a=T21JYM_20200126T134201_B8A_20m at test_vero
> swir11=T21JYM_20200126T134201_B11_20m at test_vero
> swir12=T21JYM_20200126T134201_B12_20m at test_vero
> cloud_mask=v_cloud
> cloud_raster=r_cloud
> shadow_mask=v_shadow
> shadow_raster=r_shadow
> cloud_threshold=50000
> shadow_threshold=10000
>
> mtd_file=S2A_MSIL2A_20200126T134201_N0213_R124_T21JYM_20200126T154908\S2A_MSIL2A_20200126T134201_N0213_R124_T21JYM_20200126T154908.SAFE\GRANULE\L2A_T21JYM_A024002_20200126T134204\MTD_TL.xml
>
> I'm using GRASS 7.8.2 on Windows 8 (please don't judge 😄) .
>
> In your case, the error occurs while computing the east and north clouds
> shift in particular during the v.overlay process. I think the error message
> you get is not a message defined in the code but a standard error message
> from GRASS so I don't know which error output it refers to.
>
> Ciao
> Robi
> ------------------------------
> *Da:* Veronica Andreo <veroandreo at gmail.com>
> *Inviato:* mercoledì 29 gennaio 2020 23:14
> *A:* roberta fagandini <robifagandini at hotmail.it>
> *Cc:* grass-user <grass-user at lists.osgeo.org>
> *Oggetto:* Re: [GRASS-user] error in i.sentinel.mask: nan values in xml
> file
>
> Hi Robi,
>
> So I found that i.sentinel.import fails for latlong locations. I moved to
> UTM as in the original data and all fine. I reached the cloud masking step,
> and I find a different problem now.
>
> Here's the command and an extract of the output:
> GRASS 7.9.dev (dengue_cba):~/webgis > i.sentinel.mask -s \
> >   blue=$blue \
> >   green=$green \
> >   red=$red \
> >   swir11=$swir11 \
> >   nir=$nir \
> >   swir12=$swir12 \
> >   nir8a=$nir8a \
> >   cloud_raster=cloud \
> >   shadow_raster=shadow \
> >   mtd_file=`find $out_dir -name 'MTD_TL.xml'` \
> >   scale_fac=10000
> WARNING: All subsequent operations will be limited to the current
>          computational region
> --- Start rescaling bands ---
> T21JYM_20200126T134201_B02_10m
>  100%
> T21JYM_20200126T134201_B03_10m
>  100%
> T21JYM_20200126T134201_B04_10m
>  100%
> T21JYM_20200126T134201_B08_10m
>  100%
> T21JYM_20200126T134201_B8A_20m
>  100%
> T21JYM_20200126T134201_B11_20m
>  100%
> T21JYM_20200126T134201_B12_20m
>  100%
>
>
> T21JYM_20200126T134201_B02_10m_double,T21JYM_20200126T134201_B03_10m_double,T21JYM_20200126T134201_B04_10m_double,T21JYM_20200126T134201_B08_10m_double,T21JYM_20200126T134201_B8A_20m_double,T21JYM_20200126T134201_B11_20m_double,T21JYM_20200126T134201_B12_20m_double
> --- All bands have been rescaled ---
> --- Start computing maximum values of bands ---
> T21JYM_20200126T134201_B02_10m_double
> T21JYM_20200126T134201_B03_10m_double
> T21JYM_20200126T134201_B04_10m_double
> T21JYM_20200126T134201_B08_10m_double
> T21JYM_20200126T134201_B8A_20m_double
> T21JYM_20200126T134201_B11_20m_double
> T21JYM_20200126T134201_B12_20m_double
> --- Computed maximum value: dict_values([0.9352, 1.168, 1.1056, 1.0704,
> 0.7023, 0.9381, 1.3597]) ---
> --- Statistics have been computed! ---
> --- Start clouds detection procedure ---
> --- Computing cloud mask... ---
>  100%
> --- Converting raster cloud mask into vector map ---
> Extracting areas...
>  100%
> Writing areas...
>  100%
> Building topology for vector map <cloud_v_5922 at carla>...
> Registering primitives...
> Building areas...
>  100%
> Attaching islands...
>  100%
> Attaching centroids...
>  100%
> r.to.vect complete.
> --- Cleaning geometries ---
> [...]
> --- Start removing misclassification from the shadow mask ---
> --- Data preparation... ---
> WARNING: Values in column <cat> will be overwritten
> WARNING: Number of centroids exceeds number of areas: 24 > 22
> WARNING: Number of duplicate centroids: 2
> WARNING: Values in column <cat> will be overwritten
> WARNING: Values in column <cat> will be overwritten
> --- Reading mean sun zenith and azimuth from metadata file to compute
> clouds shift ---
> --- the mean sun Zenith is: 28.732 deg ---
> --- the mean sun Azimuth is: 82.028 deg ---
> --- Start computing the east and north clouds shift at steps of 100m of
> clouds height---
> Traceback (most recent call last):
>   File "/home/veroandreo/.grass7/addons/scripts/i.sentinel.mask", line
> 638, in <module>
>     main()
>   File "/home/veroandreo/.grass7/addons/scripts/i.sentinel.mask", line
> 561, in main
>     quiet=True, stderr=subprocess.DEVNULL)
>   File
> "/home/veroandreo/software/grass79-dev/dist.x86_64-pc-linux-gnu/etc/python/grass/script/core.py",
> line 499, in run_command
>     return handle_errors(returncode, result=None, args=args, kwargs=kwargs)
>   File
> "/home/veroandreo/software/grass79-dev/dist.x86_64-pc-linux-gnu/etc/python/grass/script/core.py",
> line 392, in handle_errors
>     returncode=returncode)
> grass.exceptions.CalledModuleError: Module run v.overlay v.overlay --o --q
> ainput=addcat_5922 binput=cl_shift_5922 operator=and output=overlay_5922
> ended with error
> Process ended with non-zero return code 1. See errors in the (error)
> output.
>
> Where is this error output to inspect?
>
> This is the UUID in case you want to test:
> 541021f8-63f5-4e9d-ba28-425c4c8451df
>
> Thanks much in advance
> Vero
>
> ps: I use grass-dev with python 3.7.4 (because of the ctypes issue)
>
>
>
> El mié., 29 ene. 2020 a las 14:53, Veronica Andreo (<veroandreo at gmail.com>)
> escribió:
>
> Hi Robi,
>
> My bad, region was wrong... anyway, trying to replicate from the beginning
> I see that i.sentinel.import with -r is importing empty maps. However, the
> jp2 files have data. I can visualize them in QGIS. I'll report in a
> different thread. Once I can figure it out and reach this point again, I'll
> report back to you. Thanks for the clarification!
>
> Cheers
> Vero
>
> El mié., 29 ene. 2020 a las 9:08, roberta fagandini (<
> robifagandini at hotmail.it>) escribió:
>
> Hi Vero,
> i.sentinel.mask doesn't retrieve the maximum from the xml file but
> computes the value from the image using r.univar. The error could be due to
> the computational region..has the region been set correctly?
> Can you send the MTD_TL.xml file? I can download the image and make some
> tests.
>
> Roberta
>
> ------------------------------
> *Da:* grass-user <grass-user-bounces at lists.osgeo.org> per conto di
> Veronica Andreo <veroandreo at gmail.com>
> *Inviato:* martedì 28 gennaio 2020 19:57
> *A:* grass-user <grass-user at lists.osgeo.org>
> *Oggetto:* [GRASS-user] error in i.sentinel.mask: nan values in xml file
>
> Hello
>
> I try to use i.sentinel.mask but I get the following error:
>
> i.sentinel.mask blue=$blue green=$green red=$red swir11=$swir11 nir=$nir
> swir12=$swir12 nir8a=$nir8a cloud_mask=cloud shadow_mask=shadow
> mtd_file=`find $out_dir -name 'MTD_TL.xml'`
>
> ADVERTENCIA: All subsequent operations will be limited to the current
>              computational region
> ADVERTENCIA: Any rescale factor has been applied
> --- Start computing maximum values of bands ---
> T21JYM_20200126T134201_B02_10m
> T21JYM_20200126T134201_B03_10m
> T21JYM_20200126T134201_B04_10m
> T21JYM_20200126T134201_B08_10m
> T21JYM_20200126T134201_B8A_20m
> T21JYM_20200126T134201_B11_20m
> T21JYM_20200126T134201_B12_20m
> --- Computed maximum value: dict_values([nan, nan, nan, nan, nan, nan,
> nan]) ---
> --- Statistics have been computed! ---
> --- Start clouds detection procedure ---
> --- Computing cloud mask... ---
> Mapa no válido <nan>
> Parse error
> ERROR: error al parsear
> ERROR: An error occurred while running r.mapcalc with expression:
>        cloud_def82115 = if((((T21JYM_20200126T134201_B02_10m > (0.08*nan))
>        && (T21JYM_20200126T134201_B03_10m > (0.08*nan)) &&
>        (T21JYM_20200126T134201_B04_10m > (0.08*nan))) == 1) &&
>        (((T21JYM_20200126T134201_B04_10m < ((0.08*nan)*1.5)) &&
>        (T21JYM_20200126T134201_B04_10m >
>        T21JYM_20200126T134201_B12_20m*1.3)) == 0) &&
>        (((T21JYM_20200126T134201_B11_20m < (0.1*nan)) &&
>        (T21JYM_20200126T134201_B12_20m < (0.1*nan))) == 0) &&
>        ((if(T21JYM_20200126T134201_B8A_20m ==
>        max(T21JYM_20200126T134201_B8A_20m, 2 *
>        T21JYM_20200126T134201_B02_10m, 2 * T21JYM_20200126T134201_B03_10m,
>        2 * T21JYM_20200126T134201_B04_10m))) == 0) &&
>        ((T21JYM_20200126T134201_B02_10m > 0.2) == 1), 0, null())
>
> I checked the xml file and there are a lot of nan's yes, but there are
> also real values. Why doesn't the max function in this part:
> https://github.com/OSGeo/grass-addons/blob/master/grass7/imagery/i.sentinel/i.sentinel.mask/i.sentinel.mask.py#L321
> recognize them? Isn't there a way to avoid nan while computing max?
>
> Thanks much in advance
> Vero
>
> ps: I can provide the xml if needed.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-user/attachments/20200130/55960432/attachment.html>