[GRASS-dev] ​GSoC 2018 report week 03 - GRASS GIS module for Sentinel-2 cloud and shadow detection

Roberta Fagandini robifagandini at gmail.com
Mon Jun 4 07:14:16 PDT 2018

Hi all!

Thank you, Moritz and Nikos, for your suggestions!

2018-06-04 11:54 GMT+02:00 Nikos Alexandris <nik at nikosalexandris.net>:

> Ciao Roberta,
> great work!
> Moritz got me faster, as I intended to chime in, with similar
> suggestions.
> So +1 for:
> - a "real" module, as Moritz described it

Done! I added the folder to my github repository but I have some trouble
compiling the module with g.extension:

g.extension extension=i.sentinel.mask operation=add url=

error message:
g.extension extension=i.sentinel.mask operation=add url=
Fetching <i.sentinel.mask> from <
https://github.com/RobiFag/GRASS_clouds_and_shadows/archive/master.zip> (be
make[1]: *** No rule to make target `/tmp/grass7-roberta-701
4/tmpe7QEso/i.sentinel.mask/scripts/i.sentinel.mask', needed
by `script'. Stop.
/bin/sh: 1: cannot create /usr/lib/grass74/error.log:
Permission denied
make: *** [i.sentinel.mask] Error 2
ERROR: Compilation failed, sorry. Please check above error messages.

any suggestion?

> - for using the metadata file, and parsing it -- my experience with
>  Landsat and other high-resolution satellite products, showed it's best to
>  trust the individual metadata that accompany an acquisition.
>   Parameters may be very specific to one single acquisition.
>  Also, a module that is designed to be "dynamic" is more
>  future-proof and easy to maintain and update.
>   For example, reading band names from the meta-data, as opposed to more
>  "static" designs, i.e. expecting specific hardcoded options, such as
>  band names, or their order.

I have already thought about parsing the metadatafile to retrieve the input
bands but at the moment the module requires atmospherically corrected bands
and if users perform the atmospheric correction on their own I don't have
any control on naming. Moreover, the module needs to know which raster map
corresponds to the blue band, to the green band and so on in order to apply
the rules for clouds and shadows detection. Therefore I need users specify
each band but maybe I can manage this issue requiring a specific suffix in
the bands' names (e.g. blabla_blue, blabla_green, etc.). In this way, users
have to provide only the prefix (blabla) while the module search for all
raster maps with the specified prefix and at the same time it is able to
recognize each band.

Anyway, the aim of the next coding period is to create a python script that
wraps in a single GRASS module the download and import phase
(i.sentinel.download and i.sentinel.import), the atmospheric correction
using i.atcorr and the cloud and shadow detection procedure. In particular,
for what concerns the atmospheric correction, I'd like to implement an
iterative procedure that executes i.atcorr for all bands of the input image
changing accordingly the requested input parameters. This part should
simplify the management of input maps.

> - PEP8 rules and keeping module options in separate lines to increase
> legibility
> In line with the suggestions on style, I would stress out to fully
> spell out the names of functions, options, descriptions of flags,
> iterables and iterators.  It will make it easier for the next reader of
> your code.

Yes, you are both right!! I have to clean up the code from the style point
of view and it will be one of the tasks of this week and especially of the
last days before the GSoC Evaluation.

> Finally, it's a good idea to write even mini-tests, while you are
> progressing, for the small functions that eventually develop to keep the
> full scrip compact, functional and more legible.
> Writing a test, for your module, in the end, will be much easier.

Sorry Nikos but I don't really understand what you mean by
"mini-tests"..are you referring to the examples in the manual page?

> I am sure you know all this. Yet, discussing it is a good thing in
> my view.

Discussing it is very useful especially for me so please do not take
anything for granted and keep sending me your feedback!

> Keep up the good work, Nikos


> * Moritz Lennert <mlennert at club.worldonline.be> [2018-06-04 11:03:41
> +0200]:
> Hi Roberta !
>> On 03/06/18 14:38, Roberta Fagandini wrote:
>>> Hi all!
>>> I'm Roberta Fagandini and I'm working on my GSoC project, a GRASS GIS
>>> module for Sentinel-2 cloud and shadow detection.
>>> This is my report for the third week of coding.
>>> *1) What did I complete this week?*
>>>  * Implemented some changes from dev feedback (e.g. r.univar instead of
>>>    r.stats.zonal)
>>>  * Tested the modified python script and fixed bugs
>>>  * Prepared the python script in order to start implementing the GUI
>>>  * Started implementing the GUI
>>>  * Made some changes to the code depending on the GUI requirements (add
>>>    controls and check on input, output and temporary file, etc.)
>>>  * Cleaned up the whole code
>>>  * Tested the GUI and fixed bugs
>>>  * Frequently added the basic version of the GUI to my
>>>    GitHub repository [0]
>>>  * Shared progress with the community
>> I'm sorry, I haven't had the time to test your module, yet. A few
>> recommendation, though, based on a quick read of the code. I assume that
>> the relevant file is i.sentinel.mask.
>> - I would recommend to immediately create a real complete module that can
>> be installed with g.extension. This means creating a directory with the
>> module as .py file, a .html manual page file, but which could be empty at
>> the beginning, and a Makefile (see existing scripts in the core code or in
>> addons for examples). Being able to install everything directly from github
>> with a simple g.extension call makes testing much easier.
>> - I see that all the input bands are marked as required. Are they all
>> necessary to the calculations ? If yes, aren't all these bands always from
>> the same acquisition ? If yes, then maybe we can avoid parameter inflation
>> by either just asking for a prefix and then getting all bands from that
>> prefix (following standard sentinel naming), or provide the MTD file and
>> read the band names from there, or you could ask for a group name and it
>> would be up to the user to create the group before launching the module.
>> - A small idea in terms of coding style: I personally think that code
>> should be as easily understandable as possible. For example: in the
>> formulas you use the f_bands[] list. Maybe you could make this into a
>> dictionary and so instead of reading f_bands[0], we would read
>> f_bands['blue'] making the formulas easier to follow.
>> - Please follow PEP8 style guide (see [1]). Amongst others, this means
>> limiting lines to 79 characters. So:
>>         - The general style used for gscript.*_command calls in Python
>> scripts is
>>         gscript.run_command('v.dissolve',
>>                             input=tmp["centroid"],
>>                             column='value',
>>                             output=tmp["dissolve"],
>>                             overwrite=True,
>>                             quiet=True)
>>         i.e. each parameter on its own line. This make it easier to read
>>         and lines shorter.
>>         - The same for r.mapcalc calls: if necessary you can construct
>>         the individual rules over more than one line.
>> I'll try to find some time this week to test the module and maybe come
>> back with some more feedback on the coding.
>> Great job so far, though !
>> Moritz
>> [1] https://trac.osgeo.org/grass/wiki/Submitting/Python#Style
>> _______________________________________________
>> grass-dev mailing list
>> grass-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/grass-dev
> --
> Nikos Alexandris | Remote Sensing & Geomatics
> GPG Key Fingerprint 6F9D4506F3CA28380974D31A9053534B693C4FB3
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20180604/d2b1de40/attachment.html>

More information about the grass-dev mailing list