[GRASS-dev] GSoC 2018 report week 03 - GRASS GIS module for Sentinel-2 cloud and shadow detection
robifagandini at gmail.com
Mon Jun 4 09:42:47 PDT 2018
2018-06-04 17:23 GMT+02:00 Nikos Alexandris <nik at nikosalexandris.net>:
> * Roberta Fagandini <robifagandini at gmail.com> [2018-06-04 16:14:16 +0200]:
> Hi all!
>> Thank you, Moritz and Nikos, for your suggestions!
>> 2018-06-04 11:54 GMT+02:00 Nikos Alexandris <nik at nikosalexandris.net>:
>> Ciao Roberta,
>>> great work!
>>> Moritz got me faster, as I intended to chime in, with similar
>>> So +1 for:
>>> - a "real" module, as Moritz described it
>> Done! I added the folder to my github repository but I have some trouble
>> compiling the module with g.extension:
>> g.extension extension=i.sentinel.mask operation=add url=
>> error message:
>> g.extension extension=i.sentinel.mask operation=add url=
>> Fetching <i.sentinel.mask> from <
>> make: *** No rule to make target `/tmp/grass7-roberta-701
>> 4/tmpe7QEso/i.sentinel.mask/scripts/i.sentinel.mask', needed
>> by `script'. Stop.
>> /bin/sh: 1: cannot create /usr/lib/grass74/error.log:
>> Permission denied
>> make: *** [i.sentinel.mask] Error 2
>> ERROR: Compilation failed, sorry. Please check above error messages.
>> any suggestion?
> Maybe irrelevant, but perhaps it is best to name the script as
I named the file as you suggested but I still have some problems, now the
error message is:
Fetching <i.sentinel.mask> from <
make: *** No rule to make target `install'. Stop.
WARNING: Installation failed, sorry. Please check above error messages.
> - for using the metadata file, and parsing it -- my experience with
>>> Landsat and other high-resolution satellite products, showed it's best
>>> trust the individual metadata that accompany an acquisition.
>>> Parameters may be very specific to one single acquisition.
>>> Also, a module that is designed to be "dynamic" is more
>>> future-proof and easy to maintain and update.
>>> For example, reading band names from the meta-data, as opposed to more
>>> "static" designs, i.e. expecting specific hardcoded options, such as
>>> band names, or their order.
>> I have already thought about parsing the metadatafile to retrieve the
>> bands but at the moment the module requires atmospherically corrected
>> and if users perform the atmospheric correction on their own I don't have
>> any control on naming. Moreover, the module needs to know which raster map
>> corresponds to the blue band, to the green band and so on in order to
>> the rules for clouds and shadows detection. Therefore I need users specify
>> each band but maybe I can manage this issue requiring a specific suffix in
>> the bands' names (e.g. blabla_blue, blabla_green, etc.). In this way,
>> have to provide only the prefix (blabla) while the module search for all
>> raster maps with the specified prefix and at the same time it is able to
>> recognize each band.
>> Anyway, the aim of the next coding period is to create a python script
>> wraps in a single GRASS module the download and import phase
>> (i.sentinel.download and i.sentinel.import), the atmospheric correction
>> using i.atcorr and the cloud and shadow detection procedure. In
>> for what concerns the atmospheric correction, I'd like to implement an
>> iterative procedure that executes i.atcorr for all bands of the input
>> changing accordingly the requested input parameters. This part should
>> simplify the management of input maps.
> One thing here: massive processing workflows like to have simple
> processing building blocks. One Input -> one Process -> one Output. In
> "your" case:
> - Input: name of/request one band and associated
> metadata, and optionally other ancillary data.
> - Process: download, import, removal of atmospheric effects to estimate
> surface reflectance
> - Output: surface reflectance ("corrected" for atmospheric effects) of
> the requested input band
> This building block can run in one processing units/node. Making use of
> processing units/nodes, this block can run in parallel for different
> input bands.
> I suggest to build the module in a way that it runs for one band or as many
> bands as the user requires for his workflow. I did this in
> I don't suggest this script as an example. Yet I think the concept "for
> one or many" is useful and wanted.
Thank you! This is very interesting..
I will check your module and I will certainly take it into account even if
all the requested input bands are mandatory otherwise the clouds and
shadows detection procedures  will return wrong results.
> As a sidenote:
> I think that we don't use the term "correction"
> correctly. An image acquired by a satellite-borne sensor is not "wrong"
> to need a correction. Rather, we try to retouch the digital form of the
> acquisition, in order to estimate the energy that was actually reflected
> by the surface, before the signal travels through the atmosphere.
> I studied remote sensing with Thomas Lillesand's; Ralph W.
> Kiefer's book "Remote Sensing and Image Interpretation". The book
> "correctly" touches this very question.
> Maybe it's only a perspective, and others have another say on this.
> - PEP8 rules and keeping module options in separate lines to increase
>>> In line with the suggestions on style, I would stress out to fully
>>> spell out the names of functions, options, descriptions of flags,
>>> iterables and iterators. It will make it easier for the next reader of
>>> your code.
> Yes, you are both right!! I have to clean up the code from the style point
>> of view and it will be one of the tasks of this week and especially of the
>> last days before the GSoC Evaluation.
> Finally, it's a good idea to write even mini-tests, while you are
>>> progressing, for the small functions that eventually develop to keep the
>>> full scrip compact, functional and more legible.
>>> Writing a test, for your module, in the end, will be much easier.
> Sorry Nikos but I don't really understand what you mean by
>> "mini-tests"..are you referring to the examples in the manual page?
> For a function f(x) that takes x as an input and gives you y as output,
> test, for example, whether the output value 'y' is within the expected
> range. Test your single functions, inside the script. Use assertions, for
> example, etc. If you have 10 functions in your script, and have written
> a test for each, you will save your time later on when you will revisit
> the code to debug or modify. Spend as much time as you can in testing.
Ok, it's clearer to me now! I will certainly do it!
> Again, this is what I try to stick to. And, lastly, I am not a core
> developer, nor a mentor. I am, however, interested in seeing this module
> coming alive. So, apologies to the mentors that I am chimingi in.
Thanks, I hope you will continue to follow the development of the
project/module. As for me also your suggestions and feedback are very
> [rest deleted]
> Grazie, Nikos
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the grass-dev