<div dir="ltr">Moritz, I don't know why but today I have some trouble with your email, I receive them after some hours...anyway I read the last from the dev digest.<div><br></div><div>Regarding AERONET data, it's not possible to completely automate the procedure (search for the proper file, download, unzip, read and so on..), I had already verified and the AERONET website does not provide any API or something that allows connecting from GRASS.</div><div>However, I think I can add the interpolation suggested by Markus (thank you, Markus, for the link) so regarding the AOT value, users can choose to:</div><div><ul><li>provide their own value<br></li><li>read the "nearest" value (timestamp) from an AERONET file (this part of the script is almost ready) and apply the interpolation (using the closest available wavelength)</li></ul><div>Roberta</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">2018-06-28 15:26 GMT+02:00 Roberta Fagandini <span dir="ltr"><<a href="mailto:robifagandini@gmail.com" target="_blank">robifagandini@gmail.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">2018-06-28 14:46 GMT+02:00 Markus Metz <span dir="ltr"><<a href="mailto:markus.metz.giswork@gmail.com" target="_blank">markus.metz.giswork@gmail.com</a><wbr>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div class="m_3773683690732009653gmail-m_6807267012767800132m_134606507179174471gmail-m_-4570796769343309468h5"><br><br>On Thu, Jun 28, 2018 at 12:17 PM, Roberta Fagandini <<a href="mailto:robifagandini@gmail.com" target="_blank">robifagandini@gmail.com</a>> wrote:<br>><br>><br>><br>> 2018-06-28 10:26 GMT+02:00 Moritz Lennert <<a href="mailto:mlennert@club.worldonline.be" target="_blank">mlennert@club.worldonline.be</a>><wbr>:<br>>><br>>> On 27/06/18 15:40, Roberto Marzocchi wrote:<br>>>><br>>>><br>>>><br>>>> Il giorno mer 27 giu 2018 alle ore 15:25 Markus Metz <<a href="mailto:markus.metz.giswork@gmail.com" target="_blank">markus.metz.giswork@gmail.com</a> <mailto:<a href="mailto:markus.metz.giswork@gmail.com" target="_blank">markus.metz.giswork@gm<wbr>ail.com</a>>> ha scritto:<br>>>><br>>>><br>>>><br>>>> On Wed, Jun 27, 2018 at 3:06 PM, Roberta Fagandini<br>>>> <<a href="mailto:robifagandini@gmail.com" target="_blank">robifagandini@gmail.com</a> <mailto:<a href="mailto:robifagandini@gmail.com" target="_blank">robifagandini@gmail.co<wbr>m</a>>> wrote:<br>>>> ><br>>>> ><br>>>> ><br>>>> > 2018-06-27 14:24 GMT+02:00 Markus Metz<br>>>> <<a href="mailto:markus.metz.giswork@gmail.com" target="_blank">markus.metz.giswork@gmail.com</a> <mailto:<a href="mailto:markus.metz.giswork@gmail.com" target="_blank">markus.metz.giswork@gm<wbr>ail.com</a>>>:<br>>>> >><br>>>> >><br>>>> >><br>>>> >> On Wed, Jun 27, 2018 at 2:16 PM, Roberta Fagandini<br>>>> <<a href="mailto:robifagandini@gmail.com" target="_blank">robifagandini@gmail.com</a> <mailto:<a href="mailto:robifagandini@gmail.com" target="_blank">robifagandini@gmail.co<wbr>m</a>>> wrote:<br>>>> >> ><br>>>> >> > I'm trying to create an automatic procedure to retrieve all<br>>>> the input parameters for the control file of i.atcorr.<br>>>> >> > I need py6S because I want to automatically read and compute<br>>>> AOT at 550nm from an AERONET file.<br>>>> >><br>>>> >> With AOT you mean aerosol optical thickness right? This is user<br>>>> input to i.atcorr. There are various sources for AOT at 550nm, e.g.<br>>>> MODIS.<br>>>> ><br>>>> ><br>>>> > Right!<br>>>> ><br>>>> >><br>>>> >><br>>>> >><br>>>> >><br>>>> >> Anyway, it is probably much easier to to write your own routine<br>>>> to read the AERONET file and get the corresponding formula for AOT<br>>>> than to fight with 51821 lines of Fortran source code that is<br>>>> numerically unstable. Maybe I can help with getting AOT at 550nm<br>>>> from an AERONET file avoiding the Fortran version of 6S.<br>>>> ><br>>>> ><br>>>> > Ok but I'm not able to create this kind of routine on my own. If<br>>>> someone, more expert than me, can help me in translating the 6S/Py6S<br>>>> functions I can go on in this way otherwise for me it's easier to<br>>>> use the fortran source code.<br>>>><br>>>> Apparently the AERONET data are simple CSV files. You can import<br>>>> them with <a href="http://r.in.xyz" target="_blank">r.in.xyz</a> <<a href="http://r.in.xyz" target="_blank">http://r.in.xyz</a>> or v.in.ascii.<br>>>><br>>>> Still, I think it is up to the user to decide on the source of AOT<br>>>> values. AERONET is one of many possible sources.<br>>>><br>>>><br>>>> Ok but I know that Roberta want try to simplify the procedure for atmospheric correction. In particular she would like to add an option (not the only one) to read automatically the AOT at the wave length of 550 nm from the AERONET file.<br>>>><br>>>> The steps to do this IMHO are:<br>>>><br>>>> 1) read the file (using GRASS or maybe better Python)<br>>>><br>>>> 2) a formula to calculate the AOT550<br>>>><br>>>> Unfortunately I have no idea about the 2nd point :-(<br>>><br>>><br>>> I'm not sure either, what exactly the necessary calculations are.<br>>><br>>> Superficially reading through the relevant class in Py6S [1], I understasnd that it does two things:<br>>><br>>> 1) Get the value of AOT550<br>>> 2) Create a continuous profile by interpolating values<br>>><br>>> If all you need is 1), then all they do (see _get_aot() method at the end) is to read all the AOT_* columns and chose the value of the line with the timestamp closest to the one asked for and the wavelength closest to 550. <br>>><br>>><br>>> As the code is GPL you could probably just extract the necessary parts and integrate them into your code. The only thing I'm not to sure about is their use of pandas which creates yet again another dependency which I don't really think necessary and desirable.<br>><br>><br>> Yesterday I studied the library better and I understood that it only extracts the available AOT values closest to the wavelength 550 for the specified timestamp. No calculation is made so I'm trying to write my own routine avoiding any dependency.<br>> <br>>><br>>> <br>>><br>>> It should be enough to extract from the file the date and time (or maybe easier to work with: year and the Julian day) and all AOT_* columns, parse the AOT_* columns for their wavelength value and then identify the value which has closes time and wavelength to the time requested and 550nm.<br>><br>><br><div>> That's exactly what I want to do and I'm working on it!</div><div><br></div></div></div><div>open the file with your preferred spreadsheet program and copy the corresponding value;-)<br></div><span>> <br>>><br>>><br>>> Moritz<br>><br>><br>> I know that AERONET is not the only source and obviously in my module there will be both an option for providing manually the value, as Markus and Stefan suggested, and an option to extract the AOT from an AERONET file. I'm not an expert in MODIS data so if someone wants to give suggestions about how to retrieve the AOT from these data, I will definitely add this option too.<br></span></div></blockquote><div><br></div></div></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">ok..now I'm a bit confused..</span><br></div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><span><div><br></div></span><div>about your module: it is concerned with Sentinel-2 cloud and shadow detection. As a general rule, a module should do one thing, and should do it right. In this sense, importing AERONET data should be a separate module, and your module should do Sentinel-2 cloud and shadow detection, nothing else, this is already complex enough.</div></div></blockquote><div><br></div></span><div>the module i.sentinel.mask does only the cloud and shadow detection.</div><span class=""><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Regarding a new module e.g. v.in.aeronet:</div><div>It should interpolate AOT at 550nm considering the closest wavelengths with valid data. Unfortunately most of the AERONET data are nodata. Interpolation can be a simple linear interpolation, see</div><div><a href="https://aeronet.gsfc.nasa.gov/new_web/V2/climo_new/explain/explain_new.html" target="_blank">https://aeronet.gsfc.nasa.gov/<wbr>new_web/V2/climo_new/explain/e<wbr>xplain_new.html</a></div><div><br></div><div>It should perform spatial interpolation because the stations are scattered with large distances in between. Spatial interpolation can be done with any of the v.surf.* modules.</div><div><br></div><div>It should offer temporal interpolation in case there are no data for the required date. This can be done with the temporal GIS framework or with r.series.lwr.</div></div></blockquote><div><br></div></span><div>Regarding AERONET, this is only a small part of another possible module/script. My intention is to create an <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">iterative procedure that executes i.atcorr for all bands of the input image changing accordingly the requested input parameters,</span> this module/script should do:</div><div><ul><li>read input bands from metadata file,<br></li><li>write the control file retrieving as many parameters as possible from the metadata<br></li><li>add other parameters provided by users to the control file</li><li>change the control file according to the input band<br></li><li>run i.atcorr<br></li></ul><div>Reading the AOT from an AERONET file, it is only one possible option to further automate the module/script. Obviously users can specify manually the AOT value and they can retrieve it from their favourite source.</div></div><span class=""><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Regarding MODIS data, r.in.gdal can import MODIS data, and there is r.modis<br></div><div><br></div><div>Putting it all together:</div><div>that would be a wrapper script that helps to</div><div>1) import visibility as a map or estimate a fixed visibility value<br></div><div>2) perform atmospheric correction with i.atcorr</div><div>3) detect clouds and shadows</div></div></blockquote><div><br></div></span><div>My idea was to create a module (e.g. i.sentinel.preproc) that wraps the dowload and import phase (i.sentinel.download and i.sentinel.import), the atmosperic correction using i.atcorr (and the iterative procedure that I'm trying to implement) and the cloud and shadow detection procedure. </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Markus M<br></div><span><div><br></div><div><br></div></span></div></blockquote><span class="HOEnZb"><font color="#888888"><div>Roberta </div></font></span></div><br></div></div>
</blockquote></div><br></div>