[GRASS-SVN] r72165 - in grass-addons/grass7/raster: . r.sentinel r.sentinel/r.sentinel.download r.sentinel/r.sentinel.import

svn_grass at osgeo.org svn_grass at osgeo.org
Fri Jan 26 14:00:45 PST 2018


Author: martinl
Date: 2018-01-26 14:00:45 -0800 (Fri, 26 Jan 2018)
New Revision: 72165

Added:
   grass-addons/grass7/raster/r.sentinel/
   grass-addons/grass7/raster/r.sentinel/Makefile
   grass-addons/grass7/raster/r.sentinel/r.sentinel.download/
   grass-addons/grass7/raster/r.sentinel/r.sentinel.download/Makefile
   grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.html
   grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.py
   grass-addons/grass7/raster/r.sentinel/r.sentinel.html
   grass-addons/grass7/raster/r.sentinel/r.sentinel.import/
   grass-addons/grass7/raster/r.sentinel/r.sentinel.import/Makefile
   grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.html
   grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.py
   grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r_sentinel_import_band_4_clouds.png
Modified:
   grass-addons/grass7/raster/Makefile
Log:
r.sentinel: new toolset for downloading and importing Sentinel products

Modified: grass-addons/grass7/raster/Makefile
===================================================================
--- grass-addons/grass7/raster/Makefile	2018-01-26 11:55:23 UTC (rev 72164)
+++ grass-addons/grass7/raster/Makefile	2018-01-26 22:00:45 UTC (rev 72165)
@@ -95,6 +95,7 @@
 	r.sample.category \
 	r.scatterplot \
 	r.seasons \
+	r.sentinel \
 	r.series.decompose \
 	r.series.diversity \
 	r.series.lwr \

Added: grass-addons/grass7/raster/r.sentinel/Makefile
===================================================================
--- grass-addons/grass7/raster/r.sentinel/Makefile	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/Makefile	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,14 @@
+MODULE_TOPDIR = ../..
+
+PGM = r.sentinel
+
+SUBDIRS = \
+	r.sentinel.download \
+        r.sentinel.import
+
+include $(MODULE_TOPDIR)/include/Make/Dir.make
+
+default: parsubdirs
+
+install: installsubdirs
+	$(INSTALL_DATA) $(PGM).html $(INST_DIR)/docs/html/


Property changes on: grass-addons/grass7/raster/r.sentinel/Makefile
___________________________________________________________________
Added: svn:mime-type
   + text/x-makefile
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.download/Makefile
===================================================================
--- grass-addons/grass7/raster/r.sentinel/r.sentinel.download/Makefile	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/r.sentinel.download/Makefile	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,7 @@
+MODULE_TOPDIR = ../..
+
+PGM = r.sentinel.download
+
+include $(MODULE_TOPDIR)/include/Make/Script.make
+
+default: script


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.download/Makefile
___________________________________________________________________
Added: svn:mime-type
   + text/x-makefile
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.html
===================================================================
--- grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.html	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.html	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,106 @@
+<em>r.sentinel.download</em> allows downloading Sentinel products
+from <a href="https://scihub.copernicus.eu/">Copernicus Open Access
+Hub</a>.
+
+<p>
+To connect Copernicus Open Access Hub a <b>user</b>
+and <b>password</b> are required,
+see <a href="https://scihub.copernicus.eu/dhus/#/self-registration">Register
+new account</a> page for signing up.
+
+<h2>NOTES</h2>
+
+<p>
+By default Sentinel products are sorted by <i>cloudcoverpercentage</i>
+and <i>ingestiondate</i> (see <b>sort</b> option). Only products which
+footprint intersects current computation region extent are
+filtered. The extent can be optionally defined also by
+vector <b>map</b>. Filtered products can be reduced by <b>limit</b>
+option.
+
+<p>
+Module searches for products in last 60 days, exact date range can be
+defined by <b>start</b> and <b>end</b> options.
+
+<p>
+Sentinel products can be also filtered by <b>producttype</b> or
+maximum <b>clouds</b> cover percentage.
+
+<h2>EXAMPLES</h2>
+
+<h3>List filtered products</h3>
+
+<p>
+Find S2MSI1C products in last 60 days covering current computation region extent.
+
+<div class="code"><pre>
+r.sentinel.download -l user=myusername password=mypassword
+
+1 Sentinel product(s) found
+ae1c33ec-aa33-4303-a525-9e6481709614 2017-12-08T10:23:59Z 18% S2MSI1C
+</pre></div>
+
+Find all S2MSI2Ap products in 2017.
+
+<div class="code"><pre>
+r.sentinel.download -l user=myusername password=mypassword producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31
+    
+7 Sentinel product(s) found
+e5df8b4f-a4c6-47bd-88f3-e16b7540cc7a 2017-05-27T10:20:31Z  2% S2MSI2Ap
+b62afeda-a28d-475c-8220-91e24fc368ff 2017-05-17T10:20:31Z  2% S2MSI2Ap
+9a6bc289-98e9-4489-84eb-1aac95aaa056 2017-08-15T10:20:21Z  3% S2MSI2Ap
+35c72002-78a0-45f8-b39d-66c2d7b4ad87 2017-10-14T10:20:21Z  6% S2MSI2Ap
+c0ae8085-c1bb-4a27-89f2-2138a0866586 2017-07-06T10:20:21Z 12% S2MSI2Ap
+433ebfbc-5144-42f8-97dc-b9f4f1eb7b5a 2017-11-03T10:22:01Z 12% S2MSI2Ap
+fc56a594-d9d8-4e93-8dec-af3a58b24080 2017-09-04T10:20:21Z 19% S2MSI2Ap
+</pre></div>
+
+Sort products by <b>ingestiondate</b>, limit cloud coverage to 5% per scene.
+
+<div class="code"><pre>
+r.sentinel.download -l user=myusername password=mypassword producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 sort=ingestiondate order=desc clouds=5
+
+3 Sentinel product(s) found
+9a6bc289-98e9-4489-84eb-1aac95aaa056 2017-08-15T10:20:21Z  3% S2MSI2Ap
+b62afeda-a28d-475c-8220-91e24fc368ff 2017-05-17T10:20:31Z  2% S2MSI2Ap
+e5df8b4f-a4c6-47bd-88f3-e16b7540cc7a 2017-05-27T10:20:31Z  2% S2MSI2Ap
+</pre></div>
+
+Create vector map with <b>footprints</b>.
+
+<div class="code"><pre>
+r.sentinel.download -l user=myusername password=mypassword producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 footprints=ft
+</pre></div>
+
+<h3>Download Sentinel products</h3>
+
+Download first (<b>limit=1</b>) found S2MSI2Ap product into <i>data</i> directory.
+
+<div class="code"><pre>
+r.sentinel.download -l user=myusername password=mypassword producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 limit=1 output=data
+</pre></div>
+
+Such downloaded data can be easily imported into GRASS
+using <em><a href="r.sentinel.import.html">r.sentinel.import</a></em>
+module.
+
+<h2>REQUIREMENTS</h2>
+
+<ul>
+  <li><a href="https://pypi.python.org/pypi/sentinelsat">Sentinelsat library</a></li>
+  <li><a href="https://pypi.python.org/pypi/pandas">Pandas library</a></li>
+</ul>
+
+<h2>SEE ALSO</h2>
+
+<em>
+<a href="r.sentinel.import.html">r.sentinel.import</a>,
+<a href="v.import.html">v.import</a>
+</em>
+  
+<h2>AUTHOR</h2>
+
+Martin
+Landa, <a href="http://geomatics.fsv.cvut.cz/research/geoforall/">GeoForAll
+Lab</a>, CTU in Prague, Czech Republic with support
+of <a href="http://opengeolabs.cz/en/home/">OpenGeoLabs</a> company


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.html
___________________________________________________________________
Added: svn:mime-type
   + text/html
Added: svn:keywords
   + Author Date Id
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.py
===================================================================
--- grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.py	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.py	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,297 @@
+#!/usr/bin/env python2
+#
+############################################################################
+#
+# MODULE:      r.sentinel.download
+# AUTHOR(S):   Martin Landa
+# PURPOSE:     Downloads Sentinel data from  Copernicus Open Access Hub
+#              using sentinelsat library.
+# COPYRIGHT:   (C) 2018 by Martin Landa, and the GRASS development team
+#
+#              This program is free software under the GNU General Public
+#              License (>=v2). Read the file COPYING that comes with GRASS
+#              for details.
+#
+#############################################################################
+
+#%Module
+#% description: Downloads Sentinel data from Copernicus Open Access Hub using sentinelsat library.
+#% keyword: raster
+#% keyword: imagery
+#% keyword: sentinel
+#% keyword: download
+#%end
+#%option G_OPT_V_MAP
+#% label: Name of input vector map to define Area of Interest (AOI)
+#% description: If not given than current computational extent is used
+#% required: no
+#%end
+#%option
+#% key: clouds
+#% type: integer
+#% description: Maximum cloud cover percentage for Sentinel scene
+#% answer: 30
+#% required: no
+#%end
+#%option
+#% key: producttype
+#% type: string
+#% description: Sentinel product type to filter
+#% required: no
+#% options: SLC,GRD,OCN,S2MSI1C,S2MSI2Ap
+#% answer: S2MSI1C
+#%end
+#%option
+#% key: limit
+#% type: integer
+#% description: Limit number of Sentinel products
+#%end
+#%option
+#% key: start
+#% type: string
+#% description: Start date ('YYYY-MM-DD')
+#%end
+#%option
+#% key: end
+#% type: string
+#% description: End date ('YYYY-MM-DD')
+#%end
+#%option
+#% key: user
+#% type: string
+#% description: Username for connecting SciHub
+#% required: yes
+#%end
+#%option
+#% key: password
+#% type: string
+#% description: Password for connecting SciHub
+#% required: yes
+#%end
+#%option G_OPT_M_DIR
+#% key: output
+#% description: Name for output directory where to store downloaded Sentinel data
+#% required: no
+#%end
+#%option G_OPT_V_OUTPUT
+#% key: footprints
+#% description: Name for output vector map with footprints
+#% required: no
+#%end
+#%option
+#% key: sort
+#% description: Sort by values in given order
+#% multiple: yes
+#% options: ingestiondate,cloudcoverpercentage,footprint
+#% answer: cloudcoverpercentage,ingestiondate,footprint
+#%end
+#%option
+#% key: order
+#% description: Sort order (see sort parameter)
+#% options: asc,desc
+#% answer: asc
+#%end
+#%flag
+#% key: l
+#% description: List filtered products and exist
+#%end
+#%rules
+#% required: output,-l
+#%end
+
+import os
+import sys
+import logging
+import zipfile
+
+from collections import OrderedDict
+
+import grass.script as gs
+
+def get_aoi_box(vector=None):
+    args = {}
+    if vector:
+        args['vector'] = vector
+    info = gs.parse_command('g.region', flags='uplg', **args)
+
+    return 'POLYGON(({nw_lon} {nw_lat}, {ne_lon} {ne_lat}, {se_lon} {se_lat}, {sw_lon} {sw_lat}, {nw_lon} {nw_lat}))'.format(
+        nw_lat=info['nw_lat'], nw_lon=info['nw_long'], ne_lat=info['ne_lat'], ne_lon=info['ne_long'],
+        sw_lat=info['sw_lat'], sw_lon=info['sw_long'], se_lat=info['se_lat'], se_lon=info['se_long']
+    )
+
+class SentinelDownloader(object):
+    def __init__(self, user, password):
+        try:
+            from sentinelsat import SentinelAPI
+        except ImportError as e:
+            gs.fatal("Module requires sentinelsat library: {}".format(e))
+        try:
+            import pandas
+        except ImportError as e:
+            gs.fatal("Module requires pandas library: {}".format(e))
+
+        # init logger
+        root = logging.getLogger()
+        root.addHandler(logging.StreamHandler(
+            sys.stderr
+        ))
+
+        # connect SciHub via API
+        self._api = SentinelAPI(options['user'], options['password'],
+                                api_url='https://scihub.copernicus.eu/dhus'
+        )
+
+        self._products_df_sorted = None
+        
+    def filter(self, area,
+               clouds=None, producttype=None, limit=None,
+               start=None, end=None, sortby=[], asc=True):
+        args = {}
+        if clouds:
+            args['cloudcoverpercentage'] = (0, clouds)
+        if producttype:
+            args['producttype'] = producttype
+        if not start:
+            start = 'NOW-60DAYS'
+        else:
+            start = start.replace('-', '')
+        if not end:
+            end = 'NOW'
+        else:
+            end = end.replace('-', '')
+        products = self._api.query(
+            area=area,
+            platformname='Sentinel-2',
+            date=(start, end),
+            **args
+        )
+        products_df = self._api.to_dataframe(products)
+        if len(products_df) < 1:
+            gs.message('No product found')
+            return
+
+        # sort and limit to first sorted product
+        if sortby:
+            self._products_df_sorted = products_df.sort_values(
+                sortby,
+                ascending=[asc] * len(sortby)
+            )
+
+        if limit:
+            self._products_df_sorted = self._products_df_sorted.head(int(limit))
+
+        gs.message('{} Sentinel product(s) found'.format(len(self._products_df_sorted)))
+
+    def list(self):
+        if self._products_df_sorted is None:
+            return
+
+        for idx in range(len(self._products_df_sorted)):
+            print ('{0} {1} {2:2.0f}% {3}'.format(
+                self._products_df_sorted['uuid'][idx],
+                self._products_df_sorted['beginposition'][idx].strftime("%Y-%m-%dT%H:%M:%SZ"),
+                self._products_df_sorted['cloudcoverpercentage'][idx],
+                self._products_df_sorted['producttype'][idx],
+            ))
+
+    def download(self, output):
+        if self._products_df_sorted is None:
+            return
+
+        if not os.path.exists(output):
+            os.makedirs(output)
+        gs.message('Downloading data into <{}>...'.format(output))
+        for idx in range(len(self._products_df_sorted)):
+            gs.message('{} -> {}.SAFE'.format(
+                self._products_df_sorted['uuid'][idx],
+                os.path.join(output, self._products_df_sorted['identifier'][idx])
+            ))
+            # download
+            self._api.download(self._products_df_sorted['uuid'][idx], output)
+
+            # unzip
+            if os.path.exists(os.path.join(output, self._products_df_sorted['identifier'][idx])):
+                continue
+            filename = self._products_df_sorted['identifier'][idx] + '.zip'
+            with zipfile.ZipFile(os.path.join(output, filename), 'r') as zip_ref:
+                zip_ref.extractall(output)
+
+    def save_footprints(self, map_name):
+        if self._products_df_sorted is None:
+            return
+
+        try:
+            from osgeo import ogr, osr
+        except ImportError as e:
+            gs.fatal("Option <footprints> requires GDAL library: {}".format(e))
+
+        gs.message("Writing footprints into <{}>...".format(map_name))
+        driver = ogr.GetDriverByName("GPKG")
+        tmp_name = gs.tempfile() + '.gpkg'
+        data_source = driver.CreateDataSource(tmp_name)
+
+        srs = osr.SpatialReference()
+        srs.ImportFromEPSG(4326)
+
+        layer = data_source.CreateLayer(str(map_name), srs, ogr.wkbPolygon)
+
+        # attributes
+        attrs = OrderedDict([
+            ("uuid", ogr.OFTString),
+            ("ingestiondate", ogr.OFTString),
+            ("cloudcoverpercentage", ogr.OFTInteger),
+            ("producttype", ogr.OFTString)
+        ])
+        for key in attrs.keys():
+            field = ogr.FieldDefn(key, attrs[key])
+            layer.CreateField(field)
+
+        # features
+        for idx in range(len(self._products_df_sorted)):
+            wkt = self._products_df_sorted['footprint'][idx]
+            feature = ogr.Feature(layer.GetLayerDefn())
+            feature.SetGeometry(ogr.CreateGeometryFromWkt(wkt))
+            for key in attrs.keys():
+                if key == 'ingestiondate':
+                    value = self._products_df_sorted[key][idx].strftime("%Y-%m-%dT%H:%M:%SZ")
+                else:
+                    value = self._products_df_sorted[key][idx]
+                feature.SetField(key, value)
+            layer.CreateFeature(feature)
+            feature = None
+
+        data_source = None
+
+        gs.run_command('v.import', input=tmp_name, output=map_name,
+                       layer=map_name, quiet=True
+        )
+
+def main():
+    map_box = get_aoi_box(options['map'])
+
+    downloader = SentinelDownloader(options['user'], options['password'])
+
+    downloader.filter(area=map_box,
+                      clouds=options['clouds'],
+                      producttype=options['producttype'],
+                      limit=options['limit'],
+                      start=options['start'],
+                      end=options['end'],
+                      sortby=options['sort'].split(','),
+                      asc=options['order'] == 'asc'
+    )
+
+    if options['footprints']:
+        downloader.save_footprints(options['footprints'])
+
+    if flags['l']:
+        downloader.list()
+        return
+
+    downloader.download(options['output'])
+
+    return 0
+
+if __name__ == "__main__":
+    options, flags = gs.parser()
+    sys.exit(main())


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.download/r.sentinel.download.py
___________________________________________________________________
Added: svn:mime-type
   + text/x-python
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.html
===================================================================
--- grass-addons/grass7/raster/r.sentinel/r.sentinel.html	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/r.sentinel.html	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,49 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<html>
+<head>
+<title>GRASS GIS manual: r.sentinel</title>
+<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
+<link rel="stylesheet" href="grassdocs.css" type="text/css">
+</head>
+<body bgcolor="white">
+<div id="container">
+
+<a href="index.html"><img src="grass_logo.png" alt="GRASS logo"></a>
+<hr class="header">
+
+<h2>NAME</h2>
+
+<em><b>r.sentinel</b></em>  - Toolset for download and processing of Sentinel products.
+
+<h2>KEYWORDS</h2>
+<a href="raster.html">raster</a>, <a href="topic_import.html">import</a>, <a href="keywords.html#Sentinel">Sentinel</a>
+
+<!-- meta page description: Toolset for download and processing of Sentinel products -->
+<h2>DESCRIPTION</h2>
+
+<em>r.sentinel</em> toolset consists of two modules:
+
+<dl>
+  <dt><a href="r.sentinel.download.html">r.sentinel.download</a></dt>
+  <dd>downloads Sentinel products
+  from <a href="https://scihub.copernicus.eu/">Copernicus Open Access
+  Hub</a></dd>
+  <dt><a href="r.sentinel.import.html">r.sentinel.import</a></dt>
+  <dd>imports already downloaded Sentinel products into GRASS mapset</dd>
+</dl>
+
+<h2>REQUIREMENTS</h2>
+
+<ul>
+  <li><a href="https://pypi.python.org/pypi/sentinelsat">Sentinelsat library</a></li>
+  <li><a href="https://pypi.python.org/pypi/pandas">Pandas library</a></li>
+</ul>
+
+<h2>AUTHOR</h2>
+
+Martin
+Landa, <a href="http://geomatics.fsv.cvut.cz/research/geoforall/">GeoForAll
+Lab</a>, CTU in Prague, Czech Republic with support
+of <a href="http://opengeolabs.cz/en/home/">OpenGeoLabs</a> company
+
+<p><i>Last changed: $Date$</i>


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.html
___________________________________________________________________
Added: svn:mime-type
   + text/html
Added: svn:keywords
   + Author Date Id
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/Makefile
===================================================================
--- grass-addons/grass7/raster/r.sentinel/r.sentinel.import/Makefile	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/r.sentinel.import/Makefile	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,7 @@
+MODULE_TOPDIR = ../..
+
+PGM = r.sentinel.import
+
+include $(MODULE_TOPDIR)/include/Make/Script.make
+
+default: script


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/Makefile
___________________________________________________________________
Added: svn:mime-type
   + text/x-makefile
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.html
===================================================================
--- grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.html	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.html	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,70 @@
+<em>r.sentinel.import</em> module allows importing Sentinel products
+downloaded
+by <em><a href="r.sentinel.download.html">r.sentinel.download</a></em>
+module.
+
+<p>
+By default <em>r.sentinel.import</em> imports all Sentinel bands found
+in <b>input</b> directory
+by <em><a href="r.import.html">r.import</a></em>. Note that in the
+case that spatial reference system of input data differs from GRASS
+location, the input data are reprojected.
+
+<p>
+Optionally input data can be linked
+by <em><a href="r.external.html">r.external</a></em> when <b>-l</b> is
+given. Note that linking data requires that input data and GRASS
+location have the same spatial reference system.
+
+<p>
+Number of imported Sentinel bands can be optionally reduced
+by <b>pattern</b> option.
+  
+<h2>NOTES</h2>
+
+<p>
+If <b>-c</b> flag is given, than also cloud mask file is imported as
+vector map if available. Name of created vector map is determined from
+input Sentinel product.
+
+<h2>EXAMPLES</h2>
+
+Import all Sentinel bands found in <i>data</i> directory.
+
+<div class="code"><pre>
+r.sentinel.import input=data
+</pre></div>
+
+Limit import only to 4th and 8th bands.
+
+<div class="code"><pre>
+r.sentinel.import input=data pattern='B0[4|8]'
+</pre></div>
+
+Link data and import also cloud mask file.
+
+<div class="code"><pre>
+r.sentinel.import -l -c input=data
+</pre></div>
+
+<center>
+<img src="r_sentinel_import_band_4_clouds.png" width="600" height="356"><br>
+<i>Fig: Band 4 with imported cloud mask</i>
+</center>
+
+<h2>SEE ALSO</h2>
+
+<em>
+<a href="r.sentinel.download.html">r.sentinel.download</a>,
+<a href="r.import.html">r.import</a>,
+<a href="r.extenal.html">r.external</a>,
+<a href="v.import.html">v.import</a>
+</em>
+
+
+<h2>AUTHOR</h2>
+
+Martin
+Landa, <a href="http://geomatics.fsv.cvut.cz/research/geoforall/">GeoForAll
+Lab</a>, CTU in Prague, Czech Republic with support
+of <a href="http://opengeolabs.cz/en/home/">OpenGeoLabs</a> company


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.html
___________________________________________________________________
Added: svn:mime-type
   + text/html
Added: svn:keywords
   + Author Date Id
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.py
===================================================================
--- grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.py	                        (rev 0)
+++ grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.py	2018-01-26 22:00:45 UTC (rev 72165)
@@ -0,0 +1,125 @@
+#!/usr/bin/env python2
+#
+############################################################################
+#
+# MODULE:      r.sentinel.import
+# AUTHOR(S):   Martin Landa
+# PURPOSE:     Imports Sentinel data downloaded from Copernicus Open Access Hub
+#              using r.sentinel.download.
+# COPYRIGHT:   (C) 2018 by Martin Landa, and the GRASS development team
+#
+#              This program is free software under the GNU General Public
+#              License (>=v2). Read the file COPYING that comes with GRASS
+#              for details.
+#
+#############################################################################
+
+#%Module
+#% description: Imports Sentinel data downloaded from Copernicus Open Access Hub using r.sentinel.download.
+#% keyword: raster
+#% keyword: imagery
+#% keyword: sentinel
+#% keyword: import
+#%end
+#%option G_OPT_M_DIR
+#% key: input
+#% description: Name for input directory where downloaded Sentinel data lives
+#% required: yes
+#%end
+#%option
+#% key: pattern
+#% description: File name pattern to import
+#%end
+#%flag
+#% key: l
+#% description: Link raster data instead of importing
+#%end
+#%flag
+#% key: c
+#% description: Import cloud masks as vector maps
+#%end
+
+import os
+import sys
+import re
+
+import grass.script as gs
+from grass.exceptions import CalledModuleError
+
+class SentinelImporter(object):
+    def __init__(self, input_dir):
+        if not os.path.exists(input_dir):
+            gs.fatal('{} not exists'.format(input_dir))
+        self.input_dir = input_dir
+
+    def filter(self, pattern=None):
+        if pattern:
+            filter_p = '.*' + options['pattern'] + '.jp2$'
+        else:
+            filter_p = r'.*_B.*.jp2$'
+
+        self.files = self._filter(filter_p)
+
+    def _filter(self, filter_p):
+        pattern = re.compile(filter_p)
+        files = []
+        for rec in os.walk(self.input_dir):
+            if not rec[-1]:
+                continue
+
+            match = filter(pattern.match, rec[-1])
+            if match is None:
+                continue
+
+            for f in match:
+                files.append(os.path.join(rec[0], f))
+
+        return files
+
+    def import_products(self, link=False):
+        for f in self.files:
+            self._import_file(f, link)
+
+    def _import_file(self, filename, link=False):
+        module = 'r.external' if link else 'r.import'
+        mapname = os.path.splitext(os.path.basename(filename))[0]
+        if link:
+            gs.message('Linking <{}>...'.format(mapname))
+        else:
+            gs.message('Importing <{}>...'.format(mapname))
+        try:
+            gs.run_command(module, input=filename, output=mapname)
+        except CalledModuleError as e:
+            pass # error already printed
+
+    def import_cloud_masks(self):
+        files = self._filter("MSK_CLOUDS_B00.gml")
+
+        for f in files:
+            safe_dir = os.path.dirname(f).split(os.path.sep)[-4]
+            items = safe_dir.split('_')
+            map_name = '_'.join([items[5],items[2], 'MSK', 'CLOUDS'])
+            try:
+                gs.run_command('v.import', input=f,
+                               flags='o', # same SRS as data
+                               output=map_name,
+                               quiet=True
+                )
+            except CalledModuleError as e:
+                pass # error already printed
+
+def main():
+    importer = SentinelImporter(options['input'])
+
+    importer.filter(options['pattern'])
+
+    importer.import_products(flags['l'])
+
+    if flags['c']:
+        importer.import_cloud_masks()
+
+    return 0
+
+if __name__ == "__main__":
+    options, flags = gs.parser()
+    sys.exit(main())


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r.sentinel.import.py
___________________________________________________________________
Added: svn:mime-type
   + text/x-python
Added: svn:eol-style
   + native

Added: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r_sentinel_import_band_4_clouds.png
===================================================================
(Binary files differ)


Property changes on: grass-addons/grass7/raster/r.sentinel/r.sentinel.import/r_sentinel_import_band_4_clouds.png
___________________________________________________________________
Added: svn:mime-type
   + image/png



More information about the grass-commit mailing list