[GRASS-SVN] r72407 - in grass-addons/grass7/imagery/i.sentinel: i.sentinel.download i.sentinel.import
svn_grass at osgeo.org
svn_grass at osgeo.org
Tue Mar 20 04:52:22 PDT 2018
Author: martinl
Date: 2018-03-20 04:52:22 -0700 (Tue, 20 Mar 2018)
New Revision: 72407
Added:
grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.html
grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.py
grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.html
grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.py
grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/v_sentinel_import_band_4_clouds.png
Removed:
grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.html
grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.py
grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.html
grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.py
grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r_sentinel_import_band_4_clouds.png
Modified:
grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/Makefile
grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/Makefile
Log:
rename r.sentinel -> i.sentinel (done), see #3522
Modified: grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/Makefile
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/Makefile 2018-03-20 11:48:26 UTC (rev 72406)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/Makefile 2018-03-20 11:52:22 UTC (rev 72407)
@@ -1,6 +1,6 @@
MODULE_TOPDIR = ../..
-PGM = r.sentinel.download
+PGM = i.sentinel.download
include $(MODULE_TOPDIR)/include/Make/Script.make
Copied: grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.html (from rev 72406, grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.html)
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.html (rev 0)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.html 2018-03-20 11:52:22 UTC (rev 72407)
@@ -0,0 +1,141 @@
+<em>i.sentinel.download</em> allows downloading Sentinel products
+from <a href="https://scihub.copernicus.eu/">Copernicus Open Access
+Hub</a>.
+
+<p>
+To connect Copernicus Open Access Hub a <em>user</em>
+and <em>password</em> are required,
+see <a href="https://scihub.copernicus.eu/dhus/#/self-registration">Register
+new account</a> page for signing up.
+
+<p><em>i.sentinel.download</em> reads user credentials
+from <b>settings</b> file. The file must contain at least two lines:
+
+<div class="code"><pre>
+myusername
+mypassword
+</pre></div>
+
+Optionally on third line custom API URL can be defined. Note that
+empty lines in settings file are silently skipped.
+
+<h2>NOTES</h2>
+
+<p>User credentials can be also defined interactively
+when <b>settings=-</b> is given. Note that interactive prompt does not
+work in GUI.
+
+<div class="code"><pre>
+Insert username: myusername
+Insert password:
+Insert API URL (leave empty for https://scihub.copernicus.eu/dhus):
+</pre></div>
+
+<p>
+By default Sentinel products are sorted by <i>cloudcoverpercentage</i>
+and <i>ingestiondate</i> (see <b>sort</b> option). By default,
+only products which footprint intersects current computation region
+extent (area of interest, AOI) are filtered. The AOI can be optionally
+defined also by vector <b>map</b>. In addition the spatial relation
+between AOI and the footprint (<b>area_relation</b>) can be set to
+<ul>
+<li><i>Contains</i>: to only return scenes where the AOI is contained
+inside the footprint</li>
+<li><i>IsWithin</i>: to only return scenes where the footprint is
+contained inside the AOI</li>
+<li><i>Intersects</i>: to return all scenes where the footprint
+intersects the AOI (default)</li>
+</ul>
+Filtered products can be reduced by <b>limit</b> option.
+
+<p>
+Module searches for products in last 60 days, exact date range can be
+defined by <b>start</b> and <b>end</b> options.
+
+<p>
+Sentinel products can be also filtered by <b>producttype</b> or
+maximum <b>clouds</b> cover percentage.
+
+<h2>EXAMPLES</h2>
+
+<h3>List filtered products</h3>
+
+<p>
+Find S2MSI1C products in last 60 days covering current computation region extent.
+
+<div class="code"><pre>
+i.sentinel.download -l settings=sentinel.txt
+
+1 Sentinel product(s) found
+ae1c33ec-aa33-4303-a525-9e6481709614 2017-12-08T10:23:59Z 18% S2MSI1C
+</pre></div>
+
+Find all S2MSI2Ap products in 2017.
+
+<div class="code"><pre>
+i.sentinel.download -l settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31
+
+7 Sentinel product(s) found
+e5df8b4f-a4c6-47bd-88f3-e16b7540cc7a 2017-05-27T10:20:31Z 2% S2MSI2Ap
+b62afeda-a28d-475c-8220-91e24fc368ff 2017-05-17T10:20:31Z 2% S2MSI2Ap
+9a6bc289-98e9-4489-84eb-1aac95aaa056 2017-08-15T10:20:21Z 3% S2MSI2Ap
+35c72002-78a0-45f8-b39d-66c2d7b4ad87 2017-10-14T10:20:21Z 6% S2MSI2Ap
+c0ae8085-c1bb-4a27-89f2-2138a0866586 2017-07-06T10:20:21Z 12% S2MSI2Ap
+433ebfbc-5144-42f8-97dc-b9f4f1eb7b5a 2017-11-03T10:22:01Z 12% S2MSI2Ap
+fc56a594-d9d8-4e93-8dec-af3a58b24080 2017-09-04T10:20:21Z 19% S2MSI2Ap
+</pre></div>
+
+Sort products by <b>ingestiondate</b>, limit cloud coverage to 5% per scene.
+
+<div class="code"><pre>
+i.sentinel.download -l settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 sort=ingestiondate order=desc clouds=5
+
+3 Sentinel product(s) found
+9a6bc289-98e9-4489-84eb-1aac95aaa056 2017-08-15T10:20:21Z 3% S2MSI2Ap
+b62afeda-a28d-475c-8220-91e24fc368ff 2017-05-17T10:20:31Z 2% S2MSI2Ap
+e5df8b4f-a4c6-47bd-88f3-e16b7540cc7a 2017-05-27T10:20:31Z 2% S2MSI2Ap
+</pre></div>
+
+Create vector map with <b>footprints</b>.
+
+<div class="code"><pre>
+i.sentinel.download -l settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 footprints=ft
+</pre></div>
+
+<h3>Download Sentinel products</h3>
+
+Download first (<b>limit=1</b>) found S2MSI2Ap product into <i>data</i> directory.
+
+<div class="code"><pre>
+i.sentinel.download settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 limit=1 output=data
+</pre></div>
+
+Such downloaded data can be easily imported into GRASS
+using <em><a href="i.sentinel.import.html">i.sentinel.import</a></em>
+module.
+
+<h2>REQUIREMENTS</h2>
+
+<ul>
+ <li><a href="https://pypi.python.org/pypi/sentinelsat">Sentinelsat library</a></li>
+ <li><a href="https://pypi.python.org/pypi/pandas">Pandas library</a></li>
+</ul>
+
+<h2>SEE ALSO</h2>
+
+<em>
+<a href="i.sentinel.import.html">i.sentinel.import</a>,
+<a href="v.import.html">v.import</a>
+</em>
+
+<p>
+See
+also <a href="http://training.gismentors.eu/grass-gis-workshop-jena-2018/units/20.html">GRASS
+GIS Workshop in Jena</a> for usage examples.
+
+<h2>AUTHOR</h2>
+
+Martin
+Landa, <a href="http://geomatics.fsv.cvut.cz/research/geoforall/">GeoForAll
+Lab</a>, CTU in Prague, Czech Republic with support
+of <a href="http://opengeolabs.cz/en/home/">OpenGeoLabs</a> company
Copied: grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.py (from rev 72406, grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.py)
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.py (rev 0)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/i.sentinel.download.py 2018-03-20 11:52:22 UTC (rev 72407)
@@ -0,0 +1,341 @@
+#!/usr/bin/env python
+#
+############################################################################
+#
+# MODULE: i.sentinel.download
+# AUTHOR(S): Martin Landa
+# PURPOSE: Downloads Sentinel data from Copernicus Open Access Hub
+# using sentinelsat library.
+# COPYRIGHT: (C) 2018 by Martin Landa, and the GRASS development team
+#
+# This program is free software under the GNU General Public
+# License (>=v2). Read the file COPYING that comes with GRASS
+# for details.
+#
+#############################################################################
+
+#%module
+#% description: Downloads Sentinel data from Copernicus Open Access Hub using sentinelsat library.
+#% keyword: raster
+#% keyword: imagery
+#% keyword: sentinel
+#% keyword: download
+#%end
+#%option G_OPT_F_INPUT
+#% key: settings
+#% label: Full path to settings file (user, password)
+#% description: '-' for standard input
+#%end
+#%option G_OPT_M_DIR
+#% key: output
+#% description: Name for output directory where to store downloaded Sentinel data
+#% required: no
+#% guisection: Output
+#%end
+#%option G_OPT_V_OUTPUT
+#% key: footprints
+#% description: Name for output vector map with footprints
+#% required: no
+#% guisection: Output
+#%end
+#%option G_OPT_V_MAP
+#% label: Name of input vector map to define Area of Interest (AOI)
+#% description: If not given than current computational extent is used
+#% required: no
+#% guisection: Region
+#%end
+#%option
+#% key: area_relation
+#% type: string
+#% description: Spatial reation of footprint to AOI
+#% options: Intersects,Contains,IsWithin
+#% answer: Intersects
+#% required: no
+#% guisection: Region
+#%end
+#%option
+#% key: clouds
+#% type: integer
+#% description: Maximum cloud cover percentage for Sentinel scene
+#% answer: 30
+#% required: no
+#% guisection: Filter
+#%end
+#%option
+#% key: producttype
+#% type: string
+#% description: Sentinel product type to filter
+#% required: no
+#% options: SLC,GRD,OCN,S2MSI1C,S2MSI2Ap
+#% answer: S2MSI1C
+#% guisection: Filter
+#%end
+#%option
+#% key: start
+#% type: string
+#% description: Start date ('YYYY-MM-DD')
+#% guisection: Filter
+#%end
+#%option
+#% key: end
+#% type: string
+#% description: End date ('YYYY-MM-DD')
+#% guisection: Filter
+#%end
+#%option
+#% key: limit
+#% type: integer
+#% description: Limit number of Sentinel products
+#% guisection: Filter
+#%end
+#%option
+#% key: sort
+#% description: Sort by values in given order
+#% multiple: yes
+#% options: ingestiondate,cloudcoverpercentage,footprint
+#% answer: cloudcoverpercentage,ingestiondate,footprint
+#% guisection: Sort
+#%end
+#%option
+#% key: order
+#% description: Sort order (see sort parameter)
+#% options: asc,desc
+#% answer: asc
+#% guisection: Sort
+#%end
+#%flag
+#% key: l
+#% description: List filtered products and exist
+#% guisection: Print
+#%end
+#%rules
+#% required: output,-l
+#%end
+
+import os
+import sys
+import logging
+import zipfile
+
+from collections import OrderedDict
+
+import grass.script as gs
+
+def get_aoi_box(vector=None):
+ args = {}
+ if vector:
+ args['vector'] = vector
+ info = gs.parse_command('g.region', flags='uplg', **args)
+
+ return 'POLYGON(({nw_lon} {nw_lat}, {ne_lon} {ne_lat}, {se_lon} {se_lat}, {sw_lon} {sw_lat}, {nw_lon} {nw_lat}))'.format(
+ nw_lat=info['nw_lat'], nw_lon=info['nw_long'], ne_lat=info['ne_lat'], ne_lon=info['ne_long'],
+ sw_lat=info['sw_lat'], sw_lon=info['sw_long'], se_lat=info['se_lat'], se_lon=info['se_long']
+ )
+
+class SentinelDownloader(object):
+ def __init__(self, user, password, api_url='https://scihub.copernicus.eu/dhus'):
+ try:
+ from sentinelsat import SentinelAPI
+ except ImportError as e:
+ gs.fatal(_("Module requires sentinelsat library: {}").format(e))
+ try:
+ import pandas
+ except ImportError as e:
+ gs.fatal(_("Module requires pandas library: {}").format(e))
+
+ # init logger
+ root = logging.getLogger()
+ root.addHandler(logging.StreamHandler(
+ sys.stderr
+ ))
+
+ # connect SciHub via API
+ self._api = SentinelAPI(user, password,
+ api_url=api_url
+ )
+
+ self._products_df_sorted = None
+
+ def filter(self, area, area_relation,
+ clouds=None, producttype=None, limit=None,
+ start=None, end=None, sortby=[], asc=True):
+ args = {}
+ if clouds:
+ args['cloudcoverpercentage'] = (0, clouds)
+ if producttype:
+ args['producttype'] = producttype
+ if not start:
+ start = 'NOW-60DAYS'
+ else:
+ start = start.replace('-', '')
+ if not end:
+ end = 'NOW'
+ else:
+ end = end.replace('-', '')
+ products = self._api.query(
+ area=area, area_relation=area_relation,
+ platformname='Sentinel-2',
+ date=(start, end),
+ **args
+ )
+ products_df = self._api.to_dataframe(products)
+ if len(products_df) < 1:
+ gs.message(_('No product found'))
+ return
+
+ # sort and limit to first sorted product
+ if sortby:
+ self._products_df_sorted = products_df.sort_values(
+ sortby,
+ ascending=[asc] * len(sortby)
+ )
+
+ if limit:
+ self._products_df_sorted = self._products_df_sorted.head(int(limit))
+
+ gs.message(_('{} Sentinel product(s) found').format(len(self._products_df_sorted)))
+
+ def list(self):
+ if self._products_df_sorted is None:
+ return
+
+ for idx in range(len(self._products_df_sorted)):
+ print ('{0} {1} {2:2.0f}% {3}'.format(
+ self._products_df_sorted['uuid'][idx],
+ self._products_df_sorted['beginposition'][idx].strftime("%Y-%m-%dT%H:%M:%SZ"),
+ self._products_df_sorted['cloudcoverpercentage'][idx],
+ self._products_df_sorted['producttype'][idx],
+ ))
+
+ def download(self, output):
+ if self._products_df_sorted is None:
+ return
+
+ if not os.path.exists(output):
+ os.makedirs(output)
+ gs.message(_('Downloading data into <{}>...').format(output))
+ for idx in range(len(self._products_df_sorted)):
+ gs.message('{} -> {}.SAFE'.format(
+ self._products_df_sorted['uuid'][idx],
+ os.path.join(output, self._products_df_sorted['identifier'][idx])
+ ))
+ # download
+ self._api.download(self._products_df_sorted['uuid'][idx], output)
+
+ # unzip
+ if os.path.exists(os.path.join(output, self._products_df_sorted['identifier'][idx])):
+ continue
+ filename = self._products_df_sorted['identifier'][idx] + '.zip'
+ with zipfile.ZipFile(os.path.join(output, filename), 'r') as zip_ref:
+ zip_ref.extractall(output)
+
+ def save_footprints(self, map_name):
+ if self._products_df_sorted is None:
+ return
+
+ try:
+ from osgeo import ogr, osr
+ except ImportError as e:
+ gs.fatal(_("Option <footprints> requires GDAL library: {}").format(e))
+
+ gs.message(_("Writing footprints into <{}>...").format(map_name))
+ driver = ogr.GetDriverByName("GPKG")
+ tmp_name = gs.tempfile() + '.gpkg'
+ data_source = driver.CreateDataSource(tmp_name)
+
+ srs = osr.SpatialReference()
+ srs.ImportFromEPSG(4326)
+
+ layer = data_source.CreateLayer(str(map_name), srs, ogr.wkbPolygon)
+
+ # attributes
+ attrs = OrderedDict([
+ ("uuid", ogr.OFTString),
+ ("ingestiondate", ogr.OFTString),
+ ("cloudcoverpercentage", ogr.OFTInteger),
+ ("producttype", ogr.OFTString)
+ ])
+ for key in attrs.keys():
+ field = ogr.FieldDefn(key, attrs[key])
+ layer.CreateField(field)
+
+ # features
+ for idx in range(len(self._products_df_sorted)):
+ wkt = self._products_df_sorted['footprint'][idx]
+ feature = ogr.Feature(layer.GetLayerDefn())
+ feature.SetGeometry(ogr.CreateGeometryFromWkt(wkt))
+ for key in attrs.keys():
+ if key == 'ingestiondate':
+ value = self._products_df_sorted[key][idx].strftime("%Y-%m-%dT%H:%M:%SZ")
+ else:
+ value = self._products_df_sorted[key][idx]
+ feature.SetField(key, value)
+ layer.CreateFeature(feature)
+ feature = None
+
+ data_source = None
+
+ gs.run_command('v.import', input=tmp_name, output=map_name,
+ layer=map_name, quiet=True
+ )
+
+def main():
+ user = password = None
+ api_url='https://scihub.copernicus.eu/dhus'
+
+ if options['settings'] == '-':
+ # stdin
+ import getpass
+ user = raw_input(_('Insert username: '))
+ password = getpass.getpass(_('Insert password: '))
+ url = raw_input(_('Insert API URL (leave empty for {}): ').format(api_url))
+ if url:
+ api_url = url
+ else:
+ try:
+ with open(options['settings'], 'r') as fd:
+ lines = filter(None, (line.rstrip() for line in fd)) # non-blank lines only
+ if len(lines) < 2:
+ gs.fatal(_("Invalid settings file"))
+ user = lines[0].strip()
+ password = lines[1].strip()
+ if len(lines) > 2:
+ api_url = lines[2].strip()
+ except IOError as e:
+ gs.fatal(_("Unable to open settings file: {}").format(e))
+
+ if user is None or password is None:
+ gs.fatal(_("No user or password given"))
+
+ map_box = get_aoi_box(options['map'])
+
+ try:
+ downloader = SentinelDownloader(user, password, api_url)
+
+ downloader.filter(area=map_box,
+ area_relation=options['area_relation'],
+ clouds=options['clouds'],
+ producttype=options['producttype'],
+ limit=options['limit'],
+ start=options['start'],
+ end=options['end'],
+ sortby=options['sort'].split(','),
+ asc=options['order'] == 'asc'
+ )
+ except StandardError as e:
+ gs.fatal(_('Unable to connect Copernicus Open Access Hub: {}').format(e))
+
+ if options['footprints']:
+ downloader.save_footprints(options['footprints'])
+
+ if flags['l']:
+ downloader.list()
+ return
+
+ downloader.download(options['output'])
+
+ return 0
+
+if __name__ == "__main__":
+ options, flags = gs.parser()
+ sys.exit(main())
Deleted: grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.html
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.html 2018-03-20 11:48:26 UTC (rev 72406)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.html 2018-03-20 11:52:22 UTC (rev 72407)
@@ -1,141 +0,0 @@
-<em>r.sentinel.download</em> allows downloading Sentinel products
-from <a href="https://scihub.copernicus.eu/">Copernicus Open Access
-Hub</a>.
-
-<p>
-To connect Copernicus Open Access Hub a <em>user</em>
-and <em>password</em> are required,
-see <a href="https://scihub.copernicus.eu/dhus/#/self-registration">Register
-new account</a> page for signing up.
-
-<p><em>r.sentinel.download</em> reads user credentials
-from <b>settings</b> file. The file must contain at least two lines:
-
-<div class="code"><pre>
-myusername
-mypassword
-</pre></div>
-
-Optionally on third line custom API URL can be defined. Note that
-empty lines in settings file are silently skipped.
-
-<h2>NOTES</h2>
-
-<p>User credentials can be also defined interactively
-when <b>settings=-</b> is given. Note that interactive prompt does not
-work in GUI.
-
-<div class="code"><pre>
-Insert username: myusername
-Insert password:
-Insert API URL (leave empty for https://scihub.copernicus.eu/dhus):
-</pre></div>
-
-<p>
-By default Sentinel products are sorted by <i>cloudcoverpercentage</i>
-and <i>ingestiondate</i> (see <b>sort</b> option). By default,
-only products which footprint intersects current computation region
-extent (area of interest, AOI) are filtered. The AOI can be optionally
-defined also by vector <b>map</b>. In addition the spatial relation
-between AOI and the footprint (<b>area_relation</b>) can be set to
-<ul>
-<li><i>Contains</i>: to only return scenes where the AOI is contained
-inside the footprint</li>
-<li><i>IsWithin</i>: to only return scenes where the footprint is
-contained inside the AOI</li>
-<li><i>Intersects</i>: to return all scenes where the footprint
-intersects the AOI (default)</li>
-</ul>
-Filtered products can be reduced by <b>limit</b> option.
-
-<p>
-Module searches for products in last 60 days, exact date range can be
-defined by <b>start</b> and <b>end</b> options.
-
-<p>
-Sentinel products can be also filtered by <b>producttype</b> or
-maximum <b>clouds</b> cover percentage.
-
-<h2>EXAMPLES</h2>
-
-<h3>List filtered products</h3>
-
-<p>
-Find S2MSI1C products in last 60 days covering current computation region extent.
-
-<div class="code"><pre>
-r.sentinel.download -l settings=sentinel.txt
-
-1 Sentinel product(s) found
-ae1c33ec-aa33-4303-a525-9e6481709614 2017-12-08T10:23:59Z 18% S2MSI1C
-</pre></div>
-
-Find all S2MSI2Ap products in 2017.
-
-<div class="code"><pre>
-r.sentinel.download -l settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31
-
-7 Sentinel product(s) found
-e5df8b4f-a4c6-47bd-88f3-e16b7540cc7a 2017-05-27T10:20:31Z 2% S2MSI2Ap
-b62afeda-a28d-475c-8220-91e24fc368ff 2017-05-17T10:20:31Z 2% S2MSI2Ap
-9a6bc289-98e9-4489-84eb-1aac95aaa056 2017-08-15T10:20:21Z 3% S2MSI2Ap
-35c72002-78a0-45f8-b39d-66c2d7b4ad87 2017-10-14T10:20:21Z 6% S2MSI2Ap
-c0ae8085-c1bb-4a27-89f2-2138a0866586 2017-07-06T10:20:21Z 12% S2MSI2Ap
-433ebfbc-5144-42f8-97dc-b9f4f1eb7b5a 2017-11-03T10:22:01Z 12% S2MSI2Ap
-fc56a594-d9d8-4e93-8dec-af3a58b24080 2017-09-04T10:20:21Z 19% S2MSI2Ap
-</pre></div>
-
-Sort products by <b>ingestiondate</b>, limit cloud coverage to 5% per scene.
-
-<div class="code"><pre>
-r.sentinel.download -l settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 sort=ingestiondate order=desc clouds=5
-
-3 Sentinel product(s) found
-9a6bc289-98e9-4489-84eb-1aac95aaa056 2017-08-15T10:20:21Z 3% S2MSI2Ap
-b62afeda-a28d-475c-8220-91e24fc368ff 2017-05-17T10:20:31Z 2% S2MSI2Ap
-e5df8b4f-a4c6-47bd-88f3-e16b7540cc7a 2017-05-27T10:20:31Z 2% S2MSI2Ap
-</pre></div>
-
-Create vector map with <b>footprints</b>.
-
-<div class="code"><pre>
-r.sentinel.download -l settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 footprints=ft
-</pre></div>
-
-<h3>Download Sentinel products</h3>
-
-Download first (<b>limit=1</b>) found S2MSI2Ap product into <i>data</i> directory.
-
-<div class="code"><pre>
-r.sentinel.download settings=sentinel.txt producttype=S2MSI2Ap start=2017-01-01 end=2017-12-31 limit=1 output=data
-</pre></div>
-
-Such downloaded data can be easily imported into GRASS
-using <em><a href="r.sentinel.import.html">r.sentinel.import</a></em>
-module.
-
-<h2>REQUIREMENTS</h2>
-
-<ul>
- <li><a href="https://pypi.python.org/pypi/sentinelsat">Sentinelsat library</a></li>
- <li><a href="https://pypi.python.org/pypi/pandas">Pandas library</a></li>
-</ul>
-
-<h2>SEE ALSO</h2>
-
-<em>
-<a href="r.sentinel.import.html">r.sentinel.import</a>,
-<a href="v.import.html">v.import</a>
-</em>
-
-<p>
-See
-also <a href="http://training.gismentors.eu/grass-gis-workshop-jena-2018/units/20.html">GRASS
-GIS Workshop in Jena</a> for usage examples.
-
-<h2>AUTHOR</h2>
-
-Martin
-Landa, <a href="http://geomatics.fsv.cvut.cz/research/geoforall/">GeoForAll
-Lab</a>, CTU in Prague, Czech Republic with support
-of <a href="http://opengeolabs.cz/en/home/">OpenGeoLabs</a> company
Deleted: grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.py
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.py 2018-03-20 11:48:26 UTC (rev 72406)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.download/r.sentinel.download.py 2018-03-20 11:52:22 UTC (rev 72407)
@@ -1,341 +0,0 @@
-#!/usr/bin/env python
-#
-############################################################################
-#
-# MODULE: r.sentinel.download
-# AUTHOR(S): Martin Landa
-# PURPOSE: Downloads Sentinel data from Copernicus Open Access Hub
-# using sentinelsat library.
-# COPYRIGHT: (C) 2018 by Martin Landa, and the GRASS development team
-#
-# This program is free software under the GNU General Public
-# License (>=v2). Read the file COPYING that comes with GRASS
-# for details.
-#
-#############################################################################
-
-#%module
-#% description: Downloads Sentinel data from Copernicus Open Access Hub using sentinelsat library.
-#% keyword: raster
-#% keyword: imagery
-#% keyword: sentinel
-#% keyword: download
-#%end
-#%option G_OPT_F_INPUT
-#% key: settings
-#% label: Full path to settings file (user, password)
-#% description: '-' for standard input
-#%end
-#%option G_OPT_M_DIR
-#% key: output
-#% description: Name for output directory where to store downloaded Sentinel data
-#% required: no
-#% guisection: Output
-#%end
-#%option G_OPT_V_OUTPUT
-#% key: footprints
-#% description: Name for output vector map with footprints
-#% required: no
-#% guisection: Output
-#%end
-#%option G_OPT_V_MAP
-#% label: Name of input vector map to define Area of Interest (AOI)
-#% description: If not given than current computational extent is used
-#% required: no
-#% guisection: Region
-#%end
-#%option
-#% key: area_relation
-#% type: string
-#% description: Spatial reation of footprint to AOI
-#% options: Intersects,Contains,IsWithin
-#% answer: Intersects
-#% required: no
-#% guisection: Region
-#%end
-#%option
-#% key: clouds
-#% type: integer
-#% description: Maximum cloud cover percentage for Sentinel scene
-#% answer: 30
-#% required: no
-#% guisection: Filter
-#%end
-#%option
-#% key: producttype
-#% type: string
-#% description: Sentinel product type to filter
-#% required: no
-#% options: SLC,GRD,OCN,S2MSI1C,S2MSI2Ap
-#% answer: S2MSI1C
-#% guisection: Filter
-#%end
-#%option
-#% key: start
-#% type: string
-#% description: Start date ('YYYY-MM-DD')
-#% guisection: Filter
-#%end
-#%option
-#% key: end
-#% type: string
-#% description: End date ('YYYY-MM-DD')
-#% guisection: Filter
-#%end
-#%option
-#% key: limit
-#% type: integer
-#% description: Limit number of Sentinel products
-#% guisection: Filter
-#%end
-#%option
-#% key: sort
-#% description: Sort by values in given order
-#% multiple: yes
-#% options: ingestiondate,cloudcoverpercentage,footprint
-#% answer: cloudcoverpercentage,ingestiondate,footprint
-#% guisection: Sort
-#%end
-#%option
-#% key: order
-#% description: Sort order (see sort parameter)
-#% options: asc,desc
-#% answer: asc
-#% guisection: Sort
-#%end
-#%flag
-#% key: l
-#% description: List filtered products and exist
-#% guisection: Print
-#%end
-#%rules
-#% required: output,-l
-#%end
-
-import os
-import sys
-import logging
-import zipfile
-
-from collections import OrderedDict
-
-import grass.script as gs
-
-def get_aoi_box(vector=None):
- args = {}
- if vector:
- args['vector'] = vector
- info = gs.parse_command('g.region', flags='uplg', **args)
-
- return 'POLYGON(({nw_lon} {nw_lat}, {ne_lon} {ne_lat}, {se_lon} {se_lat}, {sw_lon} {sw_lat}, {nw_lon} {nw_lat}))'.format(
- nw_lat=info['nw_lat'], nw_lon=info['nw_long'], ne_lat=info['ne_lat'], ne_lon=info['ne_long'],
- sw_lat=info['sw_lat'], sw_lon=info['sw_long'], se_lat=info['se_lat'], se_lon=info['se_long']
- )
-
-class SentinelDownloader(object):
- def __init__(self, user, password, api_url='https://scihub.copernicus.eu/dhus'):
- try:
- from sentinelsat import SentinelAPI
- except ImportError as e:
- gs.fatal(_("Module requires sentinelsat library: {}").format(e))
- try:
- import pandas
- except ImportError as e:
- gs.fatal(_("Module requires pandas library: {}").format(e))
-
- # init logger
- root = logging.getLogger()
- root.addHandler(logging.StreamHandler(
- sys.stderr
- ))
-
- # connect SciHub via API
- self._api = SentinelAPI(user, password,
- api_url=api_url
- )
-
- self._products_df_sorted = None
-
- def filter(self, area, area_relation,
- clouds=None, producttype=None, limit=None,
- start=None, end=None, sortby=[], asc=True):
- args = {}
- if clouds:
- args['cloudcoverpercentage'] = (0, clouds)
- if producttype:
- args['producttype'] = producttype
- if not start:
- start = 'NOW-60DAYS'
- else:
- start = start.replace('-', '')
- if not end:
- end = 'NOW'
- else:
- end = end.replace('-', '')
- products = self._api.query(
- area=area, area_relation=area_relation,
- platformname='Sentinel-2',
- date=(start, end),
- **args
- )
- products_df = self._api.to_dataframe(products)
- if len(products_df) < 1:
- gs.message(_('No product found'))
- return
-
- # sort and limit to first sorted product
- if sortby:
- self._products_df_sorted = products_df.sort_values(
- sortby,
- ascending=[asc] * len(sortby)
- )
-
- if limit:
- self._products_df_sorted = self._products_df_sorted.head(int(limit))
-
- gs.message(_('{} Sentinel product(s) found').format(len(self._products_df_sorted)))
-
- def list(self):
- if self._products_df_sorted is None:
- return
-
- for idx in range(len(self._products_df_sorted)):
- print ('{0} {1} {2:2.0f}% {3}'.format(
- self._products_df_sorted['uuid'][idx],
- self._products_df_sorted['beginposition'][idx].strftime("%Y-%m-%dT%H:%M:%SZ"),
- self._products_df_sorted['cloudcoverpercentage'][idx],
- self._products_df_sorted['producttype'][idx],
- ))
-
- def download(self, output):
- if self._products_df_sorted is None:
- return
-
- if not os.path.exists(output):
- os.makedirs(output)
- gs.message(_('Downloading data into <{}>...').format(output))
- for idx in range(len(self._products_df_sorted)):
- gs.message('{} -> {}.SAFE'.format(
- self._products_df_sorted['uuid'][idx],
- os.path.join(output, self._products_df_sorted['identifier'][idx])
- ))
- # download
- self._api.download(self._products_df_sorted['uuid'][idx], output)
-
- # unzip
- if os.path.exists(os.path.join(output, self._products_df_sorted['identifier'][idx])):
- continue
- filename = self._products_df_sorted['identifier'][idx] + '.zip'
- with zipfile.ZipFile(os.path.join(output, filename), 'r') as zip_ref:
- zip_ref.extractall(output)
-
- def save_footprints(self, map_name):
- if self._products_df_sorted is None:
- return
-
- try:
- from osgeo import ogr, osr
- except ImportError as e:
- gs.fatal(_("Option <footprints> requires GDAL library: {}").format(e))
-
- gs.message(_("Writing footprints into <{}>...").format(map_name))
- driver = ogr.GetDriverByName("GPKG")
- tmp_name = gs.tempfile() + '.gpkg'
- data_source = driver.CreateDataSource(tmp_name)
-
- srs = osr.SpatialReference()
- srs.ImportFromEPSG(4326)
-
- layer = data_source.CreateLayer(str(map_name), srs, ogr.wkbPolygon)
-
- # attributes
- attrs = OrderedDict([
- ("uuid", ogr.OFTString),
- ("ingestiondate", ogr.OFTString),
- ("cloudcoverpercentage", ogr.OFTInteger),
- ("producttype", ogr.OFTString)
- ])
- for key in attrs.keys():
- field = ogr.FieldDefn(key, attrs[key])
- layer.CreateField(field)
-
- # features
- for idx in range(len(self._products_df_sorted)):
- wkt = self._products_df_sorted['footprint'][idx]
- feature = ogr.Feature(layer.GetLayerDefn())
- feature.SetGeometry(ogr.CreateGeometryFromWkt(wkt))
- for key in attrs.keys():
- if key == 'ingestiondate':
- value = self._products_df_sorted[key][idx].strftime("%Y-%m-%dT%H:%M:%SZ")
- else:
- value = self._products_df_sorted[key][idx]
- feature.SetField(key, value)
- layer.CreateFeature(feature)
- feature = None
-
- data_source = None
-
- gs.run_command('v.import', input=tmp_name, output=map_name,
- layer=map_name, quiet=True
- )
-
-def main():
- user = password = None
- api_url='https://scihub.copernicus.eu/dhus'
-
- if options['settings'] == '-':
- # stdin
- import getpass
- user = raw_input(_('Insert username: '))
- password = getpass.getpass(_('Insert password: '))
- url = raw_input(_('Insert API URL (leave empty for {}): ').format(api_url))
- if url:
- api_url = url
- else:
- try:
- with open(options['settings'], 'r') as fd:
- lines = filter(None, (line.rstrip() for line in fd)) # non-blank lines only
- if len(lines) < 2:
- gs.fatal(_("Invalid settings file"))
- user = lines[0].strip()
- password = lines[1].strip()
- if len(lines) > 2:
- api_url = lines[2].strip()
- except IOError as e:
- gs.fatal(_("Unable to open settings file: {}").format(e))
-
- if user is None or password is None:
- gs.fatal(_("No user or password given"))
-
- map_box = get_aoi_box(options['map'])
-
- try:
- downloader = SentinelDownloader(user, password, api_url)
-
- downloader.filter(area=map_box,
- area_relation=options['area_relation'],
- clouds=options['clouds'],
- producttype=options['producttype'],
- limit=options['limit'],
- start=options['start'],
- end=options['end'],
- sortby=options['sort'].split(','),
- asc=options['order'] == 'asc'
- )
- except StandardError as e:
- gs.fatal(_('Unable to connect Copernicus Open Access Hub: {}').format(e))
-
- if options['footprints']:
- downloader.save_footprints(options['footprints'])
-
- if flags['l']:
- downloader.list()
- return
-
- downloader.download(options['output'])
-
- return 0
-
-if __name__ == "__main__":
- options, flags = gs.parser()
- sys.exit(main())
Modified: grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/Makefile
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/Makefile 2018-03-20 11:48:26 UTC (rev 72406)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/Makefile 2018-03-20 11:52:22 UTC (rev 72407)
@@ -1,6 +1,6 @@
MODULE_TOPDIR = ../..
-PGM = r.sentinel.import
+PGM = i.sentinel.import
include $(MODULE_TOPDIR)/include/Make/Script.make
Copied: grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.html (from rev 72406, grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.html)
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.html (rev 0)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.html 2018-03-20 11:52:22 UTC (rev 72407)
@@ -0,0 +1,86 @@
+<em>i.sentinel.import</em> module allows importing Sentinel products
+downloaded
+by <em><a href="i.sentinel.download.html">i.sentinel.download</a></em>
+module.
+
+<p>
+By default <em>i.sentinel.import</em> imports all Sentinel bands found
+in <b>input</b> directory
+by <em><a href="r.import.html">r.import</a></em>. Note that in the
+case that spatial reference system of input data differs from GRASS
+location, the input data are reprojected.
+
+<p>
+Optionally input data can be linked
+by <em><a href="r.external.html">r.external</a></em> when <b>-l</b> is
+given. Note that linking data requires that input data and GRASS
+location have the same spatial reference system.
+
+<p>
+Number of imported Sentinel bands can be optionally reduced
+by <b>pattern</b> option.
+
+<h2>NOTES</h2>
+
+<p>
+If <b>-c</b> flag is given, than also cloud mask file is imported as
+vector map if available. Name of created vector map is determined from
+input Sentinel product.
+
+<h2>EXAMPLES</h2>
+
+At first, print list of raster files to be imported by <b>-p</b>. For
+each file also projection match with current location is printed
+including detected input data EPSG code.
+
+<div class="code"><pre>
+i.sentinel.import -p input=data
+
+data/S2B_MSIL1C_20180216T102059_N0206_R065_T32UPB_20180216T140508.SAFE/GRANULE/.../T32UPB_20180216T102059_B04.jp2 1 (EPSG: 32632)
+data/S2B_MSIL1C_20180216T102059_N0206_R065_T32UPB_20180216T140508.SAFE/GRANULE/.../T32UPB_20180216T102059_B07.jp2 1 (EPSG: 32632)
+data/S2B_MSIL1C_20180216T102059_N0206_R065_T32UPB_20180216T140508.SAFE/GRANULE/.../T32UPB_20180216T102059_B11.jp2 1 (EPSG: 32632)
+</pre></div>
+
+Import all Sentinel bands found in <i>data</i> directory.
+
+<div class="code"><pre>
+i.sentinel.import input=data
+</pre></div>
+
+Limit import only to 4th and 8th bands.
+
+<div class="code"><pre>
+i.sentinel.import input=data pattern='B0[4|8]'
+</pre></div>
+
+Link data and import also cloud mask file.
+
+<div class="code"><pre>
+i.sentinel.import -l -c input=data
+</pre></div>
+
+<center>
+<img src="i_sentinel_import_band_4_clouds.png" width="600" height="356"><br>
+<i>Fig: Band 4 with imported cloud mask</i>
+</center>
+
+<h2>SEE ALSO</h2>
+
+<em>
+<a href="i.sentinel.download.html">i.sentinel.download</a>,
+<a href="r.import.html">r.import</a>,
+<a href="r.extenal.html">r.external</a>,
+<a href="v.import.html">v.import</a>
+</em>
+
+<p>
+See
+also <a href="http://training.gismentors.eu/grass-gis-workshop-jena-2018/units/20.html">GRASS
+GIS Workshop in Jena</a> for usage examples.
+
+<h2>AUTHOR</h2>
+
+Martin
+Landa, <a href="http://geomatics.fsv.cvut.cz/research/geoforall/">GeoForAll
+Lab</a>, CTU in Prague, Czech Republic with support
+of <a href="http://opengeolabs.cz/en/home/">OpenGeoLabs</a> company
Copied: grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.py (from rev 72406, grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.py)
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.py (rev 0)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/i.sentinel.import.py 2018-03-20 11:52:22 UTC (rev 72407)
@@ -0,0 +1,199 @@
+#!/usr/bin/env python
+#
+############################################################################
+#
+# MODULE: i.sentinel.import
+# AUTHOR(S): Martin Landa
+# PURPOSE: Imports Sentinel data downloaded from Copernicus Open Access Hub
+# using i.sentinel.download.
+# COPYRIGHT: (C) 2018 by Martin Landa, and the GRASS development team
+#
+# This program is free software under the GNU General Public
+# License (>=v2). Read the file COPYING that comes with GRASS
+# for details.
+#
+#############################################################################
+
+#%Module
+#% description: Imports Sentinel data downloaded from Copernicus Open Access Hub using i.sentinel.download.
+#% keyword: raster
+#% keyword: imagery
+#% keyword: sentinel
+#% keyword: import
+#%end
+#%option G_OPT_M_DIR
+#% key: input
+#% description: Name for input directory where downloaded Sentinel data lives
+#% required: yes
+#%end
+#%option
+#% key: pattern
+#% description: File name pattern to import
+#%end
+#%flag
+#% key: r
+#% description: Reproject raster data using r.import if needed
+#%end
+#%flag
+#% key: l
+#% description: Link raster data instead of importing
+#%end
+#%flag
+#% key: c
+#% description: Import cloud masks as vector maps
+#%end
+#%flag
+#% key: p
+#% description: Print raster data to be imported and exit
+#%end
+#%rules
+#% exclusive: -l,-r,-p
+#%end
+import os
+import sys
+import re
+
+import grass.script as gs
+from grass.exceptions import CalledModuleError
+
+class SentinelImporter(object):
+ def __init__(self, input_dir):
+ if not os.path.exists(input_dir):
+ gs.fatal(_('{} not exists').format(input_dir))
+ self.input_dir = input_dir
+
+ def filter(self, pattern=None):
+ if pattern:
+ filter_p = '.*' + options['pattern'] + '.*.jp2$'
+ else:
+ filter_p = r'.*_B.*.jp2$'
+
+ self.files = self._filter(filter_p)
+
+ def _filter(self, filter_p):
+ pattern = re.compile(filter_p)
+ files = []
+ for rec in os.walk(self.input_dir):
+ if not rec[-1]:
+ continue
+
+ match = filter(pattern.match, rec[-1])
+ if match is None:
+ continue
+
+ for f in match:
+ files.append(os.path.join(rec[0], f))
+
+ return files
+
+ def import_products(self, reproject=False, link=False):
+ args = {}
+ if link:
+ module = 'r.external'
+ else:
+ if reproject:
+ module = 'r.import'
+ args['resample'] = 'bilinear'
+ args['resolution'] = 'value'
+ else:
+ module = 'r.in.gdal'
+
+ for f in self.files:
+ if link or (not link and not reproject):
+ if not self._check_projection(f):
+ gs.fatal(_('Projection of dataset does not appear to match current location. '
+ 'Force reprojecting dataset by -r flag.'))
+
+ self._import_file(f, module, args)
+
+ def _check_projection(self, filename):
+ try:
+ gs.run_command('r.in.gdal', flags='j',
+ input=filename, quiet=True)
+ except CalledModuleError as e:
+ return False
+
+ return True
+
+ def _raster_resolution(self, filename):
+ try:
+ from osgeo import gdal
+ except ImportError as e:
+ gs.fatal(_("Flag -r requires GDAL library: {}").format(e))
+ dsn = gdal.Open(filename)
+ trans = dsn.GetGeoTransform()
+
+ ret = int(trans[1])
+ dsn = None
+
+ return ret
+
+ def _raster_epsg(self, filename):
+ try:
+ from osgeo import gdal, osr
+ except ImportError as e:
+ gs.fatal(_("Flag -r requires GDAL library: {}").format(e))
+ dsn = gdal.Open(filename)
+
+ srs = osr.SpatialReference()
+ srs.ImportFromWkt(dsn.GetProjectionRef())
+
+ ret = srs.GetAuthorityCode(None)
+ dsn = None
+
+ return ret
+
+ def _import_file(self, filename, module, args):
+ mapname = os.path.splitext(os.path.basename(filename))[0]
+ gs.message(_('Processing <{}>...').format(mapname))
+ if module == 'r.import':
+ args['resolution_value'] = self._raster_resolution(filename)
+ try:
+ gs.run_command(module, input=filename, output=mapname, **args)
+ except CalledModuleError as e:
+ pass # error already printed
+
+ def import_cloud_masks(self):
+ files = self._filter("MSK_CLOUDS_B00.gml")
+
+ for f in files:
+ safe_dir = os.path.dirname(f).split(os.path.sep)[-4]
+ items = safe_dir.split('_')
+ map_name = '_'.join([items[5],items[2], 'MSK', 'CLOUDS'])
+ try:
+ gs.run_command('v.import', input=f,
+ flags='o', # same SRS as data
+ output=map_name,
+ quiet=True
+ )
+ except CalledModuleError as e:
+ pass # error already printed
+
+ def print_products(self):
+ for f in self.files:
+ sys.stdout.write('{} {} (EPSG: {}){}'.format(
+ f,
+ '1' if self._check_projection(f) else '0',
+ self._raster_epsg(f),
+ os.linesep
+ ))
+
+def main():
+ importer = SentinelImporter(options['input'])
+
+ importer.filter(options['pattern'])
+
+ if flags['p']:
+ importer.print_products()
+ return 0
+
+ importer.import_products(flags['r'], flags['l'])
+
+ if flags['c']:
+ importer.import_cloud_masks()
+
+ return 0
+
+if __name__ == "__main__":
+ options, flags = gs.parser()
+ sys.exit(main())
Deleted: grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.html
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.html 2018-03-20 11:48:26 UTC (rev 72406)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.html 2018-03-20 11:52:22 UTC (rev 72407)
@@ -1,86 +0,0 @@
-<em>r.sentinel.import</em> module allows importing Sentinel products
-downloaded
-by <em><a href="r.sentinel.download.html">r.sentinel.download</a></em>
-module.
-
-<p>
-By default <em>r.sentinel.import</em> imports all Sentinel bands found
-in <b>input</b> directory
-by <em><a href="r.import.html">r.import</a></em>. Note that in the
-case that spatial reference system of input data differs from GRASS
-location, the input data are reprojected.
-
-<p>
-Optionally input data can be linked
-by <em><a href="r.external.html">r.external</a></em> when <b>-l</b> is
-given. Note that linking data requires that input data and GRASS
-location have the same spatial reference system.
-
-<p>
-Number of imported Sentinel bands can be optionally reduced
-by <b>pattern</b> option.
-
-<h2>NOTES</h2>
-
-<p>
-If <b>-c</b> flag is given, than also cloud mask file is imported as
-vector map if available. Name of created vector map is determined from
-input Sentinel product.
-
-<h2>EXAMPLES</h2>
-
-At first, print list of raster files to be imported by <b>-p</b>. For
-each file also projection match with current location is printed
-including detected input data EPSG code.
-
-<div class="code"><pre>
-r.sentinel.import -p input=data
-
-data/S2B_MSIL1C_20180216T102059_N0206_R065_T32UPB_20180216T140508.SAFE/GRANULE/.../T32UPB_20180216T102059_B04.jp2 1 (EPSG: 32632)
-data/S2B_MSIL1C_20180216T102059_N0206_R065_T32UPB_20180216T140508.SAFE/GRANULE/.../T32UPB_20180216T102059_B07.jp2 1 (EPSG: 32632)
-data/S2B_MSIL1C_20180216T102059_N0206_R065_T32UPB_20180216T140508.SAFE/GRANULE/.../T32UPB_20180216T102059_B11.jp2 1 (EPSG: 32632)
-</pre></div>
-
-Import all Sentinel bands found in <i>data</i> directory.
-
-<div class="code"><pre>
-r.sentinel.import input=data
-</pre></div>
-
-Limit import only to 4th and 8th bands.
-
-<div class="code"><pre>
-r.sentinel.import input=data pattern='B0[4|8]'
-</pre></div>
-
-Link data and import also cloud mask file.
-
-<div class="code"><pre>
-r.sentinel.import -l -c input=data
-</pre></div>
-
-<center>
-<img src="r_sentinel_import_band_4_clouds.png" width="600" height="356"><br>
-<i>Fig: Band 4 with imported cloud mask</i>
-</center>
-
-<h2>SEE ALSO</h2>
-
-<em>
-<a href="r.sentinel.download.html">r.sentinel.download</a>,
-<a href="r.import.html">r.import</a>,
-<a href="r.extenal.html">r.external</a>,
-<a href="v.import.html">v.import</a>
-</em>
-
-<p>
-See
-also <a href="http://training.gismentors.eu/grass-gis-workshop-jena-2018/units/20.html">GRASS
-GIS Workshop in Jena</a> for usage examples.
-
-<h2>AUTHOR</h2>
-
-Martin
-Landa, <a href="http://geomatics.fsv.cvut.cz/research/geoforall/">GeoForAll
-Lab</a>, CTU in Prague, Czech Republic with support
-of <a href="http://opengeolabs.cz/en/home/">OpenGeoLabs</a> company
Deleted: grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.py
===================================================================
--- grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.py 2018-03-20 11:48:26 UTC (rev 72406)
+++ grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r.sentinel.import.py 2018-03-20 11:52:22 UTC (rev 72407)
@@ -1,199 +0,0 @@
-#!/usr/bin/env python
-#
-############################################################################
-#
-# MODULE: r.sentinel.import
-# AUTHOR(S): Martin Landa
-# PURPOSE: Imports Sentinel data downloaded from Copernicus Open Access Hub
-# using r.sentinel.download.
-# COPYRIGHT: (C) 2018 by Martin Landa, and the GRASS development team
-#
-# This program is free software under the GNU General Public
-# License (>=v2). Read the file COPYING that comes with GRASS
-# for details.
-#
-#############################################################################
-
-#%Module
-#% description: Imports Sentinel data downloaded from Copernicus Open Access Hub using r.sentinel.download.
-#% keyword: raster
-#% keyword: imagery
-#% keyword: sentinel
-#% keyword: import
-#%end
-#%option G_OPT_M_DIR
-#% key: input
-#% description: Name for input directory where downloaded Sentinel data lives
-#% required: yes
-#%end
-#%option
-#% key: pattern
-#% description: File name pattern to import
-#%end
-#%flag
-#% key: r
-#% description: Reproject raster data using r.import if needed
-#%end
-#%flag
-#% key: l
-#% description: Link raster data instead of importing
-#%end
-#%flag
-#% key: c
-#% description: Import cloud masks as vector maps
-#%end
-#%flag
-#% key: p
-#% description: Print raster data to be imported and exit
-#%end
-#%rules
-#% exclusive: -l,-r,-p
-#%end
-import os
-import sys
-import re
-
-import grass.script as gs
-from grass.exceptions import CalledModuleError
-
-class SentinelImporter(object):
- def __init__(self, input_dir):
- if not os.path.exists(input_dir):
- gs.fatal(_('{} not exists').format(input_dir))
- self.input_dir = input_dir
-
- def filter(self, pattern=None):
- if pattern:
- filter_p = '.*' + options['pattern'] + '.*.jp2$'
- else:
- filter_p = r'.*_B.*.jp2$'
-
- self.files = self._filter(filter_p)
-
- def _filter(self, filter_p):
- pattern = re.compile(filter_p)
- files = []
- for rec in os.walk(self.input_dir):
- if not rec[-1]:
- continue
-
- match = filter(pattern.match, rec[-1])
- if match is None:
- continue
-
- for f in match:
- files.append(os.path.join(rec[0], f))
-
- return files
-
- def import_products(self, reproject=False, link=False):
- args = {}
- if link:
- module = 'r.external'
- else:
- if reproject:
- module = 'r.import'
- args['resample'] = 'bilinear'
- args['resolution'] = 'value'
- else:
- module = 'r.in.gdal'
-
- for f in self.files:
- if link or (not link and not reproject):
- if not self._check_projection(f):
- gs.fatal(_('Projection of dataset does not appear to match current location. '
- 'Force reprojecting dataset by -r flag.'))
-
- self._import_file(f, module, args)
-
- def _check_projection(self, filename):
- try:
- gs.run_command('r.in.gdal', flags='j',
- input=filename, quiet=True)
- except CalledModuleError as e:
- return False
-
- return True
-
- def _raster_resolution(self, filename):
- try:
- from osgeo import gdal
- except ImportError as e:
- gs.fatal(_("Flag -r requires GDAL library: {}").format(e))
- dsn = gdal.Open(filename)
- trans = dsn.GetGeoTransform()
-
- ret = int(trans[1])
- dsn = None
-
- return ret
-
- def _raster_epsg(self, filename):
- try:
- from osgeo import gdal, osr
- except ImportError as e:
- gs.fatal(_("Flag -r requires GDAL library: {}").format(e))
- dsn = gdal.Open(filename)
-
- srs = osr.SpatialReference()
- srs.ImportFromWkt(dsn.GetProjectionRef())
-
- ret = srs.GetAuthorityCode(None)
- dsn = None
-
- return ret
-
- def _import_file(self, filename, module, args):
- mapname = os.path.splitext(os.path.basename(filename))[0]
- gs.message(_('Processing <{}>...').format(mapname))
- if module == 'r.import':
- args['resolution_value'] = self._raster_resolution(filename)
- try:
- gs.run_command(module, input=filename, output=mapname, **args)
- except CalledModuleError as e:
- pass # error already printed
-
- def import_cloud_masks(self):
- files = self._filter("MSK_CLOUDS_B00.gml")
-
- for f in files:
- safe_dir = os.path.dirname(f).split(os.path.sep)[-4]
- items = safe_dir.split('_')
- map_name = '_'.join([items[5],items[2], 'MSK', 'CLOUDS'])
- try:
- gs.run_command('v.import', input=f,
- flags='o', # same SRS as data
- output=map_name,
- quiet=True
- )
- except CalledModuleError as e:
- pass # error already printed
-
- def print_products(self):
- for f in self.files:
- sys.stdout.write('{} {} (EPSG: {}){}'.format(
- f,
- '1' if self._check_projection(f) else '0',
- self._raster_epsg(f),
- os.linesep
- ))
-
-def main():
- importer = SentinelImporter(options['input'])
-
- importer.filter(options['pattern'])
-
- if flags['p']:
- importer.print_products()
- return 0
-
- importer.import_products(flags['r'], flags['l'])
-
- if flags['c']:
- importer.import_cloud_masks()
-
- return 0
-
-if __name__ == "__main__":
- options, flags = gs.parser()
- sys.exit(main())
Deleted: grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r_sentinel_import_band_4_clouds.png
===================================================================
(Binary files differ)
Copied: grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/v_sentinel_import_band_4_clouds.png (from rev 72406, grass-addons/grass7/imagery/i.sentinel/i.sentinel.import/r_sentinel_import_band_4_clouds.png)
===================================================================
(Binary files differ)
More information about the grass-commit
mailing list