[GRASS-SVN] r72415 - in grass-addons/grass7: imagery imagery/i.modis imagery/i.modis/i.modis.download imagery/i.modis/i.modis.import imagery/i.modis/libmodis raster raster/r.modis
svn_grass at osgeo.org
svn_grass at osgeo.org
Tue Mar 20 08:00:13 PDT 2018
Author: lucadelu
Date: 2018-03-20 08:00:13 -0700 (Tue, 20 Mar 2018)
New Revision: 72415
Added:
grass-addons/grass7/imagery/i.modis/
grass-addons/grass7/imagery/i.modis/i.modis.download/
grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.html
grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.py
grass-addons/grass7/imagery/i.modis/i.modis.html
grass-addons/grass7/imagery/i.modis/i.modis.import/
grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.html
grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.py
grass-addons/grass7/raster/r.modis/
grass-addons/grass7/raster/r.modis/Makefile
Removed:
grass-addons/grass7/imagery/i.modis/i.modis.download/r.modis.download.html
grass-addons/grass7/imagery/i.modis/i.modis.download/r.modis.download.py
grass-addons/grass7/imagery/i.modis/i.modis.import/r.modis.import.html
grass-addons/grass7/imagery/i.modis/i.modis.import/r.modis.import.py
grass-addons/grass7/imagery/i.modis/r.modis.download/
grass-addons/grass7/imagery/i.modis/r.modis.html
grass-addons/grass7/imagery/i.modis/r.modis.import/
grass-addons/grass7/raster/r.modis/
Modified:
grass-addons/grass7/imagery/Makefile
grass-addons/grass7/imagery/i.modis/Makefile
grass-addons/grass7/imagery/i.modis/i.modis.download/Makefile
grass-addons/grass7/imagery/i.modis/i.modis.import/Makefile
grass-addons/grass7/imagery/i.modis/libmodis/Makefile
grass-addons/grass7/imagery/i.modis/libmodis/rmodislib.py
grass-addons/grass7/raster/Makefile
Log:
r.modis: renamed to i.modis, see #3522
Modified: grass-addons/grass7/imagery/Makefile
===================================================================
--- grass-addons/grass7/imagery/Makefile 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/Makefile 2018-03-20 15:00:13 UTC (rev 72415)
@@ -18,7 +18,8 @@
i.histo.match \
i.in.probav \
i.lmf \
- i.points.auto \
+ i.modis \
+ i.points.auto \
i.pysptools.unmix \
i.rotate \
i.segment.gsoc \
Modified: grass-addons/grass7/imagery/i.modis/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/Makefile 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/Makefile 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,10 +1,10 @@
MODULE_TOPDIR =../..
-PGM = r.modis
+PGM = i.modis
SUBDIRS = libmodis \
- r.modis.download \
- r.modis.import \
+ i.modis.download \
+ i.modis.import \
include $(MODULE_TOPDIR)/include/Make/Dir.make
Modified: grass-addons/grass7/imagery/i.modis/i.modis.download/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.download/Makefile 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/i.modis.download/Makefile 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,6 +1,6 @@
MODULE_TOPDIR = ../../..
-PGM = r.modis.download
+PGM = i.modis.download
include $(MODULE_TOPDIR)/include/Make/Script.make
@@ -10,4 +10,4 @@
# $(INSTALL) $< $@
#
# $(ETC)/r.modis/r.modis.download:
-# $(MKDIR) $@
\ No newline at end of file
+# $(MKDIR) $@
Copied: grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.html (from rev 72414, grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.html)
===================================================================
--- grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.html (rev 0)
+++ grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.html 2018-03-20 15:00:13 UTC (rev 72415)
@@ -0,0 +1,137 @@
+<h2>DESCRIPTION</h2>
+
+<em>i.modis.download</em> downloads selected MODIS products (Moderate Resolution
+Imaging Spectroradiometer, flown on the two NASA spacecrafts Terra and Aqua). The
+module can download several tiles at once and also multiple observation dates
+for each MODIS product.
+
+<h2>NOTES</h2>
+
+The <em>i.modis</em> modules need the <a href="http://www.pymodis.org">pyModis</a>
+library. Please install it beforehand.
+<p>
+To be able to download data the user needs to obtain <em>user</em> and
+<em>password</em> for the NASA Earthdata Login:
+<ul>
+<li> First time user: The user has to register at
+ <a href="https://urs.earthdata.nasa.gov/users/new">https://urs.earthdata.nasa.gov/users/new</a>;
+ then login and change to his/her profile page. Once there, under the "My applications"
+ tab, the user needs to approve the following applications: "LP DAAC Data Pool"
+ and "Earthdata Search".</li>
+<li> If the user is already registered, he/she just needs to login and enable the
+ aforementioned applications at
+ <a href="https://urs.earthdata.nasa.gov/home">https://urs.earthdata.nasa.gov/home</a>
+ if not already done.</li>
+</ul>
+<p>
+In order to download the desired MODIS product(s), the username and password must be
+provided through the <em>settings</em> option. There are three ways:
+<ul>
+<li>
+using the <a href="https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html" target="_blank">.netrc</a>
+file as showed below:
+<div class="code"><pre>
+machine e4ftl01.cr.usgs.gov
+login your_NASA_username
+password your_NASA_password
+</pre></div>
+</li>
+<li>
+pass a file in which the first row is the username, and the second row is the password, as showed below:
+<div class="code"><pre>
+your_NASA_username
+your_NASA_password
+</pre></div>
+</li>
+<li>
+the user can pass the values from the standard input when prompted.
+</li>
+</ul>
+<p>
+
+<b>Warning</b>: As per NASA policy no more than ten simultaneous connections
+are permitted in parallel with their server. Hence, no more than ten processes
+of <em>i.modis.download</em> should run at the same time to avoid that user's
+actual IP address gets blacklisted.
+<p>
+By default, the downloaded files are stored in the path in which the settings
+file was saved. The user can change this directory with the <em>folder</em>
+option. The <em>folder</em> option is required when user and password are
+passed through the standard input.
+<p>
+The needed time for the download depends on the number of requested tiles, days
+and the type of MODIS product. For example, the NDVI product ranges from 5 MB
+to 270 MB and the LST product ranges from 2 MB to 21 MB.
+
+<h2>EXAMPLES</h2>
+
+Download of the daily MODIS LST product "lst_terra_daily_1000" from the
+Terra satellite using the default options (all available tiles from newest
+available date) and passing the user and password through standard input.
+Note that when settings is read from standard input, the option folder
+must be specified:
+
+<div class="code"><pre>
+i.modis.download settings=- folder=/tmp
+</pre></div>
+
+<p>
+Reading the user and password options from a file (this will download by default
+the "lst_terra_daily_1000" product). MODIS data will be downloaded to
+the folder where the SETTING file is:
+
+<div class="code"><pre>
+i.modis.download settings=$HOME/.grass7/i.modis/SETTING
+</pre></div>
+
+<p>
+Download of the LST Terra product using the default options and change of
+the starting and ending dates to custom values:
+
+<div class="code"><pre>
+i.modis.download settings=$HOME/.grass7/i.modis/SETTING startday=2011-05-01 endday=2011-05-31
+</pre></div>
+
+<p>
+Download of a different product (here: <em>Snow eight days 500 m</em>),
+default options:
+
+<div class="code"><pre>
+i.modis.download settings=$HOME/.grass7/i.modis/SETTING product=snow_terra_eight_500
+</pre></div>
+
+<p>
+Download of a global MODIS product (here: <em>MOD13C1 - MODIS/Terra Vegetation Indices
+16-Day L3 Global 0.05Deg CMG V006</em>), of a specific month:
+
+<div class="code"><pre>
+# note: provided in Geographic Lat/Long Grid
+i.modis.download settings=$HOME/.grass7/i.modis/SETTING product=ndvi_terra_sixteen_5600 startday=2011-05-01 endday=2011-05-31
+</pre></div>
+
+<p>
+To use <em>i.modis.download</em> in a script and to concatenate it with another module,
+the user needs to set the <em>-g</em> flag to return the name of the file that
+contains the list of downloaded HDF files:
+
+<div class="code"><pre>
+i.modis.download -g settings=$HOME/.grass7/i.modis/SETTING startday=2011-05-01 endday=2011-05-31
+</pre></div>
+
+<h2>SEE ALSO</h2>
+
+<em>
+<a href="i.modis.html">i.modis</a>,
+<a href="i.modis.import.html">i.modis.import</a>
+<!-- <a href="i.modis.process.html">i.modis.process</a>,-->
+</em>
+<p>
+<a href="https://grasswiki.osgeo.org/wiki/Temporal_data_processing">GRASS GIS Wiki: temporal data processing</a>
+<p>
+<a href="https://lpdaac.usgs.gov/dataset_discovery/modis">Map of MODIS Land products' Sinusoidal grid tiling system</a>
+
+<h2>AUTHOR</h2>
+
+Luca Delucchi, Google Summer of Code 2011; subsequently updated.
+
+<p><i>Last changed: $Date$</i>
Copied: grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.py (from rev 72414, grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.py)
===================================================================
--- grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.py (rev 0)
+++ grass-addons/grass7/imagery/i.modis/i.modis.download/i.modis.download.py 2018-03-20 15:00:13 UTC (rev 72415)
@@ -0,0 +1,269 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+############################################################################
+#
+# MODULE: i.modis.download
+# AUTHOR(S): Luca Delucchi
+# PURPOSE: i.modis.download is an interface to pyModis for download
+# several tiles of MODIS produts from NASA ftp
+#
+# COPYRIGHT: (C) 2011-2017 by Luca Delucchi
+#
+# This program is free software under the GNU General Public
+# License (>=v2). Read the file COPYING that comes with GRASS
+# for details.
+#
+#############################################################################
+
+#%module
+#% description: Download single or multiple tiles of MODIS products using pyModis.
+#% keyword: raster
+#% keyword: import
+#% keyword: MODIS
+#%end
+#%flag
+#% key: d
+#% description: For debug mode, it will write more info in the log file
+#%end
+#%flag
+#% key: g
+#% description: Return the name of file containing the list of HDF tiles downloaded in shell script style
+#%end
+#%flag
+#% key: c
+#% description: Does not perform GDAL check on downloaded images
+#%end
+#%flag
+#% key: l
+#% description: List more info about the supported MODIS products
+#%end
+#%option G_OPT_F_INPUT
+#% key: settings
+#% label: Full path to settings file or '-' for standard input, empty for .netrc file
+#% required: no
+#% guisection: Define
+#%end
+#%option
+#% key: product
+#% type: string
+#% label: Name of MODIS product(s)
+#% multiple: yes
+#% required: no
+#% options: lst_terra_daily_1000, lst_aqua_daily_1000, lst_terra_eight_1000, lst_aqua_eight_1000, lst_terra_daily_5600, lst_aqua_daily_5600, lst_terra_monthly_5600, lst_aqua_monthly_5600, ndvi_terra_sixteen_250, ndvi_aqua_sixteen_250, ndvi_terra_sixteen_500, ndvi_aqua_sixteen_500, ndvi_terra_sixteen_1000, ndvi_aqua_sixteen_1000, ndvi_terra_sixteen_5600, ndvi_aqua_sixteen_5600, snow_terra_daily_500, snow_aqua_daily_500, snow_terra_eight_500, snow_aqua_eight_500, surfreflec_terra_daily_500, surfreflec_aqua_daily_500, surfreflec_terra_eight_500, surfreflec_aqua_eight_500, water_terra_250
+#% answer: lst_terra_daily_1000
+#%end
+#%option
+#% key: tiles
+#% type: string
+#% label: The name(s) of tile(s) to download (comma separated). If not set, all available tiles are downloaded.
+#% description: e.g.: h18v04
+#% required: no
+#%end
+#%option
+#% key: startday
+#% type: string
+#% label: First date to download
+#% description: Format: YYYY-MM-DD. If not set the download starts from current date and goes back 10 days. If not endday is set, the download stops 10 days after the startday
+#% required: no
+#%end
+#%option
+#% key: endday
+#% type: string
+#% label: Last date to download
+#% description: Format: YYYY-MM-DD. To use only with startday
+#% required: no
+#%end
+#%option
+#% key: folder
+#% type: string
+#% label: Folder to store the downloaded data
+#% description: If not set, path to settings file is used
+#% required: no
+#%end
+
+
+# import library
+import os
+import sys
+from datetime import *
+import grass.script as grass
+from grass.pygrass.utils import get_lib_path
+
+
+path = get_lib_path(modname='i.modis', libname='libmodis')
+if path is None:
+ grass.fatal("Not able to find the modis library directory.")
+sys.path.append(path)
+
+
+def check(home):
+ """ Check if a folder it is writable by the user that launch the process
+ """
+ if os.access(home, os.W_OK):
+ return True
+ else:
+ grass.fatal(_("Folder to write downloaded files does not "
+ "exist or is not writeable"))
+
+
+def checkdate(options):
+ """ Function to check the data and return the correct value to download the
+ the tiles
+ """
+ def check2day(second, first=None):
+ """Function to check two date"""
+ if not first:
+ valueDay = None
+ firstDay = date.today()
+ else:
+ valueDay = first
+ firstSplit = first.split('-')
+ firstDay = date(int(firstSplit[0]), int(firstSplit[1]),
+ int(firstSplit[2]))
+ lastSplit = second.split('-')
+ lastDay = date(int(lastSplit[0]), int(lastSplit[1]), int(lastSplit[2]))
+ if firstDay < lastDay:
+ grass.fatal(_("End day has to be bigger then start day"))
+ delta = firstDay - lastDay
+ valueDelta = int(delta.days)
+ return valueDay, second, valueDelta
+ # no set start and end day
+ if options['startday'] == '' and options['endday'] == '':
+ return None, None, 10
+ # set only end day
+ elif options['startday'] != '' and options['endday'] == '':
+ valueDelta = 10
+ valueEnd = options['startday']
+ firstSplit = valueEnd.split('-')
+ firstDay = date(int(firstSplit[0]), int(firstSplit[1]),
+ int(firstSplit[2]))
+ delta = timedelta(10)
+ lastday = firstDay + delta
+ valueDay = lastday.strftime("%Y-%m-%d")
+ # set only start day
+ elif options['startday'] == '' and options['endday'] != '':
+ grass.fatal(_("It is not possible to use <endday> option without "
+ "<startday> option"))
+ # set start and end day
+ elif options['startday'] != '' and options['endday'] != '':
+ valueDay, valueEnd, valueDelta = check2day(options['startday'],
+ options['endday'])
+ return valueDay, valueEnd, valueDelta
+
+
+# main function
+def main():
+ try:
+ from rmodislib import product
+ except:
+ grass.fatal("i.modis library is not installed")
+ try:
+ from pymodis.downmodis import downModis
+ except:
+ grass.fatal("pymodis library is not installed")
+ # check if you are in GRASS
+ gisbase = os.getenv('GISBASE')
+ if not gisbase:
+ grass.fatal(_('$GISBASE not defined'))
+ return 0
+ if flags['l']:
+ prod = product()
+ prod.print_prods()
+ return 0
+ # set username, password and folder if settings are insert by stdin
+ if not options['settings']:
+ user = None
+ passwd = None
+ if check(options['folder']):
+ fold = options['folder']
+ else:
+ grass.fatal(_("Set folder parameter when using stdin for passing "
+ "the username and password"))
+ elif options['settings'] == '-':
+ if options['folder'] != '':
+ import getpass
+ if check(options['folder']):
+ fold = options['folder']
+ user = raw_input(_('Insert username: '))
+ passwd = getpass.getpass(_('Insert password: '))
+ else:
+ grass.fatal(_("Set folder parameter when using stdin for passing "
+ "the username and password"))
+ # set username, password and folder by file
+ else:
+ # open the file and read the the user and password:
+ # first line is username
+ # second line is password
+ if check(options['settings']):
+ filesett = open(options['settings'], 'r')
+ fileread = filesett.readlines()
+ user = fileread[0].strip()
+ passwd = fileread[1].strip()
+ filesett.close()
+ else:
+ grass.fatal(_("File <%s> not found") % options['settings'])
+ # set the folder by option folder
+ if options['folder'] != '':
+ if check(options['folder']):
+ fold = options['folder']
+ # set the folder from path where settings file is stored
+ else:
+ path = os.path.split(options['settings'])[0]
+ if check(path):
+ fold = path
+ # check the version
+ version = grass.core.version()
+ # this is would be set automatically
+ if version['version'].find('7.') == -1:
+ grass.fatal(_('GRASS GIS version 7 required'))
+ return 0
+ # the product
+ products = options['product'].split(',')
+ # first date and delta
+ firstday, finalday, delta = checkdate(options)
+ # set tiles
+ if options['tiles'] == '':
+ tiles = None
+ grass.warning(_("Option 'tiles' not set. Downloading all available tiles"))
+ else:
+ tiles = options['tiles']
+ # set the debug
+ if flags['d']:
+ debug_opt = True
+ else:
+ debug_opt = False
+ if flags['c']:
+ checkgdal = False
+ else:
+ checkgdal = True
+ for produ in products:
+ prod = product(produ).returned()
+ #start modis class
+ modisOgg = downModis(url=prod['url'], user=user, password=passwd,
+ destinationFolder=fold, tiles=tiles, delta=delta,
+ path=prod['folder'], product=prod['prod'],
+ today=firstday, enddate=finalday, debug=debug_opt,
+ checkgdal=checkgdal)
+ # connect to ftp
+ modisOgg.connect()
+ if modisOgg.nconnection <= 20:
+ # download tha tiles
+ grass.message(_("Downloading MODIS product <%s>..." % produ))
+ modisOgg.downloadsAllDay()
+ filesize = int(os.path.getsize(modisOgg.filelist.name))
+ if flags['g'] and filesize != 0:
+ grass.message("files=%s" % modisOgg.filelist.name)
+ elif filesize == 0:
+ grass.message(_("No data download, probably they have been "
+ "previously downloaded"))
+ elif filesize != 0:
+ grass.message(_("All data have been downloaded, continue "
+ "with i.modis.import with the option "
+ "'files=%s'" % modisOgg.filelist.name))
+ else:
+ grass.fatal(_("Error during connection"))
+
+if __name__ == "__main__":
+ options, flags = grass.parser()
+ sys.exit(main())
Deleted: grass-addons/grass7/imagery/i.modis/i.modis.download/r.modis.download.html
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.html 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/i.modis.download/r.modis.download.html 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,137 +0,0 @@
-<h2>DESCRIPTION</h2>
-
-<em>r.modis.download</em> downloads selected MODIS products (Moderate Resolution
-Imaging Spectroradiometer, flown on the two NASA spacecrafts Terra and Aqua). The
-module can download several tiles at once and also multiple observation dates
-for each MODIS product.
-
-<h2>NOTES</h2>
-
-The <em>r.modis</em> modules need the <a href="http://www.pymodis.org">pyModis</a>
-library. Please install it beforehand.
-<p>
-To be able to download data the user needs to obtain <em>user</em> and
-<em>password</em> for the NASA Earthdata Login:
-<ul>
-<li> First time user: The user has to register at
- <a href="https://urs.earthdata.nasa.gov/users/new">https://urs.earthdata.nasa.gov/users/new</a>;
- then login and change to his/her profile page. Once there, under the "My applications"
- tab, the user needs to approve the following applications: "LP DAAC Data Pool"
- and "Earthdata Search".</li>
-<li> If the user is already registered, he/she just needs to login and enable the
- aforementioned applications at
- <a href="https://urs.earthdata.nasa.gov/home">https://urs.earthdata.nasa.gov/home</a>
- if not already done.</li>
-</ul>
-<p>
-In order to download the desired MODIS product(s), the username and password must be
-provided through the <em>settings</em> option. There are three ways:
-<ul>
-<li>
-using the <a href="https://www.gnu.org/software/inetutils/manual/html_node/The-_002enetrc-file.html" target="_blank">.netrc</a>
-file as showed below:
-<div class="code"><pre>
-machine e4ftl01.cr.usgs.gov
-login your_NASA_username
-password your_NASA_password
-</pre></div>
-</li>
-<li>
-pass a file in which the first row is the username, and the second row is the password, as showed below:
-<div class="code"><pre>
-your_NASA_username
-your_NASA_password
-</pre></div>
-</li>
-<li>
-the user can pass the values from the standard input when prompted.
-</li>
-</ul>
-<p>
-
-<b>Warning</b>: As per NASA policy no more than ten simultaneous connections
-are permitted in parallel with their server. Hence, no more than ten processes
-of <em>r.modis.download</em> should run at the same time to avoid that user's
-actual IP address gets blacklisted.
-<p>
-By default, the downloaded files are stored in the path in which the settings
-file was saved. The user can change this directory with the <em>folder</em>
-option. The <em>folder</em> option is required when user and password are
-passed through the standard input.
-<p>
-The needed time for the download depends on the number of requested tiles, days
-and the type of MODIS product. For example, the NDVI product ranges from 5 MB
-to 270 MB and the LST product ranges from 2 MB to 21 MB.
-
-<h2>EXAMPLES</h2>
-
-Download of the daily MODIS LST product "lst_terra_daily_1000" from the
-Terra satellite using the default options (all available tiles from newest
-available date) and passing the user and password through standard input.
-Note that when settings is read from standard input, the option folder
-must be specified:
-
-<div class="code"><pre>
-r.modis.download settings=- folder=/tmp
-</pre></div>
-
-<p>
-Reading the user and password options from a file (this will download by default
-the "lst_terra_daily_1000" product). MODIS data will be downloaded to
-the folder where the SETTING file is:
-
-<div class="code"><pre>
-r.modis.download settings=$HOME/.grass7/r.modis/SETTING
-</pre></div>
-
-<p>
-Download of the LST Terra product using the default options and change of
-the starting and ending dates to custom values:
-
-<div class="code"><pre>
-r.modis.download settings=$HOME/.grass7/r.modis/SETTING startday=2011-05-01 endday=2011-05-31
-</pre></div>
-
-<p>
-Download of a different product (here: <em>Snow eight days 500 m</em>),
-default options:
-
-<div class="code"><pre>
-r.modis.download settings=$HOME/.grass7/r.modis/SETTING product=snow_terra_eight_500
-</pre></div>
-
-<p>
-Download of a global MODIS product (here: <em>MOD13C1 - MODIS/Terra Vegetation Indices
-16-Day L3 Global 0.05Deg CMG V006</em>), of a specific month:
-
-<div class="code"><pre>
-# note: provided in Geographic Lat/Long Grid
-r.modis.download settings=$HOME/.grass7/r.modis/SETTING product=ndvi_terra_sixteen_5600 startday=2011-05-01 endday=2011-05-31
-</pre></div>
-
-<p>
-To use <em>r.modis.download</em> in a script and to concatenate it with another module,
-the user needs to set the <em>-g</em> flag to return the name of the file that
-contains the list of downloaded HDF files:
-
-<div class="code"><pre>
-r.modis.download -g settings=$HOME/.grass7/r.modis/SETTING startday=2011-05-01 endday=2011-05-31
-</pre></div>
-
-<h2>SEE ALSO</h2>
-
-<em>
-<a href="r.modis.html">r.modis</a>,
-<a href="r.modis.import.html">r.modis.import</a>
-<!-- <a href="r.modis.process.html">r.modis.process</a>,-->
-</em>
-<p>
-<a href="https://grasswiki.osgeo.org/wiki/Temporal_data_processing">GRASS GIS Wiki: temporal data processing</a>
-<p>
-<a href="https://lpdaac.usgs.gov/dataset_discovery/modis">Map of MODIS Land products' Sinusoidal grid tiling system</a>
-
-<h2>AUTHOR</h2>
-
-Luca Delucchi, Google Summer of Code 2011; subsequently updated.
-
-<p><i>Last changed: $Date$</i>
Deleted: grass-addons/grass7/imagery/i.modis/i.modis.download/r.modis.download.py
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.py 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/i.modis.download/r.modis.download.py 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,269 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-############################################################################
-#
-# MODULE: r.in.modis.download
-# AUTHOR(S): Luca Delucchi
-# PURPOSE: r.in.modis.download is an interface to pyModis for download
-# several tiles of MODIS produts from NASA ftp
-#
-# COPYRIGHT: (C) 2011-2017 by Luca Delucchi
-#
-# This program is free software under the GNU General Public
-# License (>=v2). Read the file COPYING that comes with GRASS
-# for details.
-#
-#############################################################################
-
-#%module
-#% description: Download single or multiple tiles of MODIS products using pyModis.
-#% keyword: raster
-#% keyword: import
-#% keyword: MODIS
-#%end
-#%flag
-#% key: d
-#% description: For debug mode, it will write more info in the log file
-#%end
-#%flag
-#% key: g
-#% description: Return the name of file containing the list of HDF tiles downloaded in shell script style
-#%end
-#%flag
-#% key: c
-#% description: Does not perform GDAL check on downloaded images
-#%end
-#%flag
-#% key: l
-#% description: List more info about the supported MODIS products
-#%end
-#%option G_OPT_F_INPUT
-#% key: settings
-#% label: Full path to settings file or '-' for standard input, empty for .netrc file
-#% required: no
-#% guisection: Define
-#%end
-#%option
-#% key: product
-#% type: string
-#% label: Name of MODIS product(s)
-#% multiple: yes
-#% required: no
-#% options: lst_terra_daily_1000, lst_aqua_daily_1000, lst_terra_eight_1000, lst_aqua_eight_1000, lst_terra_daily_5600, lst_aqua_daily_5600, lst_terra_monthly_5600, lst_aqua_monthly_5600, ndvi_terra_sixteen_250, ndvi_aqua_sixteen_250, ndvi_terra_sixteen_500, ndvi_aqua_sixteen_500, ndvi_terra_sixteen_1000, ndvi_aqua_sixteen_1000, ndvi_terra_sixteen_5600, ndvi_aqua_sixteen_5600, snow_terra_daily_500, snow_aqua_daily_500, snow_terra_eight_500, snow_aqua_eight_500, surfreflec_terra_daily_500, surfreflec_aqua_daily_500, surfreflec_terra_eight_500, surfreflec_aqua_eight_500, water_terra_250
-#% answer: lst_terra_daily_1000
-#%end
-#%option
-#% key: tiles
-#% type: string
-#% label: The name(s) of tile(s) to download (comma separated). If not set, all available tiles are downloaded.
-#% description: e.g.: h18v04
-#% required: no
-#%end
-#%option
-#% key: startday
-#% type: string
-#% label: First date to download
-#% description: Format: YYYY-MM-DD. If not set the download starts from current date and goes back 10 days. If not endday is set, the download stops 10 days after the startday
-#% required: no
-#%end
-#%option
-#% key: endday
-#% type: string
-#% label: Last date to download
-#% description: Format: YYYY-MM-DD. To use only with startday
-#% required: no
-#%end
-#%option
-#% key: folder
-#% type: string
-#% label: Folder to store the downloaded data
-#% description: If not set, path to settings file is used
-#% required: no
-#%end
-
-
-# import library
-import os
-import sys
-from datetime import *
-import grass.script as grass
-from grass.pygrass.utils import get_lib_path
-
-
-path = get_lib_path(modname='r.modis', libname='libmodis')
-if path is None:
- grass.fatal("Not able to find the modis library directory.")
-sys.path.append(path)
-
-
-def check(home):
- """ Check if a folder it is writable by the user that launch the process
- """
- if os.access(home, os.W_OK):
- return True
- else:
- grass.fatal(_("Folder to write downloaded files does not "
- "exist or is not writeable"))
-
-
-def checkdate(options):
- """ Function to check the data and return the correct value to download the
- the tiles
- """
- def check2day(second, first=None):
- """Function to check two date"""
- if not first:
- valueDay = None
- firstDay = date.today()
- else:
- valueDay = first
- firstSplit = first.split('-')
- firstDay = date(int(firstSplit[0]), int(firstSplit[1]),
- int(firstSplit[2]))
- lastSplit = second.split('-')
- lastDay = date(int(lastSplit[0]), int(lastSplit[1]), int(lastSplit[2]))
- if firstDay < lastDay:
- grass.fatal(_("End day has to be bigger then start day"))
- delta = firstDay - lastDay
- valueDelta = int(delta.days)
- return valueDay, second, valueDelta
- # no set start and end day
- if options['startday'] == '' and options['endday'] == '':
- return None, None, 10
- # set only end day
- elif options['startday'] != '' and options['endday'] == '':
- valueDelta = 10
- valueEnd = options['startday']
- firstSplit = valueEnd.split('-')
- firstDay = date(int(firstSplit[0]), int(firstSplit[1]),
- int(firstSplit[2]))
- delta = timedelta(10)
- lastday = firstDay + delta
- valueDay = lastday.strftime("%Y-%m-%d")
- # set only start day
- elif options['startday'] == '' and options['endday'] != '':
- grass.fatal(_("It is not possible to use <endday> option without "
- "<startday> option"))
- # set start and end day
- elif options['startday'] != '' and options['endday'] != '':
- valueDay, valueEnd, valueDelta = check2day(options['startday'],
- options['endday'])
- return valueDay, valueEnd, valueDelta
-
-
-# main function
-def main():
- try:
- from rmodislib import product
- except:
- grass.fatal("r.modis library is not installed")
- try:
- from pymodis.downmodis import downModis
- except:
- grass.fatal("pymodis library is not installed")
- # check if you are in GRASS
- gisbase = os.getenv('GISBASE')
- if not gisbase:
- grass.fatal(_('$GISBASE not defined'))
- return 0
- if flags['l']:
- prod = product()
- prod.print_prods()
- return 0
- # set username, password and folder if settings are insert by stdin
- if not options['settings']:
- user = None
- passwd = None
- if check(options['folder']):
- fold = options['folder']
- else:
- grass.fatal(_("Set folder parameter when using stdin for passing "
- "the username and password"))
- elif options['settings'] == '-':
- if options['folder'] != '':
- import getpass
- if check(options['folder']):
- fold = options['folder']
- user = raw_input(_('Insert username: '))
- passwd = getpass.getpass(_('Insert password: '))
- else:
- grass.fatal(_("Set folder parameter when using stdin for passing "
- "the username and password"))
- # set username, password and folder by file
- else:
- # open the file and read the the user and password:
- # first line is username
- # second line is password
- if check(options['settings']):
- filesett = open(options['settings'], 'r')
- fileread = filesett.readlines()
- user = fileread[0].strip()
- passwd = fileread[1].strip()
- filesett.close()
- else:
- grass.fatal(_("File <%s> not found") % options['settings'])
- # set the folder by option folder
- if options['folder'] != '':
- if check(options['folder']):
- fold = options['folder']
- # set the folder from path where settings file is stored
- else:
- path = os.path.split(options['settings'])[0]
- if check(path):
- fold = path
- # check the version
- version = grass.core.version()
- # this is would be set automatically
- if version['version'].find('7.') == -1:
- grass.fatal(_('GRASS GIS version 7 required'))
- return 0
- # the product
- products = options['product'].split(',')
- # first date and delta
- firstday, finalday, delta = checkdate(options)
- # set tiles
- if options['tiles'] == '':
- tiles = None
- grass.warning(_("Option 'tiles' not set. Downloading all available tiles"))
- else:
- tiles = options['tiles']
- # set the debug
- if flags['d']:
- debug_opt = True
- else:
- debug_opt = False
- if flags['c']:
- checkgdal = False
- else:
- checkgdal = True
- for produ in products:
- prod = product(produ).returned()
- #start modis class
- modisOgg = downModis(url=prod['url'], user=user, password=passwd,
- destinationFolder=fold, tiles=tiles, delta=delta,
- path=prod['folder'], product=prod['prod'],
- today=firstday, enddate=finalday, debug=debug_opt,
- checkgdal=checkgdal)
- # connect to ftp
- modisOgg.connect()
- if modisOgg.nconnection <= 20:
- # download tha tiles
- grass.message(_("Downloading MODIS product <%s>..." % produ))
- modisOgg.downloadsAllDay()
- filesize = int(os.path.getsize(modisOgg.filelist.name))
- if flags['g'] and filesize != 0:
- grass.message("files=%s" % modisOgg.filelist.name)
- elif filesize == 0:
- grass.message(_("No data download, probably they have been "
- "previously downloaded"))
- elif filesize != 0:
- grass.message(_("All data have been downloaded, continue "
- "with r.modis.import with the option "
- "'files=%s'" % modisOgg.filelist.name))
- else:
- grass.fatal(_("Error during connection"))
-
-if __name__ == "__main__":
- options, flags = grass.parser()
- sys.exit(main())
Copied: grass-addons/grass7/imagery/i.modis/i.modis.html (from rev 72414, grass-addons/grass7/raster/r.modis/r.modis.html)
===================================================================
--- grass-addons/grass7/imagery/i.modis/i.modis.html (rev 0)
+++ grass-addons/grass7/imagery/i.modis/i.modis.html 2018-03-20 15:00:13 UTC (rev 72415)
@@ -0,0 +1,214 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
+<html>
+<head>
+<title>GRASS GIS manual: r.modis</title>
+<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
+<link rel="stylesheet" href="grassdocs.css" type="text/css">
+</head>
+<body bgcolor="white">
+<div id="container">
+
+<a href="index.html"><img src="grass_logo.png" alt="GRASS logo"></a>
+<hr class="header">
+
+<h2>NAME</h2>
+<em><b>r.modis</b></em> - Toolset for download and processing of MODIS products using pyModis.
+
+<h2>KEYWORDS</h2>
+<a href="raster.html">raster</a>, <a href="topic_import.html">import</a>, <a href="keywords.html#MODIS">MODIS</a>
+
+<!-- meta page description: Toolset for download and processing of MODIS products using pyModis -->
+<h2>DESCRIPTION</h2>
+
+The <em>r.modis</em> suite is a toolset to import MODIS (Moderate
+Resolution Imaging Spectroradiometer, flown on the two NASA spacecrafts
+Terra and Aqua) satellite data into GRASS GIS. It uses the
+<a href="http://www.pymodis.org">pyModis</a> library and additionally
+either the <a href="http://gdal.org/">GDAL library</a> or the
+<a href="https://lpdaac.usgs.gov/tools/modis_reprojection_tool">MODIS Reprojection Tool</a> (MRT)
+software to convert, mosaic and process MODIS data.
+<p>
+The <em>r.modis</em> suite requires the pyModis library and either the GDAL or MRT
+software to be installed.
+<p>
+The <em>r.modis</em> suite offers two modules as interface to MODIS data. Each module
+is dedicated to a specific operation. The module <em>r.modis.download</em>
+is used to download MODIS HDF products from NASA servers. These files can then
+be imported with <em>r.modis.import</em> which supports import of Level 3
+MODIS products as a single image or as a mosaic into GRASS GIS.
+<p>
+Subsequently, the user can create a temporal dataset using <em>t.create</em> and, register
+the maps with <em>t.register</em>. NOTE: In order to work with the temporal framework of
+GRASS GIS the flag <em>w</em> must be set during the import with <em>r.modis.import</em>.
+<p>
+The user can choose from several MODIS products, distributed as single or multiple tiles
+and also ranges of observation days retrieving data from the related NASA servers.
+<!--
+The suite process Level 2 data using the <em>r.modis.process</em> module.
+-->
+The suite imports Level 3 (georeferenced) products either as single images or as mosaics for
+each date.
+
+<h2>Supported MODIS products</h2>
+These products are currently supported:
+
+<h3>MODIS LST - Land Surface Temperature</h3>
+<ul>
+ <li><b>Land Surface Temperature daily 1 Km (Terra/Aqua)</b>: product provides per-pixel temperature
+ and emissivity values in a sequence of swath-based to grid-based global products in Sinusoidal projection.
+ The MODIS/Terra-Aqua LST/E Daily L3 Global 1 km Grid product (MOD11A1/MYD11A1), is tile-based
+ and gridded in the Sinusoidal projection, and produced daily at 1 km spatial resolution.
+ (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod11a1_v006">MOD11A1</a> and
+ <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd11a1_v006">MYD11A1</a>
+ product pages)</li>
+ <li><b>Land Surface Temperature eight day 1 Km (Terra/Aqua)</b>: data are composed from the daily
+ 1-kilometer LST product (MOD11A1/MYD11A1) and stored on a 1-km Sinusoidal grid as the
+ average values of clear-sky LSTs during an 8-day period.<br>
+ MOD11A2/MYD11A2 is comprised of daytime and nighttime LSTs, quality assessment,
+ observation times, view angles, bits of clear sky days and nights, and
+ emissivities estimated in Bands 31 and 32 from land cover types
+ (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod11a2_v006">MOD11A2</a> and
+ <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd11a2_v006">MYD11A2</a>
+ product pages)</li>
+ <li><b>Land Surface Temperature eight day ~6 Km (Terra/Aqua)</b>: products provide per-pixel
+ temperature and emissivity values in a sequence of swath-based to grid-based
+ global products. The MODIS/Terra-Aqua LST/E Daily L3 Global 6 Km Grid
+ (Short name: MOD11B1/MYD11B1), is tile-based and gridded in the Sinusoidal projection,
+ and produced daily at 5600m spatial resolution
+ (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod11b1_v006">MOD11B1</a> and
+ <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd11b1_v006">MYD11B1</a>
+ product pages)</li>
+</ul>
+
+<h3>MODIS VI - Vegetation Indices</h3>
+
+<ul>
+ <li><b>VI sixteen days 250 m (Terra/Aqua)</b>: Global MOD13Q1/MYD13Q1 MODIS vegetation indices
+ are designed to provide consistent spatial and temporal comparisons of vegetation.
+ conditions. Blue, red, and near-infrared reflectances, centered at 469-nanometers,
+ 645-nanometers, and 858-nanometers, respectively, are used to determine
+ the MODIS daily vegetation indices.<br>
+ The MODIS Normalized Difference Vegetation Index (NDVI) complements NOAA's
+ Advanced Very High Resolution Radiometer (AVHRR) NDVI products and provides
+ continuity for time series historical applications. MODIS also includes a
+ new Enhanced Vegetation Index (EVI) that minimizes canopy background variations
+ and maintains sensitivity over dense vegetation conditions. The EVI also
+ uses the blue band to remove residual atmosphere contamination caused by
+ smoke and sub-pixel thin cloud clouds. The MODIS NDVI and EVI products are
+ computed from atmospherically corrected bi-directional surface reflectances
+ that have been masked for water, clouds, heavy aerosols, and cloud shadows.
+ Global MOD13Q1/MYD13Q1 data are provided every 16 days at 250-meter spatial resolution
+ as a gridded level-3 product in the Sinusoidal projection. Lacking a 250m
+ blue band, the EVI algorithm uses the 500m blue band to correct for residual
+ atmospheric effects, with negligible spatial artifacts
+ (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13q1_v006">MOD13Q1</a> and
+ <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13q1_v006">MYD13Q1</a>
+ product pages)</li>
+ <li><b>VI sixteen days 500 m (Terra/Aqua)</b>: Global MOD13A1/MYD13A1 MODIS vegetation indices
+ are designed to provide consistent spatial and temporal comparisons of vegetation
+ conditions. Blue, red, and near-infrared reflectances, centered at
+ 469-nanometers, 645-nanometers, and 858-nanometers, respectively, are used
+ to determine the MODIS daily vegetation indices.<br>
+ The MODIS Normalized Difference Vegetation Index (NDVI) complements NOAA's
+ Advanced Very High Resolution Radiometer (AVHRR) NDVI products provide
+ continuity for time series historical applications. MODIS also includes a
+ new Enhanced Vegetation Index (EVI) that minimizes canopy background
+ variations and maintains sensitivity over dense vegetation conditions.
+ The EVI also uses the blue band to remove residual atmosphere contamination
+ caused by smoke and sub-pixel thin cloud clouds. The MODIS NDVI and EVI
+ products are computed from atmospherically corrected bi-directional surface
+ reflectances that have been masked for water, clouds, heavy aerosols, and
+ cloud shadows.<br>
+ Global MOD13A1/MYD13A1 data are provided every 16 days at 500-meter spatial
+ resolution as a gridded level-3 product in the Sinusoidal projection.
+ Vegetation indices are used for global monitoring of vegetation conditions
+ and are used in products displaying land cover and land cover changes.
+ These data may be used as input for modeling global biogeochemical and
+ hydrologic processes and global and regional climate. These data also may
+ be used for characterizing land surface biophysical properties and processes,
+ including primary production and land cover conversion
+ (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13a1_v006">MOD13A1</a> and
+ <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13a1_v006">MYD13A1</a>
+ product pages)</li>
+ <li><b>VI sixteen days 1 Km (Terra/Aqua)</b>:
+ The MOD13A2 product provides a Vegetation Index (VI) value at a per pixel basis. There
+ are 2 primary vegetation layers. The first is the Normalized Difference Vegetation Index (NDVI)
+ which is referred to as the continuity index to the existing National Oceanic and Atmospheric
+ Administration-Advanced Very High Resolution Radiometer (NOAA-AVHRR) derived NDVI. The second
+ vegetation layer is the Enhanced Vegetation Index (EVI), which has improved sensitivity over
+ high biomass regions. The data are provided at 1000 m resolution as a gridded level-3 product
+ in the Sinusoidal projection
+ (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13a2_v006">MOD13A2</a> and
+ <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13a2_v006">MYD13A2</a>
+ product pages)</li>
+ <li><b>VI sixteen days Global 0.05Deg CMG (Terra/Aqua)</b>:
+ The MOD13C1 product provides a Vegetation Index (VI) value at a per pixel basis. There
+ are 2 primary vegetation layers. The first is the Normalized Difference Vegetation Index (NDVI)
+ which is referred to as the continuity index to the existing National Oceanic and Atmospheric
+ Administration-Advanced Very High Resolution Radiometer (NOAA-AVHRR) derived NDVI. The second
+ vegetation layer is the Enhanced Vegetation Index (EVI), which has improved sensitivity over
+ high biomass regions. The Climate Modeling Grid (CMG) consists 3600 rows and 7200 columns
+ of 5600 m pixels and is provided as a global latitude/longitude grid
+ (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13c1_v006">MOD13C1</a> and
+ <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13c1_v006">MYD13C1</a>
+ product pages)</li>
+</ul>
+
+<h3>MODIS Snow</h3>
+<ul>
+ <li><b>Snow eight days 500 m (Terra/Aqua)</b>: The MOD10A2 and MYD10A2 products
+ are composites of eight days of snow maps in the sinusoidal grid.
+ An eight-day compositing period was chosen because that is the exact ground
+ track repeat period of the Terra and Aqua platforms. Snow cover over eight
+ days is mapped as maximum snow extent in one SDS and as a chronology of
+ observations in the other SDS. Eight-day periods begin on the first day of
+ the year and extend into the next year. The product can be produced with
+ two to eight days of input. There may not always be eight days of input,
+ because of various reasons, so the user should check the attributes to
+ determine on what days observations were obtained. See the validation webpage
+ for details on the validation and validation definitions
+ (related <a href="https://nsidc.org/data/MOD10A2">MOD10A2</a> and
+ <a href="https://nsidc.org/data/MYD10A2">MYD10A2</a>
+ product pages)</li>
+ <li><b>Snow daily 500 m (Terra/Aqua)</b>: MOD10A1 and MYD10A1 are tiles of daily
+ snow cover at 500 m spatial resolution. The daily observation selected from
+ multiple observations in a MOD10A1 (or MYD10A1) cell is the observation
+ acquired nearest nadir and having the greatest coverage of the grid cell.
+ The daily MOD10A1 and MYD10A1 snow products are tiles of data gridded in the
+ sinusoidal projection. Tiles are approximately 1200 x 1200 km in area. A
+ single scientific data set (SDS) of snow cover and a single SDS of QA data
+ along with local and global attributes comprise the data product file. The
+ daily level 3 snow product is the result of selecting an observation from
+ the multiple observations mapped to a cell of the MOD10_L2G (or MYD10_L2G)
+ product. See the validation webpage for details on the validation and
+ validation definitions
+ (related <a href="https://nsidc.org/data/MOD10A1">MOD10A1</a> and
+ <a href="https://nsidc.org/data/MYD10A1">MYD10A1</a>
+ product pages)</li>
+</ul>
+
+<h2>NOTES</h2>
+
+The <em>r.modis</em> modules need the <a href="http://www.pymodis.org">pyModis</a>
+library. Please install it beforehand.
+
+<h2>SEE ALSO</h2>
+
+<em>
+<a href="r.modis.import.html">r.modis.import</a>,
+<a href="r.modis.download.html">r.modis.download</a>
+<!-- <a href="r.modis.process.html">r.modis.process</a>,-->
+</em>
+<p>
+<ul>
+ <li><a href="https://lpdaac.usgs.gov/tools/modis_reprojection_tool">MODIS Reprojection Tool</a></li>
+ <li> <a href="http://modis-land.gsfc.nasa.gov/">MODIS Land homepage</a></li>
+ <li> <a href="http://modis-snow-ice.gsfc.nasa.gov/">MODIS Snow homepage</a></li>
+ <li> <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table">MODIS Land products table</a></li>
+</ul>
+
+<h2>AUTHOR</h2>
+
+Luca Delucchi, Initial version: Google Summer of Code 2011; subsequently updated
+
+<p><i>Last changed: $Date$</i>
Modified: grass-addons/grass7/imagery/i.modis/i.modis.import/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.import/Makefile 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/i.modis.import/Makefile 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,6 +1,6 @@
MODULE_TOPDIR = ../../..
-PGM = r.modis.import
+PGM = i.modis.import
include $(MODULE_TOPDIR)/include/Make/Script.make
@@ -10,4 +10,4 @@
# $(INSTALL) $< $@
#
# $(ETC)/r.modis/r.modis.download:
-# $(MKDIR) $@
\ No newline at end of file
+# $(MKDIR) $@
Copied: grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.html (from rev 72414, grass-addons/grass7/raster/r.modis/r.modis.import/r.modis.import.html)
===================================================================
--- grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.html (rev 0)
+++ grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.html 2018-03-20 15:00:13 UTC (rev 72415)
@@ -0,0 +1,198 @@
+<h2>DESCRIPTION</h2>
+
+<em>i.modis.import</em> imports Level-3 MODIS (Moderate Resolution Imaging
+Spectroradiometer, flown on the two NASA spacecrafts Terra and Aqua)
+products into GRASS GIS.
+
+<h2>NOTES</h2>
+
+The <em>i.modis</em> modules need the <a href="http://www.pymodis.org">pyModis</a>
+library. Please install it beforehand.
+
+<p>
+The input file is given as a list of full paths to the MODIS HDF files,
+one per line. The input file(s) have to be inside the folder where
+the HDF files are stored.
+<p>
+<b>If <em>mrtpath</em> is not used, pyModis will use GDAL to convert HDF
+files to TIF (which is faster).</b>
+<p>
+The <em>mrtpath</em> option is the path to the main folder of the
+MODIS Reprojection Tools (MRT) binaries, i.e. the folder which contains
+the bin/ and the data/ folder, since these two folders are essential for
+obtaining a successful result.
+<p>
+
+<b>Warning</b>:
+<ul>
+ <li>When using the MODIS Reprojection Tools to convert HDF files, only
+ the following projection systems are supported: Lambert Azimuthal Equal Area,
+ Lambert Conformal Conic, Mercator, Polar Stereographic, Transverse Mercator,
+ Universal Transverse Mercator.</li>
+ <li>Using GDAL allows to handle all projections supported by Proj4.</li>
+</ul>
+
+<p>
+NOTE: In order to work with the temporal framework of GRASS GIS the flag
+<em>w</em> must be set during the import with <em>i.modis.import</em>.
+
+<h2>EXAMPLES</h2>
+
+<h3>General examples</h3>
+
+Import of a single file with all the subsets (QA layers included) using GDAL:
+
+<div class="code"><pre>
+i.modis.import input=/path/to/file
+</pre></div>
+
+<p>
+Import of files from a list with all the subsets using MRT (if mrtpath is not
+provided, GDAL is used):
+
+<div class="code"><pre>
+i.modis.import files=/path/to/listfile mrtpath=/path/to/mrt
+</pre></div>
+
+<p>
+Import of files from a list as mosaics per date without QA layers using MRT:
+
+<div class="code"><pre>
+i.modis.import -mq files=/path/to/listfile mrtpath=/path/to/mrt
+</pre></div>
+
+<p>
+Import of a single file with user-specific subset of layers using GDAL:
+
+<div class="code"><pre>
+i.modis.import input=/path/to/file spectral="( 1 0 1 0 )"
+</pre></div>
+
+<p>
+Import of files from a list with user-specific subset of layers and without QA
+layer using MRT:
+
+<div class="code"><pre>
+i.modis.import -q files=/path/to/listfile mrtpath=/path/to/mrt spectral="( 1 )"
+</pre></div>
+
+<p>
+Import of a single subset of layers (i.e.: spectral="( 1 )") from each file of a
+list and write an <em>outfile</em> to be used with <em>t.register</em> to assign
+timestamps to maps in the temporal database and register them in a spacetime
+dataset. This option uses GDAL:
+
+<div class="code"><pre>
+i.modis.import -wq files=/path/to/listfile spectral="( 1 )" outfile=/path/to/list_for_tregister.csv
+</pre></div>
+
+<h3>Import of global MODIS NDVI data</h3>
+
+The MOD13C1 is a global NDVI/EVI 16 days map product which can be downloaded
+and imported as follows in a latitude-longitude GRASS GIS location:
+
+<div class="code"><pre>
+# download the two years worth of data
+i.modis.download settings=~/.rmodis product=ndvi_terra_sixteen_5600 \
+ startday=2015-01-01 endday=2016-12-31 folder=$USER/data/ndvi_MOD13C1.006
+# import band 1 = NDVI
+i.modis.import files=$USER/data/ndvi_MOD13C1.006/listfileMOD13C1.006.txt spectral="( 1 )" \
+ method=bilinear outfile=$USER/data/ndvi_MOD13C1.006/list_for_tregister.csv -w
+# create empty temporal DB
+t.create type=strds temporaltype=absolute output=ndvi_16_5600m title="Global NDVI 16 days MOD13C1" \
+ description="MOD13C1 Global NDVI 16 days" semantictype=mean
+# register maps within spacetime datasets (the file name is provided by
+# i.modis.import using -w flag and outfile option)
+t.register input=ndvi_16_5600m file=$USER/data/ndvi_MOD13C1.006/list_for_tregister.csv
+
+# verify and visualize timeline
+t.rast.list ndvi_16_5600m
+g.gui.timeline ndvi_16_5600m
+</pre></div>
+
+<h3>Example of a complete workflow</h3>
+
+Download the data: MOD11A1 from 2016-12-23 to 2016-12-31, tiles h18v04,h18v05
+
+<div class="code"><pre>
+i.modis.download settings=$HOME/SETTING product=lst_terra_eight_1000 \
+ tiles=h18v04,h18v05 startday=2016-12-23 endday=2016-12-31
+</pre></div>
+
+Import mosaics of LST Day and QC Day bands for all dates in the list of files:
+
+<div class="code"><pre>
+i.modis.import -m files=$HOME/listfileMOD11A1.006.txt \
+ spectral="( 1 1 0 0 0 0 0 0 0 0 0 0 )"
+</pre></div>
+
+Extract and apply the mandatory QA band (for more details see
+<a href="i.modis.qc.html">i.modis.qc</a>):
+
+<div class="code"><pre>
+for map in `g.list type=raster pattern="*_QC_Day"` ; do
+ i.modis.qc input=${map} output=${map}_mandatory_qa \
+ productname=mod11A1 qcname=mandatory_qa_11A1
+done
+
+for m in `g.list rast pat=*2016*LST_Day_1km` ; do
+ # get name of product and date from filenames
+ i=`echo $m | cut -c 1-16`
+ # apply qa flags
+ r.mapcalc --o expression="${m} = if(${i}_mosaic_QC_Day_mandatory_qa < 2, ${m}, null())"
+done
+</pre></div>
+
+Create the time series (i.e.: spacetime dataset) and register maps in it:
+
+<div class="code"><pre>
+t.create type=strds temporaltype=absolute output=LST_Day_daily \
+ title="Daily LST Day 1km" \
+ description="Daily LST Day 1km MOD11A1.006, December 2016"
+t.register -i input=LST_Day_daily \
+ maps=`g.list type=raster pattern="*2016*LST_Day_1km" separator=comma` \
+ start="2016-12-23" increment="1 day"
+</pre></div>
+
+Verify list of maps and dates and visualize timeline:
+
+<div class="code"><pre>
+t.rast.list LST_Day_daily
+g.gui.timeline LST_Day_daily
+</pre></div>
+
+<p>
+It is also possible to create a time series using the list of maps with start and
+end time written by <em>i.modis.import</em> with the <em>w</em> flag and outfile option.
+
+<div class="code"><pre>
+# Import mosaics of LST Day for all dates using the list of downloaded files from
+# i.modis.download (see above) and get a list of the imported files along with
+# dates to use with t.register in the temporal framework
+i.modis.import -mw files=$HOME/listfileMOD11A1.006.txt \
+ spectral="( 1 0 0 0 0 0 0 0 0 0 0 0 )" outfile=$HOME/list_for_tregister.csv
+
+# Create time series and register maps
+t.create type=strds temporaltype=absolute output=LST_Day_daily \
+ title="Daily LST Day 1km" \
+ description="Daily LST Day 1km MOD11A1.006, December 2016"
+t.register input=LST_Day_daily file=$HOME/list_for_tregister.csv
+</pre></div>
+
+<h2>SEE ALSO</h2>
+
+<em>
+<a href="i.modis.html">i.modis</a>,
+<a href="i.modis.download.html">i.modis.download</a>,
+<a href="i.modis.qc.html">i.modis.qc</a>
+</em>
+<p>
+<a href="https://grasswiki.osgeo.org/wiki/Temporal_data_processing">GRASS GIS Wiki: temporal data processing</a>
+<p>
+<a href="https://lpdaac.usgs.gov/dataset_discovery/modis">Map of MODIS Land products' Sinusoidal grid tiling system</a>
+
+<h2>AUTHOR</h2>
+
+Luca Delucchi, Google Summer of Code 2011; subsequently updated.
+
+<p><i>Last changed: $Date$</i>
Copied: grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.py (from rev 72414, grass-addons/grass7/raster/r.modis/r.modis.import/r.modis.import.py)
===================================================================
--- grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.py (rev 0)
+++ grass-addons/grass7/imagery/i.modis/i.modis.import/i.modis.import.py 2018-03-20 15:00:13 UTC (rev 72415)
@@ -0,0 +1,607 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+############################################################################
+#
+# MODULE: i.modis.import
+# AUTHOR(S): Luca Delucchi
+# PURPOSE: i.modis.import is an interface to pyModis for import into
+# GRASS GIS level 3 MODIS produts
+#
+# COPYRIGHT: (C) 2011-2017 by Luca Delucchi
+#
+# This program is free software under the GNU General Public
+# License (>=v2). Read the file COPYING that comes with GRASS
+# for details.
+#
+#############################################################################
+#
+# REQUIREMENTS:
+# - either MRT tools, https://lpdaac.usgs.gov/lpdaac/tools/modis_reprojection_tool
+# - or GDAL
+#
+#############################################################################
+
+#%module
+#% description: Import single or multiple tiles of MODIS products using pyModis.
+#% keyword: raster
+#% keyword: import
+#% keyword: MODIS
+#%end
+#%flag
+#% key: m
+#% description: Create a mosaic for each date
+#%end
+#%flag
+#% key: t
+#% description: Preserve temporary files (TIF and HDF mosaic)
+#%end
+#%flag
+#% key: q
+#% description: Ignore the QA map layer
+#%end
+#%flag
+#% key: w
+#% description: Create a text file to use with t.register
+#%end
+#%flag
+#% key: l
+#% description: List more info about the supported MODIS products
+#%end
+#%flag
+#% key: g
+#% description: Print output message in shell script style
+#%end
+#%option
+#% key: input
+#% type: string
+#% key_desc: path
+#% description: Full path to single HDF file
+#% gisprompt: old,file,input
+#% required: no
+#%end
+#%option G_OPT_F_INPUT
+#% key: files
+#% description: Full path to file with list of HDF files
+#% required: no
+#%end
+#%option
+#% key: method
+#% type: string
+#% key_desc: resampling
+#% description: Name of spatial resampling method
+#% options: nearest, bilinear, cubic
+#% answer: nearest
+#% required: no
+#%end
+#%option
+#% key: mrtpath
+#% type: string
+#% key_desc: path
+#% description: Full path to MRT directory
+#% gisprompt: old,dir,input
+#% required: no
+#%end
+#%option
+#% key: spectral
+#% type: string
+#% key_desc: spectral subset
+#% description: String of the form "( 1 0 1 0 )" to choose a subset of HDF layers to import
+#% required: no
+#%end
+#%option G_OPT_F_OUTPUT
+#% key: outfile
+#% description: Full path to output file to use with t.register
+#% required: no
+#%end
+
+import os
+import sys
+import string
+import glob
+import shutil
+import grass.script as grass
+from datetime import datetime
+from datetime import timedelta
+from grass.pygrass.utils import get_lib_path
+import tempfile
+path = get_lib_path(modname='i.modis', libname='libmodis')
+if path is None:
+ grass.fatal("Not able to find the modis library directory.")
+sys.path.append(path)
+
+
+class grassParseModis:
+ """Class to reproduce parseModis class when VRT is used for mosaic
+
+ :param str filename: the name of MODIS hdf file
+ """
+
+ def __init__(self, filename, date):
+ self.hdfname = filename
+ self.date = date
+
+ def retRangeTime(self):
+ return {'RangeBeginningDate': self.date}
+
+
+def list_files(opt, mosaik=False):
+ """If used in function single(): Return a list of HDF files from the file
+ list. If used in function mosaic(): Return a dictionary with a list of HDF
+ files for each day
+ """
+ # read the file with the list of HDF
+ if opt['files'] != '':
+ if os.path.exists(opt['files']):
+ listoffile = open(opt['files'], 'r')
+ basedir = os.path.split(listoffile.name)[0]
+ else:
+ grass.fatal(_("File {name} does not exist".format(name=opt['files'])))
+ # if mosaic create a dictionary
+ if mosaik:
+ filelist = {}
+ # if not mosaic create a list
+ else:
+ filelist = []
+ # append hdf files
+ for line in listoffile:
+ if string.find(line, 'xml') == -1 and mosaik is False:
+ filelist.append(line.strip())
+ # for mosaic create a list of hdf files for each day
+ elif string.find(line, 'xml') == -1 and mosaik is True:
+ day = line.split('/')[-1].split('.')[1]
+ if day in filelist:
+ filelist[day].append(line.strip())
+ else:
+ filelist[day] = [line.strip()]
+ # create a list for each file
+ elif options['input'] != '':
+ filelist = [options['input']]
+ basedir = os.path.split(filelist[0])[0]
+ return filelist, basedir
+
+
+def spectral(opts, prod, q, m=False):
+ """Return spectral string"""
+ # return the spectral set selected by the user
+ if opts['spectral'] != '':
+ spectr = opts['spectral']
+ # return the spectral by default
+ else:
+ if q:
+ if prod['spec_qa']:
+ spectr = prod['spec_qa']
+ else:
+ spectr = prod['spec']
+ else:
+ spectr = prod['spec']
+ if m:
+ spectr = spectr.replace(' 0', '')
+ return spectr
+
+
+def confile(pm, opts, q, mosaik=False):
+ """Create the configuration file for MRT software"""
+ try:
+ # try to import pymodis (modis) and some classes for i.modis.download
+ from rmodislib import resampling, product, projection
+ except:
+ grass.fatal("i.modis library is not installed")
+ # return projection and datum
+ projObj = projection()
+ proj = projObj.returned()
+ dat = projObj.datum()
+ if proj == 'UTM':
+ zone = projObj.utmzone()
+ else:
+ zone = None
+ cod = os.path.split(pm.hdfname)[1].split('.')[0]
+ prod = product().fromcode(cod)
+ if mosaik:
+ # if mosaic it remove all the 0 from the subset string to convert all
+ # the right layer
+ spectr = spectral(opts, prod, q, True)
+ else:
+ spectr = spectral(opts, prod, q)
+ # out prefix
+ pref = modis_prefix(pm.hdfname)
+ # resampling
+ resampl = resampling(opts['method']).returned()
+ # projpar
+ projpar = projObj.return_params()
+ if projpar != "( 0, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 )":
+ dat = "NODATUM"
+ # resolution
+ if proj != 'GEO':
+ res = int(prod['res']) * int(projObj.proj['meters'])
+ else:
+ res = None
+ try:
+ conf = pm.confResample(spectr, res, pref, dat, resampl,
+ proj, zone, projpar)
+ return conf
+ except IOError, e:
+ grass.fatal(e)
+
+
+def metadata(pars, mapp):
+ """ Set metadata to the imported files """
+ # metadata
+ grass.run_command('r.support', quiet=True, map=mapp, source1="MODIS NASA",
+ hist="Imported with i.modis.import")
+ # timestamp
+ rangetime = pars.retRangeTime()
+ data = rangetime['RangeBeginningDate'].split('-')
+ dataobj = datetime(int(data[0]), int(data[1]), int(data[2]))
+ grass.run_command('r.timestamp', map=mapp, quiet=True,
+ date=dataobj.strftime("%d %b %Y"))
+ return dataobj
+ # color
+# if string.find(mapp, 'QC') != -1 or string.find(mapp, 'Quality') != -1 or \
+# string.find(mapp, 'QA') != -1:
+# grass.run_command('r.colors', quiet=True, map=mapp, color=coll)
+# elif string.find(mapp, 'NDVI') != -1:
+# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[0])
+# elif string.find(mapp, 'EVI') != -1:
+# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[1])
+# elif string.find(mapp, 'LST') != -1:
+# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[0])
+# elif string.find(mapp, 'Snow') != -1:
+# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[0])
+
+
+def modis_prefix(inp, mosaic=False):
+ """return the modis prefix"""
+ modlist = os.path.split(inp)[1].split('.')
+ if mosaic:
+ return '.'.join(modlist[:2])
+ else:
+ return '.'.join(modlist[:3])
+
+
+def import_tif(basedir, rem, write, pm, prod, target=None, listfile=None):
+ """Import TIF files"""
+ # list of tif files
+ pref = modis_prefix(pm.hdfname)
+ tifiles = glob.glob1(basedir, "{pr}*.tif".format(pr=pref))
+ if not tifiles:
+ tifiles = glob.glob1(os.getcwd(), "{pr}*.tif".format(pr=pref))
+ if not tifiles:
+ grass.fatal(_('Error during the conversion'))
+ outfile = []
+ # for each file import it
+ for t in tifiles:
+ basename = os.path.splitext(t)[0]
+ basename = basename.replace(' ', '_')
+ name = os.path.join(basedir, t)
+ if not os.path.exists(name):
+ name = os.path.join(os.getcwd(), t)
+ if not os.path.exists(name):
+ grass.warning(_("File %s doesn't find" % name))
+ continue
+ filesize = int(os.path.getsize(name))
+ if filesize < 1000:
+ grass.warning(_('Probably some error occur during the conversion'
+ 'for file <%s>. Escape import' % name))
+ continue
+ try:
+ grass.run_command('r.in.gdal', input=name, output=basename,
+ overwrite=write, quiet=True)
+ outfile.append(basename)
+ except:
+ grass.warning(_('Error during import of %s' % basename))
+ continue
+ data = metadata(pm, basename)
+ if rem:
+ os.remove(name)
+ if target:
+ if target != basedir:
+ shutil.move(name, target)
+ if listfile:
+ days = prod['days']
+ fdata = data + timedelta(days)
+ if days == 31:
+ fdata = datetime(fdata.year, fdata.month, 1)
+ listfile.write("{name}|{sd}|{fd}\n".format(name=basename,
+ sd=data.strftime("%Y-%m-%d"),
+ fd=fdata.strftime("%Y-%m-%d")))
+ return outfile
+
+
+def findfile(pref, suff):
+ """ Check if a file exists on mapset """
+ if grass.find_file(pref + suff)['file']:
+ return grass.find_file(pref + suff)
+ else:
+ grass.warning(_("Raster map <%s> not found") % (pref + suff))
+
+
+def doy2date(modis):
+ """From he MODIS code to YYYY-MM-DD string date"""
+ year = modis[:4]
+ doy = modis[-3:]
+ dat = datetime.strptime('{ye} {doy}'.format(ye=year, doy=doy), '%Y %j')
+ return dat.strftime('%Y-%m-%d')
+
+
+def single(options, remove, an, ow, fil):
+ """Convert the HDF file to TIF and import it
+ """
+ try:
+ # try to import pymodis (modis) and some classes for i.modis.download
+ from rmodislib import product, projection, get_proj
+ except:
+ grass.fatal("i.modis library is not installed")
+ try:
+ from pymodis.convertmodis import convertModis
+ from pymodis.convertmodis_gdal import convertModisGDAL
+ from pymodis.parsemodis import parseModis
+ except:
+ grass.fatal("pymodis library is not installed")
+ listfile, basedir = list_files(options)
+ # for each file
+ for i in listfile:
+ if os.path.exists(i):
+ hdf = i
+ else:
+ # the full path to hdf file
+ hdf = os.path.join(basedir, i)
+ if not os.path.exists(hdf):
+ grass.warning(_("%s not found" % i))
+ continue
+ pm = parseModis(hdf)
+ if options['mrtpath']:
+ # create conf file fro mrt tools
+ confname = confile(pm, options, an)
+ # create convertModis class and convert it in tif file
+ execmodis = convertModis(hdf, confname, options['mrtpath'])
+ else:
+ projwkt = get_proj('w')
+ projObj = projection()
+ pref = i.split('/')[-1]
+ prod = product().fromcode(pref.split('.')[0])
+ spectr = spectral(options, prod, an)
+ if projObj.returned() != 'GEO':
+ res = int(prod['res']) * int(projObj.proj['meters'])
+ else:
+ res = None
+ outname = "%s.%s.%s.single" % (pref.split('.')[0],
+ pref.split('.')[1],
+ pref.split('.')[2])
+ outname = outname.replace(' ', '_')
+ execmodis = convertModisGDAL(hdf, outname, spectr, res,
+ wkt=projwkt)
+ try:
+ execmodis.run(quiet=True)
+ except:
+ execmodis.run()
+ import_tif(basedir=basedir, rem=remove, write=ow, pm=pm,
+ listfile=fil, prod=prod)
+ if options['mrtpath']:
+ os.remove(confname)
+
+
+def mosaic(options, remove, an, ow, fil):
+ """Create a daily mosaic of HDF files convert to TIF and import it
+ """
+ try:
+ # try to import pymodis (modis) and some classes for i.modis.download
+ from rmodislib import product, projection, get_proj
+ except:
+ grass.fatal("i.modis library is not installed")
+ try:
+ from pymodis.convertmodis import convertModis, createMosaic
+ from pymodis.convertmodis_gdal import createMosaicGDAL, convertModisGDAL
+ from pymodis.parsemodis import parseModis
+ except:
+ grass.fatal("pymodis library is not installed")
+ dictfile, targetdir = list_files(options, True)
+ pid = str(os.getpid())
+ # for each day
+ for dat, listfiles in dictfile.iteritems():
+ pref = listfiles[0].split('/')[-1]
+ prod = product().fromcode(pref.split('.')[0])
+ spectr = spectral(options, prod, an)
+ spectr = spectr.lstrip('( ').rstrip(' )')
+ outname = "%s.%s_mosaic" % (pref.split('.')[0], pref.split('.')[1])
+ outname = outname.replace(' ', '_')
+ # create mosaic
+ if options['mrtpath']:
+ # create the file with the list of name
+ tempfile = open(os.path.join(targetdir, pid), 'w')
+ tempfile.writelines(listfiles)
+ tempfile.close()
+ # basedir of tempfile, where hdf files are write
+ basedir = os.path.split(tempfile.name)[0]
+ # return the spectral subset in according mrtmosaic tool format
+ cm = createMosaic(tempfile.name, outname, options['mrtpath'],
+ spectr)
+ cm.run()
+ hdfiles = glob.glob1(basedir, outname + "*.hdf")
+ else:
+ basedir = targetdir
+ listfiles = [os.path.join(basedir, i) for i in listfiles]
+ cm = createMosaicGDAL(listfiles, spectr)
+ try:
+ cm.write_vrt(os.path.join(basedir,outname), quiet=True)
+ except:
+ cm.write_vrt(os.path.join(basedir,outname))
+ hdfiles = glob.glob1(basedir, outname + "*.vrt")
+ for i in hdfiles:
+ # the full path to hdf file
+ hdf = os.path.join(basedir, i)
+ try:
+ pm = parseModis(hdf)
+ except:
+ out = i.replace('.vrt', '')
+ data = doy2date(dat[1:])
+ pm = grassParseModis(out, data)
+ # create convertModis class and convert it in tif file
+ if options['mrtpath']:
+ # create conf file fro mrt tools
+ confname = confile(pm, options, an, True)
+ execmodis = convertModis(hdf, confname, options['mrtpath'])
+ else:
+ confname = None
+ projwkt = get_proj('w')
+ projObj = projection()
+ if projObj.returned() != 'GEO':
+ res = int(prod['res']) * int(projObj.proj['meters'])
+ else:
+ res = None
+ execmodis = convertModisGDAL(hdf, out, spectr, res, wkt=projwkt,
+ vrt=True)
+ try:
+ execmodis.run(quiet=True)
+ except:
+ execmodis.run()
+ # remove hdf
+ if remove:
+ # import tif files
+ import_tif(basedir=basedir, rem=remove, write=ow,
+ pm=pm, listfile=fil, prod=prod)
+ try:
+ os.remove(hdf)
+ os.remove(hdf + '.xml')
+ except OSError:
+ pass
+ # move the hdf and hdf.xml to the dir where are the original files
+ else:
+ # import tif files
+ import_tif(basedir=basedir, rem=remove, write=ow,
+ pm=pm, target=targetdir, listfile=fil, prod=prod)
+ if i not in os.listdir(targetdir):
+ try:
+ shutil.move(hdf, targetdir)
+ shutil.move(hdf + '.xml', targetdir)
+ except OSError:
+ pass
+ # remove the conf file
+ try:
+ os.remove(confname)
+ except (OSError, TypeError) as e:
+ pass
+ if options['mrtpath']:
+ grass.try_remove(tempfile.name)
+ grass.try_remove(os.path.join(targetdir, 'mosaic', pid))
+
+
+def main():
+ # check if you are in GRASS
+ gisbase = os.getenv('GISBASE')
+ if not gisbase:
+ grass.fatal(_('$GISBASE not defined'))
+ return 0
+ if flags['l']:
+ try:
+ from rmodislib import product
+ except:
+ grass.fatal("i.modis library is not installed")
+ prod = product()
+ prod.print_prods()
+ return 0
+ # return an error if q and spectral are set
+ if not flags['q'] and options['spectral'] != '':
+ grass.warning(_('If no QA layer chosen in the "spectral" option'
+ ' the command will report an error'))
+ # return an error if both input and files option are set or not
+ if options['input'] == '' and options['files'] == '':
+ grass.fatal(_('Choose one of "input" or "files" options'))
+ return 0
+ elif options['input'] != '' and options['files'] != '':
+ grass.fatal(_('It is not possible set "input" and "files"'
+ ' options together'))
+ return 0
+ # check the version
+ version = grass.core.version()
+ # this is would be set automatically
+ if version['version'].find('7.') == -1:
+ grass.fatal(_('GRASS GIS version 7 required'))
+ return 0
+ # check if remove the files or not
+ if flags['t']:
+ remove = False
+ else:
+ remove = True
+ if grass.overwrite():
+ over = True
+ else:
+ over = False
+ # check if do check quality, rescaling and setting of colors
+ if flags['q']:
+ analyze = False
+ else:
+ analyze = True
+ # return the number of select layer from HDF files
+ if options['spectral']:
+ count = options['spectral'].strip('(').strip(')').split().count('1')
+ else:
+ count = 0
+
+ outfile = None
+ # check if file for t.register has to been created
+ if options['outfile']:
+ outfile = open(options['outfile'], 'w')
+ if count > 1:
+ grass.warning("The spectral subsets are more than one so the "
+ " output file will be renamed")
+ elif flags['w'] and not options['outfile']:
+ outfile = tempfile.NamedTemporaryFile(delete=False)
+
+ # check if import simple file or mosaic
+ if flags['m'] and options['input'] != '':
+ grass.fatal(_('It is not possible to create a mosaic with a single'
+ ' HDF file'))
+ return 0
+ elif flags['m']:
+ mosaic(options, remove, analyze, over, outfile)
+ else:
+ single(options, remove, analyze, over, outfile)
+ # if t.register file is create
+ if outfile:
+ outfile.close()
+ tempdir = grass.tempdir()
+ # one layer only
+ if count == 1:
+ if flags['g']:
+ grass.message(_("file={name}".format(name=outfile.name)))
+ else:
+ grass.message(_("You can use temporal framework, registering"
+ " the maps using t.register input=your_strds "
+ "'file={name}'".format(name=outfile.name)))
+ # for more layer create several files with only a subset for each layer
+ elif count > 1:
+ tfile = open(outfile.name)
+ outfiles = {}
+ lines = tfile.readlines()
+ # get the codes from only one HDF
+ for line in lines[:count]:
+ if flags['m']:
+ code = '_'.join(line.split('|')[0].split('_')[2:])
+ else:
+ code = line.split('|')[0].split('.')[-1]
+ outfiles[code] = open(os.path.join(tempdir, "{co}.txt".format(co=code)), 'w')
+ # split the lines for each code
+ for line in lines:
+ if flags['m']:
+ code = '_'.join(line.split('|')[0].split('_')[2:])
+ else:
+ code = line.split('|')[0].split('.')[-1]
+ outfiles[code].write(line)
+ for k, v in outfiles.items():
+ v.close()
+ if flags['g']:
+ message = ""
+ else:
+ message = "You can use temporal framework, registering the " \
+ "maps in different temporal datasets using " \
+ "t.register and \n"
+ tfile.close()
+ for fil in outfiles.values():
+ message += "'file={name}'\n".format(name=fil.name)
+ grass.message(_(message))
+
+
+if __name__ == "__main__":
+ options, flags = grass.parser()
+ sys.exit(main())
Deleted: grass-addons/grass7/imagery/i.modis/i.modis.import/r.modis.import.html
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.import/r.modis.import.html 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/i.modis.import/r.modis.import.html 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,199 +0,0 @@
-<h2>DESCRIPTION</h2>
-
-<em>r.modis.import</em> imports Level-3 MODIS (Moderate Resolution Imaging
-Spectroradiometer, flown on the two NASA spacecrafts Terra and Aqua)
-products into GRASS GIS.
-
-<h2>NOTES</h2>
-
-The <em>r.modis</em> modules need the <a href="http://www.pymodis.org">pyModis</a>
-library. Please install it beforehand.
-
-<p>
-The input file is given as a list of full paths to the MODIS HDF files,
-one per line. The input file(s) have to be inside the folder where
-the HDF files are stored.
-<p>
-<b>If <em>mrtpath</em> is not used, pyModis will use GDAL to convert HDF
-files to TIF (which is faster).</b>
-<p>
-The <em>mrtpath</em> option is the path to the main folder of the
-MODIS Reprojection Tools (MRT) binaries, i.e. the folder which contains
-the bin/ and the data/ folder, since these two folders are essential for
-obtaining a successful result.
-<p>
-
-<b>Warning</b>:
-<ul>
- <li>When using the MODIS Reprojection Tools to convert HDF files, only
- the following projection systems are supported: Lambert Azimuthal Equal Area,
- Lambert Conformal Conic, Mercator, Polar Stereographic, Transverse Mercator,
- Universal Transverse Mercator.</li>
- <li>Using GDAL allows to handle all projections supported by Proj4.</li>
-</ul>
-
-<p>
-NOTE: In order to work with the temporal framework of GRASS GIS the flag
-<em>w</em> must be set during the import with <em>r.modis.import</em>.
-
-<h2>EXAMPLES</h2>
-
-<h3>General examples</h3>
-
-Import of a single file with all the subsets (QA layers included) using GDAL:
-
-<div class="code"><pre>
-r.modis.import input=/path/to/file
-</pre></div>
-
-<p>
-Import of files from a list with all the subsets using MRT (if mrtpath is not
-provided, GDAL is used):
-
-<div class="code"><pre>
-r.modis.import files=/path/to/listfile mrtpath=/path/to/mrt
-</pre></div>
-
-<p>
-Import of files from a list as mosaics per date without QA layers using MRT:
-
-<div class="code"><pre>
-r.modis.import -mq files=/path/to/listfile mrtpath=/path/to/mrt
-</pre></div>
-
-<p>
-Import of a single file with user-specific subset of layers using GDAL:
-
-<div class="code"><pre>
-r.modis.import input=/path/to/file spectral="( 1 0 1 0 )"
-</pre></div>
-
-<p>
-Import of files from a list with user-specific subset of layers and without QA
-layer using MRT:
-
-<div class="code"><pre>
-r.modis.import -q files=/path/to/listfile mrtpath=/path/to/mrt spectral="( 1 )"
-</pre></div>
-
-<p>
-Import of a single subset of layers (i.e.: spectral="( 1 )") from each file of a
-list and write an <em>outfile</em> to be used with <em>t.register</em> to assign
-timestamps to maps in the temporal database and register them in a spacetime
-dataset. This option uses GDAL:
-
-<div class="code"><pre>
-r.modis.import -wq files=/path/to/listfile spectral="( 1 )" outfile=/path/to/list_for_tregister.csv
-</pre></div>
-
-<h3>Import of global MODIS NDVI data</h3>
-
-The MOD13C1 is a global NDVI/EVI 16 days map product which can be downloaded
-and imported as follows in a latitude-longitude GRASS GIS location:
-
-<div class="code"><pre>
-# download the two years worth of data
-r.modis.download settings=~/.rmodis product=ndvi_terra_sixteen_5600 \
- startday=2015-01-01 endday=2016-12-31 folder=$USER/data/ndvi_MOD13C1.006
-# import band 1 = NDVI
-r.modis.import files=$USER/data/ndvi_MOD13C1.006/listfileMOD13C1.006.txt spectral="( 1 )" \
- method=bilinear outfile=$USER/data/ndvi_MOD13C1.006/list_for_tregister.csv -w
-# create empty temporal DB
-t.create type=strds temporaltype=absolute output=ndvi_16_5600m title="Global NDVI 16 days MOD13C1" \
- description="MOD13C1 Global NDVI 16 days" semantictype=mean
-# register maps within spacetime datasets (the file name is provided by
-# r.modis.import using -w flag and outfile option)
-t.register input=ndvi_16_5600m file=$USER/data/ndvi_MOD13C1.006/list_for_tregister.csv
-
-# verify and visualize timeline
-t.rast.list ndvi_16_5600m
-g.gui.timeline ndvi_16_5600m
-</pre></div>
-
-<h3>Example of a complete workflow</h3>
-
-Download the data: MOD11A1 from 2016-12-23 to 2016-12-31, tiles h18v04,h18v05
-
-<div class="code"><pre>
-r.modis.download settings=$HOME/SETTING product=lst_terra_eight_1000 \
- tiles=h18v04,h18v05 startday=2016-12-23 endday=2016-12-31
-</pre></div>
-
-Import mosaics of LST Day and QC Day bands for all dates in the list of files:
-
-<div class="code"><pre>
-r.modis.import -m files=$HOME/listfileMOD11A1.006.txt \
- spectral="( 1 1 0 0 0 0 0 0 0 0 0 0 )"
-</pre></div>
-
-Extract and apply the mandatory QA band (for more details see
-<a href="i.modis.qc.html">i.modis.qc</a>):
-
-<div class="code"><pre>
-for map in `g.list type=raster pattern="*_QC_Day"` ; do
- i.modis.qc input=${map} output=${map}_mandatory_qa \
- productname=mod11A1 qcname=mandatory_qa_11A1
-done
-
-for m in `g.list rast pat=*2016*LST_Day_1km` ; do
- # get name of product and date from filenames
- i=`echo $m | cut -c 1-16`
- # apply qa flags
- r.mapcalc --o expression="${m} = if(${i}_mosaic_QC_Day_mandatory_qa < 2, ${m}, null())"
-done
-</pre></div>
-
-Create the time series (i.e.: spacetime dataset) and register maps in it:
-
-<div class="code"><pre>
-t.create type=strds temporaltype=absolute output=LST_Day_daily \
- title="Daily LST Day 1km" \
- description="Daily LST Day 1km MOD11A1.006, December 2016"
-t.register -i input=LST_Day_daily \
- maps=`g.list type=raster pattern="*2016*LST_Day_1km" separator=comma` \
- start="2016-12-23" increment="1 day"
-</pre></div>
-
-Verify list of maps and dates and visualize timeline:
-
-<div class="code"><pre>
-t.rast.list LST_Day_daily
-g.gui.timeline LST_Day_daily
-</pre></div>
-
-<p>
-It is also possible to create a time series using the list of maps with start and
-end time written by <em>r.modis.import</em> with the <em>w</em> flag and outfile option.
-
-<div class="code"><pre>
-# Import mosaics of LST Day for all dates using the list of downloaded files from
-# r.modis.download (see above) and get a list of the imported files along with
-# dates to use with t.register in the temporal framework
-r.modis.import -mw files=$HOME/listfileMOD11A1.006.txt \
- spectral="( 1 0 0 0 0 0 0 0 0 0 0 0 )" outfile=$HOME/list_for_tregister.csv
-
-# Create time series and register maps
-t.create type=strds temporaltype=absolute output=LST_Day_daily \
- title="Daily LST Day 1km" \
- description="Daily LST Day 1km MOD11A1.006, December 2016"
-t.register input=LST_Day_daily file=$HOME/list_for_tregister.csv
-</pre></div>
-
-<h2>SEE ALSO</h2>
-
-<em>
-<a href="r.modis.html">r.modis</a>,
-<a href="r.modis.download.html">r.modis.download</a>,
-<a href="i.modis.qc.html">i.modis.qc</a>
-<!-- <a href="r.modis.process.html">r.modis.process</a>,-->
-</em>
-<p>
-<a href="https://grasswiki.osgeo.org/wiki/Temporal_data_processing">GRASS GIS Wiki: temporal data processing</a>
-<p>
-<a href="https://lpdaac.usgs.gov/dataset_discovery/modis">Map of MODIS Land products' Sinusoidal grid tiling system</a>
-
-<h2>AUTHOR</h2>
-
-Luca Delucchi, Google Summer of Code 2011; subsequently updated.
-
-<p><i>Last changed: $Date$</i>
Deleted: grass-addons/grass7/imagery/i.modis/i.modis.import/r.modis.import.py
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.import/r.modis.import.py 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/i.modis.import/r.modis.import.py 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,607 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-############################################################################
-#
-# MODULE: r.modis.import
-# AUTHOR(S): Luca Delucchi
-# PURPOSE: r.modis.import is an interface to pyModis for import into
-# GRASS GIS level 3 MODIS produts
-#
-# COPYRIGHT: (C) 2011-2017 by Luca Delucchi
-#
-# This program is free software under the GNU General Public
-# License (>=v2). Read the file COPYING that comes with GRASS
-# for details.
-#
-#############################################################################
-#
-# REQUIREMENTS:
-# - either MRT tools, https://lpdaac.usgs.gov/lpdaac/tools/modis_reprojection_tool
-# - or GDAL
-#
-#############################################################################
-
-#%module
-#% description: Import single or multiple tiles of MODIS products using pyModis.
-#% keyword: raster
-#% keyword: import
-#% keyword: MODIS
-#%end
-#%flag
-#% key: m
-#% description: Create a mosaic for each date
-#%end
-#%flag
-#% key: t
-#% description: Preserve temporary files (TIF and HDF mosaic)
-#%end
-#%flag
-#% key: q
-#% description: Ignore the QA map layer
-#%end
-#%flag
-#% key: w
-#% description: Create a text file to use with t.register
-#%end
-#%flag
-#% key: l
-#% description: List more info about the supported MODIS products
-#%end
-#%flag
-#% key: g
-#% description: Print output message in shell script style
-#%end
-#%option
-#% key: input
-#% type: string
-#% key_desc: path
-#% description: Full path to single HDF file
-#% gisprompt: old,file,input
-#% required: no
-#%end
-#%option G_OPT_F_INPUT
-#% key: files
-#% description: Full path to file with list of HDF files
-#% required: no
-#%end
-#%option
-#% key: method
-#% type: string
-#% key_desc: resampling
-#% description: Name of spatial resampling method
-#% options: nearest, bilinear, cubic
-#% answer: nearest
-#% required: no
-#%end
-#%option
-#% key: mrtpath
-#% type: string
-#% key_desc: path
-#% description: Full path to MRT directory
-#% gisprompt: old,dir,input
-#% required: no
-#%end
-#%option
-#% key: spectral
-#% type: string
-#% key_desc: spectral subset
-#% description: String of the form "( 1 0 1 0 )" to choose a subset of HDF layers to import
-#% required: no
-#%end
-#%option G_OPT_F_OUTPUT
-#% key: outfile
-#% description: Full path to output file to use with t.register
-#% required: no
-#%end
-
-import os
-import sys
-import string
-import glob
-import shutil
-import grass.script as grass
-from datetime import datetime
-from datetime import timedelta
-from grass.pygrass.utils import get_lib_path
-import tempfile
-path = get_lib_path(modname='r.modis', libname='libmodis')
-if path is None:
- grass.fatal("Not able to find the modis library directory.")
-sys.path.append(path)
-
-
-class grassParseModis:
- """Class to reproduce parseModis class when VRT is used for mosaic
-
- :param str filename: the name of MODIS hdf file
- """
-
- def __init__(self, filename, date):
- self.hdfname = filename
- self.date = date
-
- def retRangeTime(self):
- return {'RangeBeginningDate': self.date}
-
-
-def list_files(opt, mosaik=False):
- """If used in function single(): Return a list of HDF files from the file
- list. If used in function mosaic(): Return a dictionary with a list of HDF
- files for each day
- """
- # read the file with the list of HDF
- if opt['files'] != '':
- if os.path.exists(opt['files']):
- listoffile = open(opt['files'], 'r')
- basedir = os.path.split(listoffile.name)[0]
- else:
- grass.fatal(_("File {name} does not exist".format(name=opt['files'])))
- # if mosaic create a dictionary
- if mosaik:
- filelist = {}
- # if not mosaic create a list
- else:
- filelist = []
- # append hdf files
- for line in listoffile:
- if string.find(line, 'xml') == -1 and mosaik is False:
- filelist.append(line.strip())
- # for mosaic create a list of hdf files for each day
- elif string.find(line, 'xml') == -1 and mosaik is True:
- day = line.split('/')[-1].split('.')[1]
- if day in filelist:
- filelist[day].append(line.strip())
- else:
- filelist[day] = [line.strip()]
- # create a list for each file
- elif options['input'] != '':
- filelist = [options['input']]
- basedir = os.path.split(filelist[0])[0]
- return filelist, basedir
-
-
-def spectral(opts, prod, q, m=False):
- """Return spectral string"""
- # return the spectral set selected by the user
- if opts['spectral'] != '':
- spectr = opts['spectral']
- # return the spectral by default
- else:
- if q:
- if prod['spec_qa']:
- spectr = prod['spec_qa']
- else:
- spectr = prod['spec']
- else:
- spectr = prod['spec']
- if m:
- spectr = spectr.replace(' 0', '')
- return spectr
-
-
-def confile(pm, opts, q, mosaik=False):
- """Create the configuration file for MRT software"""
- try:
- # try to import pymodis (modis) and some classes for r.modis.download
- from rmodislib import resampling, product, projection
- except:
- grass.fatal("r.modis library is not installed")
- # return projection and datum
- projObj = projection()
- proj = projObj.returned()
- dat = projObj.datum()
- if proj == 'UTM':
- zone = projObj.utmzone()
- else:
- zone = None
- cod = os.path.split(pm.hdfname)[1].split('.')[0]
- prod = product().fromcode(cod)
- if mosaik:
- # if mosaic it remove all the 0 from the subset string to convert all
- # the right layer
- spectr = spectral(opts, prod, q, True)
- else:
- spectr = spectral(opts, prod, q)
- # out prefix
- pref = modis_prefix(pm.hdfname)
- # resampling
- resampl = resampling(opts['method']).returned()
- # projpar
- projpar = projObj.return_params()
- if projpar != "( 0, 0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0 )":
- dat = "NODATUM"
- # resolution
- if proj != 'GEO':
- res = int(prod['res']) * int(projObj.proj['meters'])
- else:
- res = None
- try:
- conf = pm.confResample(spectr, res, pref, dat, resampl,
- proj, zone, projpar)
- return conf
- except IOError, e:
- grass.fatal(e)
-
-
-def metadata(pars, mapp):
- """ Set metadata to the imported files """
- # metadata
- grass.run_command('r.support', quiet=True, map=mapp, source1="MODIS NASA",
- hist="Imported with r.modis.import")
- # timestamp
- rangetime = pars.retRangeTime()
- data = rangetime['RangeBeginningDate'].split('-')
- dataobj = datetime(int(data[0]), int(data[1]), int(data[2]))
- grass.run_command('r.timestamp', map=mapp, quiet=True,
- date=dataobj.strftime("%d %b %Y"))
- return dataobj
- # color
-# if string.find(mapp, 'QC') != -1 or string.find(mapp, 'Quality') != -1 or \
-# string.find(mapp, 'QA') != -1:
-# grass.run_command('r.colors', quiet=True, map=mapp, color=coll)
-# elif string.find(mapp, 'NDVI') != -1:
-# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[0])
-# elif string.find(mapp, 'EVI') != -1:
-# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[1])
-# elif string.find(mapp, 'LST') != -1:
-# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[0])
-# elif string.find(mapp, 'Snow') != -1:
-# grass.run_command('r.colors', quiet=True, map=mapp, color=coll[0])
-
-
-def modis_prefix(inp, mosaic=False):
- """return the modis prefix"""
- modlist = os.path.split(inp)[1].split('.')
- if mosaic:
- return '.'.join(modlist[:2])
- else:
- return '.'.join(modlist[:3])
-
-
-def import_tif(basedir, rem, write, pm, prod, target=None, listfile=None):
- """Import TIF files"""
- # list of tif files
- pref = modis_prefix(pm.hdfname)
- tifiles = glob.glob1(basedir, "{pr}*.tif".format(pr=pref))
- if not tifiles:
- tifiles = glob.glob1(os.getcwd(), "{pr}*.tif".format(pr=pref))
- if not tifiles:
- grass.fatal(_('Error during the conversion'))
- outfile = []
- # for each file import it
- for t in tifiles:
- basename = os.path.splitext(t)[0]
- basename = basename.replace(' ', '_')
- name = os.path.join(basedir, t)
- if not os.path.exists(name):
- name = os.path.join(os.getcwd(), t)
- if not os.path.exists(name):
- grass.warning(_("File %s doesn't find" % name))
- continue
- filesize = int(os.path.getsize(name))
- if filesize < 1000:
- grass.warning(_('Probably some error occur during the conversion'
- 'for file <%s>. Escape import' % name))
- continue
- try:
- grass.run_command('r.in.gdal', input=name, output=basename,
- overwrite=write, quiet=True)
- outfile.append(basename)
- except:
- grass.warning(_('Error during import of %s' % basename))
- continue
- data = metadata(pm, basename)
- if rem:
- os.remove(name)
- if target:
- if target != basedir:
- shutil.move(name, target)
- if listfile:
- days = prod['days']
- fdata = data + timedelta(days)
- if days == 31:
- fdata = datetime(fdata.year, fdata.month, 1)
- listfile.write("{name}|{sd}|{fd}\n".format(name=basename,
- sd=data.strftime("%Y-%m-%d"),
- fd=fdata.strftime("%Y-%m-%d")))
- return outfile
-
-
-def findfile(pref, suff):
- """ Check if a file exists on mapset """
- if grass.find_file(pref + suff)['file']:
- return grass.find_file(pref + suff)
- else:
- grass.warning(_("Raster map <%s> not found") % (pref + suff))
-
-
-def doy2date(modis):
- """From he MODIS code to YYYY-MM-DD string date"""
- year = modis[:4]
- doy = modis[-3:]
- dat = datetime.strptime('{ye} {doy}'.format(ye=year, doy=doy), '%Y %j')
- return dat.strftime('%Y-%m-%d')
-
-
-def single(options, remove, an, ow, fil):
- """Convert the HDF file to TIF and import it
- """
- try:
- # try to import pymodis (modis) and some classes for r.modis.download
- from rmodislib import product, projection, get_proj
- except:
- grass.fatal("r.modis library is not installed")
- try:
- from pymodis.convertmodis import convertModis
- from pymodis.convertmodis_gdal import convertModisGDAL
- from pymodis.parsemodis import parseModis
- except:
- grass.fatal("pymodis library is not installed")
- listfile, basedir = list_files(options)
- # for each file
- for i in listfile:
- if os.path.exists(i):
- hdf = i
- else:
- # the full path to hdf file
- hdf = os.path.join(basedir, i)
- if not os.path.exists(hdf):
- grass.warning(_("%s not found" % i))
- continue
- pm = parseModis(hdf)
- if options['mrtpath']:
- # create conf file fro mrt tools
- confname = confile(pm, options, an)
- # create convertModis class and convert it in tif file
- execmodis = convertModis(hdf, confname, options['mrtpath'])
- else:
- projwkt = get_proj('w')
- projObj = projection()
- pref = i.split('/')[-1]
- prod = product().fromcode(pref.split('.')[0])
- spectr = spectral(options, prod, an)
- if projObj.returned() != 'GEO':
- res = int(prod['res']) * int(projObj.proj['meters'])
- else:
- res = None
- outname = "%s.%s.%s.single" % (pref.split('.')[0],
- pref.split('.')[1],
- pref.split('.')[2])
- outname = outname.replace(' ', '_')
- execmodis = convertModisGDAL(hdf, outname, spectr, res,
- wkt=projwkt)
- try:
- execmodis.run(quiet=True)
- except:
- execmodis.run()
- import_tif(basedir=basedir, rem=remove, write=ow, pm=pm,
- listfile=fil, prod=prod)
- if options['mrtpath']:
- os.remove(confname)
-
-
-def mosaic(options, remove, an, ow, fil):
- """Create a daily mosaic of HDF files convert to TIF and import it
- """
- try:
- # try to import pymodis (modis) and some classes for r.modis.download
- from rmodislib import product, projection, get_proj
- except:
- grass.fatal("r.modis library is not installed")
- try:
- from pymodis.convertmodis import convertModis, createMosaic
- from pymodis.convertmodis_gdal import createMosaicGDAL, convertModisGDAL
- from pymodis.parsemodis import parseModis
- except:
- grass.fatal("pymodis library is not installed")
- dictfile, targetdir = list_files(options, True)
- pid = str(os.getpid())
- # for each day
- for dat, listfiles in dictfile.iteritems():
- pref = listfiles[0].split('/')[-1]
- prod = product().fromcode(pref.split('.')[0])
- spectr = spectral(options, prod, an)
- spectr = spectr.lstrip('( ').rstrip(' )')
- outname = "%s.%s_mosaic" % (pref.split('.')[0], pref.split('.')[1])
- outname = outname.replace(' ', '_')
- # create mosaic
- if options['mrtpath']:
- # create the file with the list of name
- tempfile = open(os.path.join(targetdir, pid), 'w')
- tempfile.writelines(listfiles)
- tempfile.close()
- # basedir of tempfile, where hdf files are write
- basedir = os.path.split(tempfile.name)[0]
- # return the spectral subset in according mrtmosaic tool format
- cm = createMosaic(tempfile.name, outname, options['mrtpath'],
- spectr)
- cm.run()
- hdfiles = glob.glob1(basedir, outname + "*.hdf")
- else:
- basedir = targetdir
- listfiles = [os.path.join(basedir, i) for i in listfiles]
- cm = createMosaicGDAL(listfiles, spectr)
- try:
- cm.write_vrt(os.path.join(basedir,outname), quiet=True)
- except:
- cm.write_vrt(os.path.join(basedir,outname))
- hdfiles = glob.glob1(basedir, outname + "*.vrt")
- for i in hdfiles:
- # the full path to hdf file
- hdf = os.path.join(basedir, i)
- try:
- pm = parseModis(hdf)
- except:
- out = i.replace('.vrt', '')
- data = doy2date(dat[1:])
- pm = grassParseModis(out, data)
- # create convertModis class and convert it in tif file
- if options['mrtpath']:
- # create conf file fro mrt tools
- confname = confile(pm, options, an, True)
- execmodis = convertModis(hdf, confname, options['mrtpath'])
- else:
- confname = None
- projwkt = get_proj('w')
- projObj = projection()
- if projObj.returned() != 'GEO':
- res = int(prod['res']) * int(projObj.proj['meters'])
- else:
- res = None
- execmodis = convertModisGDAL(hdf, out, spectr, res, wkt=projwkt,
- vrt=True)
- try:
- execmodis.run(quiet=True)
- except:
- execmodis.run()
- # remove hdf
- if remove:
- # import tif files
- import_tif(basedir=basedir, rem=remove, write=ow,
- pm=pm, listfile=fil, prod=prod)
- try:
- os.remove(hdf)
- os.remove(hdf + '.xml')
- except OSError:
- pass
- # move the hdf and hdf.xml to the dir where are the original files
- else:
- # import tif files
- import_tif(basedir=basedir, rem=remove, write=ow,
- pm=pm, target=targetdir, listfile=fil, prod=prod)
- if i not in os.listdir(targetdir):
- try:
- shutil.move(hdf, targetdir)
- shutil.move(hdf + '.xml', targetdir)
- except OSError:
- pass
- # remove the conf file
- try:
- os.remove(confname)
- except (OSError, TypeError) as e:
- pass
- if options['mrtpath']:
- grass.try_remove(tempfile.name)
- grass.try_remove(os.path.join(targetdir, 'mosaic', pid))
-
-
-def main():
- # check if you are in GRASS
- gisbase = os.getenv('GISBASE')
- if not gisbase:
- grass.fatal(_('$GISBASE not defined'))
- return 0
- if flags['l']:
- try:
- from rmodislib import product
- except:
- grass.fatal("r.modis library is not installed")
- prod = product()
- prod.print_prods()
- return 0
- # return an error if q and spectral are set
- if not flags['q'] and options['spectral'] != '':
- grass.warning(_('If no QA layer chosen in the "spectral" option'
- ' the command will report an error'))
- # return an error if both input and files option are set or not
- if options['input'] == '' and options['files'] == '':
- grass.fatal(_('Choose one of "input" or "files" options'))
- return 0
- elif options['input'] != '' and options['files'] != '':
- grass.fatal(_('It is not possible set "input" and "files"'
- ' options together'))
- return 0
- # check the version
- version = grass.core.version()
- # this is would be set automatically
- if version['version'].find('7.') == -1:
- grass.fatal(_('GRASS GIS version 7 required'))
- return 0
- # check if remove the files or not
- if flags['t']:
- remove = False
- else:
- remove = True
- if grass.overwrite():
- over = True
- else:
- over = False
- # check if do check quality, rescaling and setting of colors
- if flags['q']:
- analyze = False
- else:
- analyze = True
- # return the number of select layer from HDF files
- if options['spectral']:
- count = options['spectral'].strip('(').strip(')').split().count('1')
- else:
- count = 0
-
- outfile = None
- # check if file for t.register has to been created
- if options['outfile']:
- outfile = open(options['outfile'], 'w')
- if count > 1:
- grass.warning("The spectral subsets are more than one so the "
- " output file will be renamed")
- elif flags['w'] and not options['outfile']:
- outfile = tempfile.NamedTemporaryFile(delete=False)
-
- # check if import simple file or mosaic
- if flags['m'] and options['input'] != '':
- grass.fatal(_('It is not possible to create a mosaic with a single'
- ' HDF file'))
- return 0
- elif flags['m']:
- mosaic(options, remove, analyze, over, outfile)
- else:
- single(options, remove, analyze, over, outfile)
- # if t.register file is create
- if outfile:
- outfile.close()
- tempdir = grass.tempdir()
- # one layer only
- if count == 1:
- if flags['g']:
- grass.message(_("file={name}".format(name=outfile.name)))
- else:
- grass.message(_("You can use temporal framework, registering"
- " the maps using t.register input=your_strds "
- "'file={name}'".format(name=outfile.name)))
- # for more layer create several files with only a subset for each layer
- elif count > 1:
- tfile = open(outfile.name)
- outfiles = {}
- lines = tfile.readlines()
- # get the codes from only one HDF
- for line in lines[:count]:
- if flags['m']:
- code = '_'.join(line.split('|')[0].split('_')[2:])
- else:
- code = line.split('|')[0].split('.')[-1]
- outfiles[code] = open(os.path.join(tempdir, "{co}.txt".format(co=code)), 'w')
- # split the lines for each code
- for line in lines:
- if flags['m']:
- code = '_'.join(line.split('|')[0].split('_')[2:])
- else:
- code = line.split('|')[0].split('.')[-1]
- outfiles[code].write(line)
- for k, v in outfiles.items():
- v.close()
- if flags['g']:
- message = ""
- else:
- message = "You can use temporal framework, registering the " \
- "maps in different temporal datasets using " \
- "t.register and \n"
- tfile.close()
- for fil in outfiles.values():
- message += "'file={name}'\n".format(name=fil.name)
- grass.message(_(message))
-
-
-if __name__ == "__main__":
- options, flags = grass.parser()
- sys.exit(main())
Modified: grass-addons/grass7/imagery/i.modis/libmodis/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/libmodis/Makefile 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/libmodis/Makefile 2018-03-20 15:00:13 UTC (rev 72415)
@@ -5,7 +5,7 @@
MODULES = rmodislib __init__
-ETCDIR = $(ETC)/r.modis
+ETCDIR = $(ETC)/i.modis
PYFILES := $(patsubst %,$(ETCDIR)/%.py,$(MODULES))
PYCFILES := $(patsubst %,$(ETCDIR)/%.pyc,$(MODULES))
Modified: grass-addons/grass7/imagery/i.modis/libmodis/rmodislib.py
===================================================================
--- grass-addons/grass7/raster/r.modis/libmodis/rmodislib.py 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/libmodis/rmodislib.py 2018-03-20 15:00:13 UTC (rev 72415)
@@ -3,9 +3,9 @@
############################################################################
#
-# MODULE: r.modis
+# MODULE: i.modis
# AUTHOR(S): Luca Delucchi
-# PURPOSE: Here there are some important class to run r.modis modules
+# PURPOSE: Here there are some important class to run i.modis modules
#
#
# COPYRIGHT: (C) 2011-2017 by Luca Delucchi
Deleted: grass-addons/grass7/imagery/i.modis/r.modis.html
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.html 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/imagery/i.modis/r.modis.html 2018-03-20 15:00:13 UTC (rev 72415)
@@ -1,214 +0,0 @@
-<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
-<html>
-<head>
-<title>GRASS GIS manual: r.modis</title>
-<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
-<link rel="stylesheet" href="grassdocs.css" type="text/css">
-</head>
-<body bgcolor="white">
-<div id="container">
-
-<a href="index.html"><img src="grass_logo.png" alt="GRASS logo"></a>
-<hr class="header">
-
-<h2>NAME</h2>
-<em><b>r.modis</b></em> - Toolset for download and processing of MODIS products using pyModis.
-
-<h2>KEYWORDS</h2>
-<a href="raster.html">raster</a>, <a href="topic_import.html">import</a>, <a href="keywords.html#MODIS">MODIS</a>
-
-<!-- meta page description: Toolset for download and processing of MODIS products using pyModis -->
-<h2>DESCRIPTION</h2>
-
-The <em>r.modis</em> suite is a toolset to import MODIS (Moderate
-Resolution Imaging Spectroradiometer, flown on the two NASA spacecrafts
-Terra and Aqua) satellite data into GRASS GIS. It uses the
-<a href="http://www.pymodis.org">pyModis</a> library and additionally
-either the <a href="http://gdal.org/">GDAL library</a> or the
-<a href="https://lpdaac.usgs.gov/tools/modis_reprojection_tool">MODIS Reprojection Tool</a> (MRT)
-software to convert, mosaic and process MODIS data.
-<p>
-The <em>r.modis</em> suite requires the pyModis library and either the GDAL or MRT
-software to be installed.
-<p>
-The <em>r.modis</em> suite offers two modules as interface to MODIS data. Each module
-is dedicated to a specific operation. The module <em>r.modis.download</em>
-is used to download MODIS HDF products from NASA servers. These files can then
-be imported with <em>r.modis.import</em> which supports import of Level 3
-MODIS products as a single image or as a mosaic into GRASS GIS.
-<p>
-Subsequently, the user can create a temporal dataset using <em>t.create</em> and, register
-the maps with <em>t.register</em>. NOTE: In order to work with the temporal framework of
-GRASS GIS the flag <em>w</em> must be set during the import with <em>r.modis.import</em>.
-<p>
-The user can choose from several MODIS products, distributed as single or multiple tiles
-and also ranges of observation days retrieving data from the related NASA servers.
-<!--
-The suite process Level 2 data using the <em>r.modis.process</em> module.
--->
-The suite imports Level 3 (georeferenced) products either as single images or as mosaics for
-each date.
-
-<h2>Supported MODIS products</h2>
-These products are currently supported:
-
-<h3>MODIS LST - Land Surface Temperature</h3>
-<ul>
- <li><b>Land Surface Temperature daily 1 Km (Terra/Aqua)</b>: product provides per-pixel temperature
- and emissivity values in a sequence of swath-based to grid-based global products in Sinusoidal projection.
- The MODIS/Terra-Aqua LST/E Daily L3 Global 1 km Grid product (MOD11A1/MYD11A1), is tile-based
- and gridded in the Sinusoidal projection, and produced daily at 1 km spatial resolution.
- (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod11a1_v006">MOD11A1</a> and
- <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd11a1_v006">MYD11A1</a>
- product pages)</li>
- <li><b>Land Surface Temperature eight day 1 Km (Terra/Aqua)</b>: data are composed from the daily
- 1-kilometer LST product (MOD11A1/MYD11A1) and stored on a 1-km Sinusoidal grid as the
- average values of clear-sky LSTs during an 8-day period.<br>
- MOD11A2/MYD11A2 is comprised of daytime and nighttime LSTs, quality assessment,
- observation times, view angles, bits of clear sky days and nights, and
- emissivities estimated in Bands 31 and 32 from land cover types
- (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod11a2_v006">MOD11A2</a> and
- <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd11a2_v006">MYD11A2</a>
- product pages)</li>
- <li><b>Land Surface Temperature eight day ~6 Km (Terra/Aqua)</b>: products provide per-pixel
- temperature and emissivity values in a sequence of swath-based to grid-based
- global products. The MODIS/Terra-Aqua LST/E Daily L3 Global 6 Km Grid
- (Short name: MOD11B1/MYD11B1), is tile-based and gridded in the Sinusoidal projection,
- and produced daily at 5600m spatial resolution
- (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod11b1_v006">MOD11B1</a> and
- <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd11b1_v006">MYD11B1</a>
- product pages)</li>
-</ul>
-
-<h3>MODIS VI - Vegetation Indices</h3>
-
-<ul>
- <li><b>VI sixteen days 250 m (Terra/Aqua)</b>: Global MOD13Q1/MYD13Q1 MODIS vegetation indices
- are designed to provide consistent spatial and temporal comparisons of vegetation.
- conditions. Blue, red, and near-infrared reflectances, centered at 469-nanometers,
- 645-nanometers, and 858-nanometers, respectively, are used to determine
- the MODIS daily vegetation indices.<br>
- The MODIS Normalized Difference Vegetation Index (NDVI) complements NOAA's
- Advanced Very High Resolution Radiometer (AVHRR) NDVI products and provides
- continuity for time series historical applications. MODIS also includes a
- new Enhanced Vegetation Index (EVI) that minimizes canopy background variations
- and maintains sensitivity over dense vegetation conditions. The EVI also
- uses the blue band to remove residual atmosphere contamination caused by
- smoke and sub-pixel thin cloud clouds. The MODIS NDVI and EVI products are
- computed from atmospherically corrected bi-directional surface reflectances
- that have been masked for water, clouds, heavy aerosols, and cloud shadows.
- Global MOD13Q1/MYD13Q1 data are provided every 16 days at 250-meter spatial resolution
- as a gridded level-3 product in the Sinusoidal projection. Lacking a 250m
- blue band, the EVI algorithm uses the 500m blue band to correct for residual
- atmospheric effects, with negligible spatial artifacts
- (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13q1_v006">MOD13Q1</a> and
- <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13q1_v006">MYD13Q1</a>
- product pages)</li>
- <li><b>VI sixteen days 500 m (Terra/Aqua)</b>: Global MOD13A1/MYD13A1 MODIS vegetation indices
- are designed to provide consistent spatial and temporal comparisons of vegetation
- conditions. Blue, red, and near-infrared reflectances, centered at
- 469-nanometers, 645-nanometers, and 858-nanometers, respectively, are used
- to determine the MODIS daily vegetation indices.<br>
- The MODIS Normalized Difference Vegetation Index (NDVI) complements NOAA's
- Advanced Very High Resolution Radiometer (AVHRR) NDVI products provide
- continuity for time series historical applications. MODIS also includes a
- new Enhanced Vegetation Index (EVI) that minimizes canopy background
- variations and maintains sensitivity over dense vegetation conditions.
- The EVI also uses the blue band to remove residual atmosphere contamination
- caused by smoke and sub-pixel thin cloud clouds. The MODIS NDVI and EVI
- products are computed from atmospherically corrected bi-directional surface
- reflectances that have been masked for water, clouds, heavy aerosols, and
- cloud shadows.<br>
- Global MOD13A1/MYD13A1 data are provided every 16 days at 500-meter spatial
- resolution as a gridded level-3 product in the Sinusoidal projection.
- Vegetation indices are used for global monitoring of vegetation conditions
- and are used in products displaying land cover and land cover changes.
- These data may be used as input for modeling global biogeochemical and
- hydrologic processes and global and regional climate. These data also may
- be used for characterizing land surface biophysical properties and processes,
- including primary production and land cover conversion
- (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13a1_v006">MOD13A1</a> and
- <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13a1_v006">MYD13A1</a>
- product pages)</li>
- <li><b>VI sixteen days 1 Km (Terra/Aqua)</b>:
- The MOD13A2 product provides a Vegetation Index (VI) value at a per pixel basis. There
- are 2 primary vegetation layers. The first is the Normalized Difference Vegetation Index (NDVI)
- which is referred to as the continuity index to the existing National Oceanic and Atmospheric
- Administration-Advanced Very High Resolution Radiometer (NOAA-AVHRR) derived NDVI. The second
- vegetation layer is the Enhanced Vegetation Index (EVI), which has improved sensitivity over
- high biomass regions. The data are provided at 1000 m resolution as a gridded level-3 product
- in the Sinusoidal projection
- (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13a2_v006">MOD13A2</a> and
- <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13a2_v006">MYD13A2</a>
- product pages)</li>
- <li><b>VI sixteen days Global 0.05Deg CMG (Terra/Aqua)</b>:
- The MOD13C1 product provides a Vegetation Index (VI) value at a per pixel basis. There
- are 2 primary vegetation layers. The first is the Normalized Difference Vegetation Index (NDVI)
- which is referred to as the continuity index to the existing National Oceanic and Atmospheric
- Administration-Advanced Very High Resolution Radiometer (NOAA-AVHRR) derived NDVI. The second
- vegetation layer is the Enhanced Vegetation Index (EVI), which has improved sensitivity over
- high biomass regions. The Climate Modeling Grid (CMG) consists 3600 rows and 7200 columns
- of 5600 m pixels and is provided as a global latitude/longitude grid
- (related <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/mod13c1_v006">MOD13C1</a> and
- <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table/myd13c1_v006">MYD13C1</a>
- product pages)</li>
-</ul>
-
-<h3>MODIS Snow</h3>
-<ul>
- <li><b>Snow eight days 500 m (Terra/Aqua)</b>: The MOD10A2 and MYD10A2 products
- are composites of eight days of snow maps in the sinusoidal grid.
- An eight-day compositing period was chosen because that is the exact ground
- track repeat period of the Terra and Aqua platforms. Snow cover over eight
- days is mapped as maximum snow extent in one SDS and as a chronology of
- observations in the other SDS. Eight-day periods begin on the first day of
- the year and extend into the next year. The product can be produced with
- two to eight days of input. There may not always be eight days of input,
- because of various reasons, so the user should check the attributes to
- determine on what days observations were obtained. See the validation webpage
- for details on the validation and validation definitions
- (related <a href="https://nsidc.org/data/MOD10A2">MOD10A2</a> and
- <a href="https://nsidc.org/data/MYD10A2">MYD10A2</a>
- product pages)</li>
- <li><b>Snow daily 500 m (Terra/Aqua)</b>: MOD10A1 and MYD10A1 are tiles of daily
- snow cover at 500 m spatial resolution. The daily observation selected from
- multiple observations in a MOD10A1 (or MYD10A1) cell is the observation
- acquired nearest nadir and having the greatest coverage of the grid cell.
- The daily MOD10A1 and MYD10A1 snow products are tiles of data gridded in the
- sinusoidal projection. Tiles are approximately 1200 x 1200 km in area. A
- single scientific data set (SDS) of snow cover and a single SDS of QA data
- along with local and global attributes comprise the data product file. The
- daily level 3 snow product is the result of selecting an observation from
- the multiple observations mapped to a cell of the MOD10_L2G (or MYD10_L2G)
- product. See the validation webpage for details on the validation and
- validation definitions
- (related <a href="https://nsidc.org/data/MOD10A1">MOD10A1</a> and
- <a href="https://nsidc.org/data/MYD10A1">MYD10A1</a>
- product pages)</li>
-</ul>
-
-<h2>NOTES</h2>
-
-The <em>r.modis</em> modules need the <a href="http://www.pymodis.org">pyModis</a>
-library. Please install it beforehand.
-
-<h2>SEE ALSO</h2>
-
-<em>
-<a href="r.modis.import.html">r.modis.import</a>,
-<a href="r.modis.download.html">r.modis.download</a>
-<!-- <a href="r.modis.process.html">r.modis.process</a>,-->
-</em>
-<p>
-<ul>
- <li><a href="https://lpdaac.usgs.gov/tools/modis_reprojection_tool">MODIS Reprojection Tool</a></li>
- <li> <a href="http://modis-land.gsfc.nasa.gov/">MODIS Land homepage</a></li>
- <li> <a href="http://modis-snow-ice.gsfc.nasa.gov/">MODIS Snow homepage</a></li>
- <li> <a href="https://lpdaac.usgs.gov/dataset_discovery/modis/modis_products_table">MODIS Land products table</a></li>
-</ul>
-
-<h2>AUTHOR</h2>
-
-Luca Delucchi, Initial version: Google Summer of Code 2011; subsequently updated
-
-<p><i>Last changed: $Date$</i>
Modified: grass-addons/grass7/raster/Makefile
===================================================================
--- grass-addons/grass7/raster/Makefile 2018-03-20 14:32:12 UTC (rev 72414)
+++ grass-addons/grass7/raster/Makefile 2018-03-20 15:00:13 UTC (rev 72415)
@@ -71,7 +71,6 @@
r.mcda.roughset \
r.meb \
r.mess \
- r.modis \
r.mwprecip \
r.neighborhoodmatrix \
r.niche.similarity \
Added: grass-addons/grass7/raster/r.modis/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/Makefile (rev 0)
+++ grass-addons/grass7/raster/r.modis/Makefile 2018-03-20 15:00:13 UTC (rev 72415)
@@ -0,0 +1,2 @@
+default:
+ $(error Extension has been renamed to i.modis)
More information about the grass-commit
mailing list