[GRASS-SVN] r46634 - in grass-addons/grass7/raster/r.modis: .
libmodis r.modis.download
svn_grass at osgeo.org
svn_grass at osgeo.org
Thu Jun 9 04:24:42 EDT 2011
Author: lucadelu
Date: 2011-06-09 01:24:42 -0700 (Thu, 09 Jun 2011)
New Revision: 46634
Added:
grass-addons/grass7/raster/r.modis/Makefile
grass-addons/grass7/raster/r.modis/libmodis/
grass-addons/grass7/raster/r.modis/libmodis/Makefile
grass-addons/grass7/raster/r.modis/libmodis/modis.py
grass-addons/grass7/raster/r.modis/libmodis/rmodislib.py
grass-addons/grass7/raster/r.modis/r.modis.download/
grass-addons/grass7/raster/r.modis/r.modis.download/Makefile
grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.html
grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.py
grass-addons/grass7/raster/r.modis/r.modis.html
Log:
first version of r.modis.download to tested more
Added: grass-addons/grass7/raster/r.modis/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/Makefile (rev 0)
+++ grass-addons/grass7/raster/r.modis/Makefile 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,11 @@
+MODULE_TOPDIR =../..
+
+PGM = r.modis
+
+SUBDIRS = \
+ r.modis.download \
+ libmodis
+
+include $(MODULE_TOPDIR)/include/Make/Dir.make
+
+default: parsubdirs htmldir
\ No newline at end of file
Added: grass-addons/grass7/raster/r.modis/libmodis/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/libmodis/Makefile (rev 0)
+++ grass-addons/grass7/raster/r.modis/libmodis/Makefile 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,16 @@
+MODULE_TOPDIR = ../../..
+
+PGM = libmodis
+
+include $(MODULE_TOPDIR)/include/Make/Script.make
+
+SRCFILES = rmodislib.py modis.py
+DSTFILES := $(patsubst %,$(ETC)/r.modis/%,$(SRCFILES))
+
+default: $(DSTFILES)
+
+$(ETC)/r.modis/%: % | $(ETC)/r.modis
+ $(INSTALL) $< $@
+
+$(ETC)/r.modis:
+ $(MKDIR) $@
Added: grass-addons/grass7/raster/r.modis/libmodis/modis.py
===================================================================
--- grass-addons/grass7/raster/r.modis/libmodis/modis.py (rev 0)
+++ grass-addons/grass7/raster/r.modis/libmodis/modis.py 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,640 @@
+#!/usr/bin/python
+# -*- coding: utf-8 -*-
+# class to download modis data
+#
+# (c) Copyright Luca Delucchi 2010
+# Authors: Luca Delucchi
+# Email: luca dot delucchi at iasma dot it
+#
+##################################################################
+#
+# Modis class is licensed under the terms of GNU GPL 2
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation; either version 2 of
+# the License,or (at your option) any later version.
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+# See the GNU General Public License for more details.
+#
+##################################################################
+# History
+##################################################################
+#
+# 0.3.0 Fix the choosing of days, change name modisClass to downModis
+# and add parseModis (2011-05-24)
+# 0.2.1 Little change in the logging option (2011-01-21)
+# 0.2.0 Add logging and change something in debug methods (2010-12-01)
+# 0.1.3 Correct a little problem with "Connection timed out"
+# 0.1.2 Add two debug methods (2010-13-08)
+# 0.1.1 Add moveFile method (2010-07-02)
+# 0.1.0 First Version of Modis Class (2010-06-19)
+#
+##################################################################
+
+# tilesUsed="h17v04,h17v05,h18v02,h18v03,h18v05,h19v04,h19v05"
+# writePath="/home/luca/test_modis"
+
+__version__ = '0.3.0'
+
+from datetime import *
+import string
+import os
+import sys
+import glob
+import logging
+import socket
+from ftplib import FTP
+import ftplib
+
+class downModis:
+ """A class to download modis data from nasa ftp repository"""
+ def __init__(self,
+ user,
+ password,
+ destinationFolder,
+ url = "e4ftl01u.ecs.nasa.gov",
+ tiles = None,
+ path = "MOLT/MOD11A1.005",
+ today = None,
+ enddate = None,
+ delta = 10,
+ jpg = False,
+ debug = False
+ ):
+ """Initialization function :
+ user=is your username
+ password=is your password
+ destinationFolder=where your file are storage
+ url=the url where download data
+ path=the directory where the data that you want download are
+ storaged in the ftp server
+ tiles=a list of tiles that you want downloads, None == all tiles
+ today=the day to start download, to pass a date different to
+ today use this format year-month-day
+ delta=timelag i.e. the number of days starting from today
+ (backward)
+
+ Create ftp istance, connect user to ftp server and go to the
+ directory where data are storage
+ """
+
+ # url modis
+ self.url = url
+ # user for download
+ self.user = user
+ # password for download
+ self.password = password
+ # directory where data are collected
+ self.path = path
+ # tiles to downloads
+ if tiles:
+ self.tiles = tiles.split(',')
+ else:
+ self.tiles = tiles
+ # set destination folder
+ if os.access(destinationFolder,os.W_OK):
+ self.writeFilePath = destinationFolder
+ else:
+ raise IOError("Folder to write downloaded files doesn't exist or is not" \
+ + "writeable")
+ # return the name of product
+ if len(self.path.split('/')) == 2:
+ self.product = self.path.split('/')[1]
+ elif len(self.path.split('/')) == 3:
+ self.product = self.path.split('/')[2]
+ # write a file with the name of file downloaded
+ self.filelist = open(os.path.join(self.writeFilePath, 'listfile' \
+ + self.product + '.txt'),'w')
+ # set jpg download
+ self.jpeg = jpg
+ # today
+ self.today = today
+ # force the last day
+ self.enday = enddate
+ # delta of days
+ self.delta = delta
+ # status of tile download
+ self.status = True
+ # for debug, you can download only xml files
+ self.debug = debug
+ # for logging
+ LOG_FILENAME = os.path.join(self.writeFilePath, 'modis' \
+ + self.product + '.log')
+ LOGGING_FORMAT='%(asctime)s - %(levelname)s - %(message)s'
+ logging.basicConfig(filename=LOG_FILENAME, level=logging.DEBUG, \
+ format=LOGGING_FORMAT)
+
+ def connectFTP(self):
+ """ set connection to ftp server, move to path where data are storaged
+ and create a list of directory for all days"""
+ try:
+ # connect to ftp server
+ self.ftp = FTP(self.url)
+ self.ftp.login(self.user,self.password)
+ # enter in directory
+ self.ftp.cwd(self.path)
+ self.dirData = []
+ # return data inside directory
+ self.ftp.dir(self.dirData.append)
+ # reverse order of data for have first the nearest to today
+ self.dirData.reverse()
+ # check if dirData contain only directory, delete all files
+ self.dirData = [elem.split()[-1] for elem in self.dirData if elem.startswith("d")]
+ if self.debug==True:
+ logging.debug("Open connection %s" % self.url)
+ except EOFError:
+ logging.error('Error in connection')
+ self.connectFTP()
+
+ def closeFTP(self):
+ """ Close ftp connection """
+ self.ftp.quit()
+ self.filelist.close()
+ if self.debug==True:
+ logging.debug("Close connection %s" % self.url)
+
+ def setDirectoryIn(self,day):
+ """ Enter in the directory of the day """
+ try:
+ self.ftp.cwd(day)
+ except (ftplib.error_reply,socket.error), e:
+ logging.error("Error %s entering in directory %s" % e, day)
+ self.setDirectoryIn(day)
+
+ def setDirectoryOver(self):
+ """ Come back to old path """
+ try:
+ self.ftp.cwd('..')
+ except (ftplib.error_reply,socket.error), e:
+ logging.error("Error %s when try to come back" % e)
+ self.setDirectoryOver()
+
+ def str2date(self,strin):
+ """Return a date object from a string"""
+ todaySplit = strin.split('-')
+ return date(int(todaySplit[0]), int(todaySplit[1]),int(todaySplit[2]))
+
+ def getToday(self):
+ """Return the first day for start to download"""
+ if self.today == None:
+ # set today variable to today
+ self.today = date.today()
+ else:
+ # set today variable to data pass from user
+ self.today = self.str2date(self.today)
+ # set enday variable to data
+ if self.enday != None:
+ self.enday = self.str2date(self.enday)
+
+ def getListDays(self):
+ """ Return a list of all days selected """
+ self.getToday()
+
+ today_s = self.today.strftime("%Y.%m.%d")
+ # dirData is reverse sorted
+ for i, d in enumerate(self.dirData):
+ if d <= today_s:
+ today_avail = d
+ today_index = i
+ break
+ else:
+ logging.error("No data available for requested days")
+ import sys
+ sys.exit()
+ days = self.dirData[today_index:][:self.delta]
+ # this is useful for 8/16 days data, delta could download more images
+ # that you want
+ if self.enday != None:
+ enday_s = self.enday.strftime("%Y.%m.%d")
+ delta = 0
+ # it make a for cicle from the last value and find the internal delta
+ #to remove file outside temporaly range
+ for i in range(-(len(days)),0):
+ if days[i] < enday_s:
+ break
+ else:
+ delta = delta + 1
+ # remove days outside new delta
+ days = days[:delta]
+ return days
+
+ def getFilesList(self):
+ """ Create a list of files to download, is possible choose if download
+ also jpeg files or only hdf"""
+ def cicle_file(jpeg=False,tile=True):
+ finalList = []
+ for i in self.listfiles:
+ File = i.split('.')
+ # distinguish jpeg files from hdf files by the number of index
+ # where find the tile index
+ if not tile and not (File.count('jpg') or File.count('BROWSE')):
+ finalList.append(i)
+ if tile and self.tiles.count(File[3]) == 1 and jpeg: #is a jpeg of tiles number
+ finalList.append(i)
+ if tile and self.tiles.count(File[2]) == 1: #is a hdf of tiles number
+ finalList.append(i)
+ return finalList
+
+ # return the file's list inside the directory of each day
+ try:
+ self.listfiles = self.ftp.nlst()
+ # download also jpeg
+ if self.jpeg:
+ # finallist is ugual to all file with jpeg file
+ if not self.tiles:
+ finalList = self.listfiles
+ # finallist is ugual to tiles file with jpeg file
+ else:
+ finalList = cicle_file(jpeg=True)
+ # not download jpeg
+ else:
+ if not self.tiles:
+ finalList = cicle_file(tile=False)
+ else:
+ finalList = cicle_file()
+ if self.debug==True:
+ logging.debug("The number of file to download is: %i" % len(finalList))
+ return finalList
+ except (ftplib.error_reply,socket.error), e:
+ logging.error("Error %s when try to receive list of files" % e)
+ self.getFilesList()
+
+ def checkDataExist(self,listNewFile, move = 0):
+ """ Check if a data already exist in the directory of download
+ Move serve to know if function is called from download or move function"""
+ fileInPath = []
+ # add all files in the directory where we will save new modis data
+ for f in os.listdir(self.writeFilePath):
+ if os.path.isfile(os.path.join(self.writeFilePath, f)):
+ fileInPath.append(f)
+ # different return if this method is used from downloadsAllDay() or
+ # moveFile()
+ if move == 0:
+ listOfDifferent = list(set(listNewFile) - set(fileInPath))
+ elif move == 1:
+ listOfDifferent = list(set(fileInPath) - set(listNewFile))
+ return listOfDifferent
+
+ def getNewerVersion(self,oldFile,newFile):
+ """ Return newer version of a file"""
+ oldFileSplit = oldFile.split('.')
+ newFileSplit = newFile.split('.')
+ if oldFileSplit[4] > newFileSplit[4]:
+ return oldFile
+ else:
+ return newFile
+
+ def downloadFile(self,filDown,filSave):
+ """Download the single file"""
+ #try to download file
+ try:
+ self.ftp.retrbinary("RETR " + filDown, filSave.write)
+ self.filelist.write("%s\n" % filDown)
+ if self.debug==True:
+ logging.debug("File %s downloaded" % filDown)
+ #if it have an error it try to download again the file
+ except (ftplib.error_reply,socket.error), e:
+ logging.error("Cannot download %s, retry.." % filDown)
+ self.connectFTP()
+ self.downloadFile(filDown,filSave)
+
+ def dayDownload(self,listFilesDown):
+ """ Downloads tiles are in files_hdf_consider """
+ # for each file in files' list
+ for i in listFilesDown:
+ fileSplit = i.split('.')
+ filePrefix = fileSplit[0] + '.' + fileSplit[1] + '.' + fileSplit[2] \
+ + '.' + fileSplit[3]
+ #for debug, download only xml
+ if (self.debug and fileSplit[-1] == 'xml') or not self.debug:
+ # check data exist in the return directory, if it doesn't exist
+ oldFile = glob.glob1(self.writeFilePath, filePrefix + "*" \
+ + fileSplit[-1])
+ numFiles = len(oldFile)
+ if numFiles == 0:
+ file_hdf = open(os.path.join(self.writeFilePath,i), "wb")
+ elif numFiles == 1:
+ # check the version of file
+ fileDown = self.getNewerVersion(oldFile[0],i)
+ if fileDown != oldFile[0]:
+ os.remove(os.path.join(self.writeFilePath,oldFile[0]))
+ file_hdf = open(os.path.join(self.writeFilePath,fileDown), "wb")
+ elif numFiles > 1:
+ logging.error("There are to much files for %s" % i)
+ #raise EOFError("There are to much file with the same prefix")
+ if numFiles == 0 or (numFiles == 1 and fileDown != oldFile[0]):
+ self.downloadFile(i,file_hdf)
+
+ def downloadsAllDay(self):
+ """ Downloads all the tiles considered """
+ #return the days to download
+ days = self.getListDays()
+ if self.debug==True:
+ logging.debug("The number of days to download is: %i" % len(days))
+ #for each day
+ for day in days:
+ #enter in the directory of day
+ self.setDirectoryIn(day)
+ #obtain list of all files
+ listAllFiles = self.getFilesList()
+ #obtain list of files to download
+ listFilesDown = self.checkDataExist(listAllFiles)
+ #download files for a day
+ self.dayDownload(listFilesDown)
+ self.setDirectoryOver()
+ self.closeFTP()
+ if self.debug==True:
+ logging.debug("Download terminated")
+ return 0
+
+ def debugLog(self):
+ # create logger
+ logger = logging.getLogger("PythonLibModis debug")
+ logger.setLevel(logging.DEBUG)
+ # create console handler and set level to debug
+ ch = logging.StreamHandler()
+ ch.setLevel(logging.DEBUG)
+ # create formatter
+ formatter = logging.Formatter("%(asctime)s - %(name)s - " \
+ + "%(levelname)s - %(message)s")
+ # add formatter to ch
+ ch.setFormatter(formatter)
+ # add ch to logger
+ logger.addHandler(ch)
+ return logger
+
+ def debugDays(self):
+ """This function is useful to debug the number of days"""
+ logger = debugLog()
+ days = self.getListDays()
+ # if lenght of list of days and the delta of day they are different
+ if len(days) != self.delta:
+ # for each day
+ for i in range(1,self.delta+1):
+ # calculate the current day
+ delta = timedelta(days = i)
+ day = self.today - delta
+ day = day.strftime("%Y.%m.%d")
+ # check if day is in the days list
+ if day not in days:
+ logger.critical("This day %s is not present on list" % day)
+ # the lenght of list of days and delta are ugual
+ else:
+ logger.info("All right!!")
+
+ def debugMaps(self):
+ """This function is useful to debug the number of maps to download for
+ each day"""
+ logger = debugLog()
+ days = self.getListDays()
+ for day in days:
+ self.setDirectoryIn(day)
+ listAllFiles = self.getFilesList()
+ string = day + ": " + str(len(listAllFiles)) + "\n"
+ logger.debug(string)
+ self.setDirectoryOver()
+
+class parseModis:
+ def __init__(self, filename):
+
+ from xml.etree import ElementTree
+
+ self.hdfname = filename
+ self.xmlname = self.hdfname + '.xml'
+ self.tifname = self.hdfname.replace('.hdf','.tif')
+ with open(self.xmlname) as f:
+ self.tree = ElementTree.parse(f)
+ # return the name of product
+ if len(self.path.split('/')) == 2:
+ self.product = self.path.split('/')[1]
+ elif len(self.path.split('/')) == 3:
+ self.product = self.path.split('/')[2]
+
+ def __str__(self):
+ """Print the file without xml tags"""
+ retString = ""
+ try:
+ for node in self.tree.iter():
+ if node.text.strip() != '':
+ retString = "%s = %s\n" % (node.tag,node.text)
+ except:
+ for node in self.tree.getiterator():
+ if node.text.strip() != '':
+ retString = "%s = %s\n" % (node.tag,node.text)
+ return retString
+
+ def getRoot(self):
+ """Set the root element"""
+ self.rootree = self.tree.getroot()
+
+ def retDTD(self):
+ """Return the DTDVersion element"""
+ self.getRoot()
+ return self.rootree.find('DTDVersion').text
+
+ def retDataCenter(self):
+ """Return the DataCenterId element"""
+ self.getRoot()
+ return self.rootree.find('DataCenterId').text
+
+ def getGranule(self):
+ """Set the GranuleURMetaData element"""
+ self.getRoot()
+ self.granule = self.rootree.find('GranuleURMetaData')
+
+ def retGranuleUR(self):
+ """Return the GranuleUR element"""
+ self.getGranule()
+ return self.granule.find('GranuleUR')
+
+ def retDbID(self):
+ """Return the DbID element"""
+ self.getGranule()
+ return self.granule.find('DbID')
+
+ def retInsertTime(self):
+ """Return the DbID element"""
+ self.getGranule()
+ return self.granule.find('InsertTime')
+
+ def retLastUpdate(self):
+ """Return the DbID element"""
+ self.getGranule()
+ return self.granule.find('LastUpdate')
+
+ def retCollectionMetaData(self):
+ """Return the CollectionMetaData element"""
+ self.getGranule()
+ collect = {}
+ for i in self.granule.find('CollectionMetaData').getiterator():
+ if i.text.strip() != '':
+ collect[i.tag] = i.text
+ return collect
+
+ def retDataFiles(self):
+ """Return the DataFiles element"""
+ self.getGranule()
+ collect = {}
+ datafiles = self.granule.find('DataFiles')
+ for i in datafiles.find('DataFileContainer').getiterator():
+ if i.text.strip() != '':
+ collect[i.tag] = i.text
+ return collect
+
+ def retDataGranule(self):
+ """Return the ECSDataGranule elements"""
+ self.getGranule()
+ datagran = {}
+ for i in self.granule.find('ECSDataGranule').getiterator():
+ if i.text.strip() != '':
+ datagran[i.tag] = i.text
+ return datagran
+
+ def retPGEVersion(self):
+ """Return the PGEVersion element"""
+ self.getGranule()
+ return self.granule.find('PGEVersionClass').find('PGEVersion').text
+
+ def retRangeTime(self):
+ """Return the RangeDateTime elements inside a dictionary with the element
+ name like dictionary key
+ """
+ self.getGranule()
+ rangeTime = {}
+ for i in self.granule.find('RangeDateTime').getiterator():
+ if i.text.strip() != '':
+ rangeTime[i.tag] = i.text
+ return rangeTime
+
+ def retBoundary(self):
+ """Return the maximum extend of the MODIS file inside a dictionary"""
+ self.getGranule()
+ self.boundary = []
+ lat = []
+ lon = []
+ spatialContainer = self.granule.find('SpatialDomainContainer')
+ horizontal = spatialContainer.find('HorizontalSpatialDomainContainer')
+ boundary = horizontal.find('GPolygon').find('Boundary')
+ for i in boundary.findall('Point'):
+ la = float(i.find('PointLongitude').text)
+ lo = float(i.find('PointLatitude').text)
+ lon.append(la)
+ lat.append(lo)
+ self.boundary.append({'lat': la, 'lon':lo})
+ extent = {'min_lat':min(lat),'max_lat':max(lat),'min_lon':min(lon),
+ 'max_lon':max(lon)}
+ return extent
+
+ def retMeasure(self):
+ """Return statistics inside a dictionary"""
+ value = {}
+ self.getGranule()
+ mes = self.granule.find('MeasuredParameter')
+ value['ParameterName'] = mes.find('ParameterName').text
+ meStat = mes.find('MeasuredParameterContainer').find('QAStats')
+ qastat = {}
+ for i in meStat.getiterator():
+ qastat[i.tag] = i.text
+ value['QAStats'] = qastat
+ meFlag = mes.find('MeasuredParameterContainer').find('QAFlags')
+ flagstat = {}
+ for i in meStat.getiterator():
+ flagstat[i.tag] = i.text
+ value['QAFlags'] = flagstat
+ return value
+
+ def retPlatform(self):
+ """Return the platform values inside a dictionary."""
+ value = {}
+ self.getGranule()
+ value['PlatformShortName'] = self.granule.find('PlatformShortName').text
+ instr = self.granule.find('Instrument')
+ value['InstrumentShortName'] = instr.find('InstrumentShortName').text
+ sensor = instr.find('Sensor')
+ value['SensorShortName'] = sensor.find('SensorShortName').text
+ return value
+
+ def retPSA(self):
+ """Return the PSA values inside a dictionary, the PSAName is he key and
+ and PSAValue is the value
+ """
+ value = {}
+ self.getGranule()
+ psas = self.granule.find('PSAs')
+ for i in psas.findall('PSA'):
+ value[i.find('PSAName').text] = i.find('PSAValue').text
+ return value
+
+ def retInputGranule(self):
+ value = []
+ self.getGranule()
+ for i in self.granule.find('InputGranule').getiterator():
+ value.append(i.text)
+ return value
+
+ def retBrowseProduct(self):
+ """Return the PGEVersion element"""
+ self.getGranule()
+ return self.granule.find('BrowseProduct').find('BrowseGranuleId').text
+
+ def confResample(self, filePath, output = None,
+ resampl = 'NEAREST_NEIGHBOR', projtype = 'GEO',
+ projpar = '( 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )',
+ datum = 'WGS84'
+ ):
+ if not output:
+ fileout = self.tifname
+ else:
+ fileout = output
+ """Write a configuration file for resample mrt software (TO TEST)"""
+ filename = os.join.path(filePath,'%s_mrt_resample.conf' % self.product)
+ conFile = open(filename, 'w')
+ conFile.write("INPUT_FILENAME = %s" % self.hdfname)
+ conFile.write("SPECTRAL_SUBSET = ( 1 1 )")
+ conFile.write("SPATIAL_SUBSET_TYPE = INPUT_LAT_LONG")
+ bound = self.retBoundary()
+ # Order: UL: N W - LR: S E
+ conFile.write("SPATIAL_SUBSET_UL_CORNER = ( %f %f )" % (bound['max_lat'],bound['min_lon']))
+ conFile.write("SPATIAL_SUBSET_LR_CORNER = ( %f %f )" % (bound['min_lat'],bound['max_lon']))
+ conFile.write("OUTPUT_FILENAME = %s" % output)
+ conFile.write("RESAMPLING_TYPE = %s" % resampl)
+ conFile.write("OUTPUT_PROJECTION_TYPE = %s" % projtype)
+ conFile.write("OUTPUT_PROJECTION_PARAMETERS = %s" % projpar)
+ conFile.write("DATUM = %s" % datum)
+ conFile.close()
+ return filename
+
+class convertModis:
+ """A class to convert modis data from hdf to tif using resample (mrt tools)"""
+ def __init__(self,
+ hdfname,
+ confile,
+ mrtpath):
+ if os.path.exists(hdfname):
+ self.name = hdfname
+ else:
+ raise IOError('%s not exist' % hdfname)
+ if os.path.exists(confile):
+ self.conf = confile
+ else:
+ raise IOError('%s not exist' % confile)
+ if os.path.exists(mrtpath):
+ self.mrtpath = mrtpath
+ else:
+ raise IOError('The path %s not exist' % mrtpath)
+
+ def executable(self):
+ """return the executable
+ on windows an exe file
+ """
+ if sys.platform.count('linux') != -1:
+ return 'resample'
+ elif sys.platform.count('win32') != -1:
+ return 'resample.exe'
+
+#class createMosaic:
+ #def __init__(self
+ #listfile,
+ #mrtpath)
Added: grass-addons/grass7/raster/r.modis/libmodis/rmodislib.py
===================================================================
--- grass-addons/grass7/raster/r.modis/libmodis/rmodislib.py (rev 0)
+++ grass-addons/grass7/raster/r.modis/libmodis/rmodislib.py 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,53 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+############################################################################
+#
+# MODULE: r.modis
+# AUTHOR(S): Luca Delucchi
+# PURPOSE: Here there are some important class to run r.modis modules
+#
+#
+# COPYRIGHT: (C) 2011 by Luca Delucchi
+#
+# This program is free software under the GNU General Public
+# License (>=v2). Read the file COPYING that comes with GRASS
+# for details.
+#
+#############################################################################
+
+class product:
+ """Definition of modis product with url and path in the ftp server
+ """
+ def __init__(self,value):
+ urlbase = 'e4ftl01u.ecs.nasa.gov'
+ usrsnow = 'n4ftl01u.ecs.nasa.gov'
+ self.prod = value
+ self.products = {'lst_aqua_daily':{'url':urlbase,
+ 'folder':'MOLA/MYD11A1.005'},'lst_terra_daily':{'url':urlbase,
+ 'folder':'MOLT/MOD11A1.005'},'snow_terra_eight':{'url':usrsnow,
+ 'folder':'SAN/MOST/MOD10A2.005'}, 'ndvi_terra_sixte':{'url':urlbase,
+ 'folder':'MOLT/MOD13Q1.005'} }
+
+ def returned(self):
+ return self.products[self.prod]
+
+ def __str__(self):
+ prod = self.returned()
+ string = "url: " + prod['url'] + ", folder: " + prod['folder']
+ return string
+
+class resampling:
+ """Return the resampling value from the code used in the modules
+ """
+ def __init__(self,value):
+ self.code = value
+ self.resampling = {'NN': 'NEAREST_NEIGHBOR','BI' : 'BICUBIC',
+ 'CC' : '','NONE' : 'NONE'}
+
+ def returned(self):
+ return self.resampling[self.code]
+
+class projection:
+ """Definition of projection for convert from sinusoidal projection to
+ another one. Not all projection systems are supported"""
Added: grass-addons/grass7/raster/r.modis/r.modis.download/Makefile
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.download/Makefile (rev 0)
+++ grass-addons/grass7/raster/r.modis/r.modis.download/Makefile 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,13 @@
+MODULE_TOPDIR = ../../..
+
+PGM = r.modis.download
+
+include $(MODULE_TOPDIR)/include/Make/Script.make
+
+default: script
+
+# $(ETC)/r.modis/r.modis.download/%: % | $(ETC)/r.modis/r.modis.download
+# $(INSTALL) $< $@
+#
+# $(ETC)/r.modis/r.modis.download:
+# $(MKDIR) $@
\ No newline at end of file
Added: grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.html
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.html (rev 0)
+++ grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.html 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,69 @@
+<h2>DESCRIPTION</h2>
+
+<em>r.modis.download</em> download MODIS products. It can download several tiles and days of each MODIS product.
+
+<h2>NOTE</h2>
+
+The first time that you use <em>r.modis</em> suite you have to do some easy operations to work corectly with <em>r.modis</em> modules.
+<ul>
+ <li>Create a r.modis folder inside $HOME/.grass7</li>
+ <li>Create SETTING.py file inside $HOME/.grass7/r.modis and compile like below:
+ <div class="code"><pre>
+ #!/usr/bin/python
+
+ USERNAME='your_NASA_FTP_username'
+ PASSWORD='your_NASA_FTP_password'
+ MRTPATH='your_path_to_MRT_software'
+ </pre></div>
+ </li>
+</ul>
+
+The username and password should not be required but it's better set them; the username is anonymous and password is your mail. Instead NASA FTP server it is open
+<b>Warning</b>: NASA policy don't permit to open more then ten connections with the FTP servers. Please don't run more 10 the <em>r.modis.download</em> process at the same time.<br>
+By default the data downladed are store in <em>$HOME/.grass7/r.modis/download/</em>; you can change the directory with <em>folder</em> options.<br>
+The time of download change with the number of tiles, days and the type of product, for example NDVI product it is from 5 MB to 270 MB and LST is from 2 MB to 21 MB.
+
+<h4>Products</h4>
+The products already supported:
+<ul>
+ <li>Land Surface Temperature daily (Terra): product provides per-pixel temperature and emissivity values in a sequence of swath-based to grid-based global products. The MODIS/Terra LST/E Daily L3 Global 1 km Grid product (MOD11A1), is tile-based and gridded in the Sinusoidal projection, and produced daily at 1 km spatial resolution. </li>
+ <li>Land Surface Temperature daily (Aqua): product provides per-pixel temperature and emissivity values in a sequence of swath-based to grid-based global products. The MODIS/Aqua LST/E Daily L3 Global 1 Km Grid product (MYD11A1), is tile-based and gridded in the Sinusoidal projection, and produced daily at 1 Km spatial resolution.</li>
+ <li>NDVI sixteen days (Terra): MODIS Normalized Difference Vegetation Index (NDVI) complements NOAA's Advanced Very High Resolution Radiometer (AVHRR) NDVI products and provides continuity for time series historical applications. data are provided every 16 days at 250-meter spatial resolution as a gridded level-3 product (MOD13Q1) in the Sinusoidal projection.</li>
+ <li>Snow eight days: </li>
+</ul>
+
+<h2>EXAMPLES</h2>
+
+To download LST Terra product using the default options (using $HOME/.grass7/r.modis/ as folder, SETTING.py file for username and password, starting from today until ten days ago) you can launch
+<div class="code"><pre>
+ r.modis.download
+</pre></div>
+
+To download LST Terra product using the default options (using $HOME/.grass7/r.modis/ as folder, SETTING.py file for username and password) and change the starting and ending dates
+<div class="code"><pre>
+ r.modis.download startday=2011-05-31 endday=2011-05-01
+</pre></div>
+
+To downlad another product (for example <em>Snow eight days</em>), with other default options
+<div class="code"><pre>
+ r.modis.download product=snow_terra_eight
+</pre></div>
+
+To use username and password specified (note that the command
+lines history will store the password in this way) and use the other default options
+<div class="code"><pre>
+ r.modis.download username=user password=*****
+</pre></div>
+
+<h2>SEE ALSO</h2>
+<ul>
+ <li> <a href="r.modis.html">r.modis</a></li>
+<!-- <li> <a href="r.modis.import.html">r.modis.import</a></li> -->
+<!-- <li> <a href="r.modis.process.html">r.modis.process</a>: Calculates range of patch area size on a raster map</li>-->
+</ul>
+
+<h2>AUTHOR</h2>
+
+Luca Delucchi, Google Summer of Code 2011
+
+<p><i>Last changed: $Date: 2011-06-03 14:44:59 +0200 (Mon, 09 May 2011) $</i>
\ No newline at end of file
Added: grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.py
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.py (rev 0)
+++ grass-addons/grass7/raster/r.modis/r.modis.download/r.modis.download.py 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,217 @@
+#!/usr/bin/env python
+# -*- coding: utf-8 -*-
+
+############################################################################
+#
+# MODULE: r.in.modis.download
+# AUTHOR(S): Luca Delucchi
+# PURPOSE: r.in.modis.download is an internafe to pyModis to download
+# several tiles of MODIS produts from NASA ftp
+#
+# COPYRIGHT: (C) 2011 by Luca Delucchi
+#
+# This program is free software under the GNU General Public
+# License (>=v2). Read the file COPYING that comes with GRASS
+# for details.
+#
+#############################################################################
+
+#%module
+#% description: Download several tiles of MODIS products using pyModis
+#% keywords: raster
+#%end
+#%option
+#% key: product
+#% type: string
+#% description: Name of MODIS product
+#% required: yes
+#% options: lst_aqua_daily, lst_terra_daily, snow_terra_eight, ndvi_terra_sixte
+#% answer: lst_terra_daily
+#%end
+#%option
+#% key: username
+#% type: string
+#% key_desc: username
+#% description: Name of user to NASA ftp access, it's better use SETTING.py
+#% required: no
+#%end
+#%option
+#% key: password
+#% type: string
+#% key_desc: password
+#% description: Password of user to NASA ftp access, it's better use SETTING.py
+#% required: no
+#%end
+#%option
+#% key: folder
+#% type: string
+#% description: Folder where saves the data, full path
+#% answer: $HOME/.grass7/r.modis/download
+#% required: no
+#%end
+#%option
+#% key: tiles
+#% type: string
+#% description: The names of tile/s to download
+#% required: no
+#%end
+#%option
+#% key: startday
+#% type: string
+#% description: The day from start download. If not set the download starts from today
+#% required: no
+#%end
+#%option
+#% key: endday
+#% type: string
+#% description: The day to stop download. If not set the download stops 10 day before the start day
+#% required: no
+#%end
+#%flag
+#% key: d
+#% description: For debug mode, it will write more info in the log file
+#%end
+
+# import library
+import os, sys
+from datetime import *
+import grass.script as grass
+
+# add the folder containing libraries to python path
+libmodis = os.path.join(os.getenv('GISBASE'), 'etc', 'r.modis')
+sys.path.append(libmodis)
+# try to import pymodis (modis) and some class for r.modis.download
+try:
+ from rmodislib import product
+ from modis import downModis
+except ImportError:
+ pass
+
+
+def check(home,suffix):
+ """ Function to check the path necessary to work this module
+ """
+
+ # check if ~/.grass7/r.modis/download exist or create it
+ if not os.path.exists(home):
+ os.mkdir(home)
+ os.mkdir(os.path.join(home,suffix))
+ elif not os.path.exists(os.path.join(home,suffix)):
+ os.mkdir(os.path.join(home,suffix))
+ return 1
+
+def checkdate(options):
+ """ Function to check the data and return the correct value to download the
+ the tiles
+ """
+ def check2day(second,first=None):
+ """Function to check two date"""
+ if not first:
+ valueDay = None
+ firstDay = date.today()
+ else:
+ valueDay = first
+ firstSplit = first.split('-')
+ firstDay = date(int(firstSplit[0]),int(firstSplit[1]),int(firstSplit[2]))
+ lastSplit = second.split('-')
+ lastDay = date(int(lastSplit[0]),int(lastSplit[1]),int(lastSplit[2]))
+ delta = firstDay-lastDay
+ valueDelta = int(delta.days)
+ return valueDay, second, valueDelta
+
+ # no set start and end day
+ if options['startday'] == '' and options['endday'] == '':
+ return None, None, 10
+ # set only end day
+ elif options['startday'] == '' and options['endday'] != '':
+ today = date.today().strftime("%Y-%m-%d")
+ import pdb; pdb.set_trace()
+ if today <= options['endday']:
+ grass.fatal(_('The last day cannot before >= of the first day'))
+ return 0
+ valueDay, valueEnd, valueDelta = check2day(options['endday'])
+ # set only start day
+ elif options['startday'] != '' and options['endday'] == '':
+ valueDay = options['startday']
+ valueEnd = None
+ valueDelta = 10
+ # set start and end day
+ elif options['startday'] != '' and options['endday'] != '':
+ valueDay, valueEnd, valueDelta = check2day(options['endday'],options['startday'])
+ return valueDay, valueEnd, valueDelta
+# main function
+def main():
+
+ # check if you are in GRASS
+ gisbase = os.getenv('GISBASE')
+ if not gisbase:
+ grass.fatal(_('$GISBASE not defined'))
+ return 0
+
+ # set the home path
+ home = os.path.expanduser('~')
+
+ # check the version
+ version = grass.core.version()
+ # this is would be set automatically
+ if version['version'].find('7.') != -1:
+ session_path = '.grass7'
+ grass_fold = os.path.join(home,session_path)
+ if not os.path.exists(grass_fold):
+ os.mkdir(grass_fold)
+ else:
+ grass.fatal(_('You are not in GRASS GIS version 7'))
+ return 0
+ # path to ~/grass7/r.modis
+ path = os.path.join(grass_fold,'r.modis')
+ # first date and delta
+ firstday, finalday, delta = checkdate(options)
+ #username and password are optional because the command lines history
+ #will store the password in this these options; so you are the possibility
+ #to use a file
+ if options['username'] == '' and options['password'] == '':
+ sett_file = os.path.join(path,'SETTING.py')
+ grass.message("For setting will use %s" % sett_file)
+ try:
+ sys.path.append(path)
+ import SETTING
+ except ImportError:
+ grass.fatal("%s not exist or is not well formatted. Create "\
+ "the file with inside:\n\n#!/usr/bin/env python\n\nusername="\
+ "your_user\npassword=xxxxx" % sett_file)
+ return 0
+ user = SETTING.username
+ passwd = SETTING.username
+ else:
+ grass.message("For setting will use username and password options")
+ user = options['username']
+ passwd = options['password']
+
+ prod = product(options['product']).returned()
+ # set folder
+ if options['folder'].find('HOME/.grass7/r.modis/download') == 1:
+ fold = os.path.join(path,'download')
+ # some check
+ check(path,'download')
+ else:
+ fold = options['folder']
+ # set tiles
+ if options['tiles'] == '':
+ tiles = None
+ else:
+ tiles = options['tiles']
+
+ # start modis class
+ modisOgg = downModis(url = prod['url'], user = user,password = passwd,
+ destinationFolder = fold, tiles = tiles, path = prod['folder'],
+ today = firstday, enddate = finalday, delta = delta)
+ # connect to ftp
+ modisOgg.connectFTP()
+ # download tha tiles
+ modisOgg.downloadsAllDay()
+ grass.message("All data are downloaded, now you can use r.in.modis.import "\
+ "or r.in.modis.process with option 'conf=" + modisOgg.filelist.name)
+
+if __name__ == "__main__":
+ options, flags = grass.parser()
+ sys.exit(main())
Added: grass-addons/grass7/raster/r.modis/r.modis.html
===================================================================
--- grass-addons/grass7/raster/r.modis/r.modis.html (rev 0)
+++ grass-addons/grass7/raster/r.modis/r.modis.html 2011-06-09 08:24:42 UTC (rev 46634)
@@ -0,0 +1,34 @@
+<h2>DESCRIPTION</h2>
+
+The <em>r.modis</em> suite is a toolset to import MODIS data in GRASS GIS. It use the <a href="http://gis.cri.fmach.it/development/pyModis">pyModis</a> library and the <a href="https://lpdaac.usgs.gov/lpdaac/tools/modis_reprojection_tool">MODIS Reprojection Tool</a> softwares to convert, mosaik and process MODIS data. Please check to have <a href="https://lpdaac.usgs.gov/lpdaac/tools/modis_reprojection_tool">MODIS Reprojection Tool</a> on your Operation System, <a href="http://gis.cri.fmach.it/development/pyModis">pyModis</a> it is distributed with <em>r.modis</em> program.
+The suite offers three modules for interface with MODIS data. Each modules it is useful for a specific operation, read below for more info.
+It can download several products, tiles and days of MODIS data from NASA FTP servers. After, it can import the 3 Level products as single image or daily mosaik or process the raw data.
+
+<h2>NOTE</h2>
+The first time that you use <em>r.modis</em> suite you have to do some easy operations to work corectly with <em>r.modis</em> modules.
+<ul>
+ <li>Create a r.modis folder inside $HOME/.grass7</li>
+ <li>Create SETTING.py file inside $HOME/.grass7/r.modis and compile like below:
+ <div class="code"><pre>
+ #!/usr/bin/python
+
+ USERNAME='your_NASA_FTP_username'
+ PASSWORD='your_NASA_FTP_password'
+ MRTPATH='your_path_to_MRT_software'
+ </pre></div>
+ </li>
+</ul>
+$HOME/.grass7/r.modis is also the default path where <em>r.modis</em> suite creates some subfolder to store the different MODIS files. Each module permits to change the previous folder with another directory using an module's option.
+
+<h2>SEE ALSO</h2>
+<ul>
+ <li> <a href="r.modis.download.html">r.modis.download</a>: Download MODIS products</li>
+<!-- <li> <a href="r.modis.import.html">r.modis.import</a>: Import into GRASS GIS 3 Level MODIS products as single image or daily mosaik</li> -->
+<!-- <li> <a href="r.modis.process.html">r.modis.process</a>: Calculates range of patch area size on a raster map</li>-->
+</ul>
+
+<h2>AUTHOR</h2>
+
+Luca Delucchi, Google Summer of Code 2011
+
+<p><i>Last changed: $Date: 2011-06-03 14:44:59 +0200 (Mon, 09 May 2011) $</i>
More information about the grass-commit
mailing list