[GRASS-user] Archiving GPM 3hr rainrate in GRASS

[Laurent C.]

Hello all,

I wrote a module that automatically download, import and register GPM
IMERG maps in GRASS.
You will need a NASA Earthdata Login to download the data. The code
can be found here:
https://bitbucket.org/lrntct/t.rast.in.gpm

For now, it is limited to IMERGHH data.
Hope it could help people working with those data.

Regards,
Laurent


2015-03-19 10:50 GMT-06:00 Laurent C. <lrntct at gmail.com>:
> Hi all,
>
> I'm coming back to that subject because the GPM datas are finally available
> since December 2014. The original data are stored in HDF5, which is
> supported by GDAL.
> However, I had a problem with the geo-referencing of the data. It seems that
> the lat/long coordinates are flipped.
> The problem seems to come from GDAL, as the HDF5 driver still don't support
> GPM data:
> http://www.gdal.org/frmt_hdf5.html
>
> I've actually downloaded a regional extracts the data in NetCDF from a NASA
> website, although I can't remember where exactly, as there are several
> services. Those data suffer from the same problem of inverted coordinates.
>
> So I used the ncpdq from NCO package to fix the NetCDF directly, and a tiny
> Python script to batch process the files :
>
> ###
> #!/usr/bin/env python
> from os import listdir
> import subprocess
> for input_file in listdir(main_path):
>     output_file = input_file + '_fixed'
>     subprocess.call(['ncpdq','-a', 'lat,lon', input_file, output_file])
> ###
>
> After batch import the data in GRASS, maintaining the original name
> including timestep, which looks like this:
> 3B-HHR.MS.MRG.3IMERG.20140619-S000000-E002959.0000.V03D
>
> A Python script to register the maps in temporal framework:
>
> #####
> #!/usr/bin/env python
> import grass.script as grass
>
> # retrieve the list of maps
> maplist = grass.read_command('g.list', type = 'raster',
>     pattern = '*IMERG*')
>
> # turn result in a list
> maplist = maplist.split()
>
> file_name = 'gpm_timestamp.txt'
>
> # creating the file containing the timepstamp
> list_file = open(file_name,'w')
>
> # iterate through the maps
> for input_map in maplist:
>     # split line to keep only the timestamp
>     raw_ts = input_map.split('.')[4]
>     # isolate the date
>     raw_mapdate = raw_ts.split('-')[0]
>     # put the date in form
>     mapdate = raw_mapdate[:4] + '-' + raw_mapdate[4:6] + \
>         '-' + raw_mapdate[6:]
>     # isolate the start time
>     raw_start_time = raw_ts.split('-')[1]
>     # put the date in form
>     start_time = raw_start_time[1:3] + ':' + \
>         raw_start_time[3:5] + ':' + raw_start_time[5:]
>     # isolate the end time
>     raw_end_time = raw_ts.split('-')[2]
>     # put the end time in form
>     end_time = raw_end_time[1:3] + ':' + \
>         raw_end_time[3:5] + ':' + raw_end_time[5:]
>     # put timestamp in form
>     timestamp = mapdate + ' ' + start_time + '|' + mapdate + \
>         ' ' + end_time
>
>     # format the whole line
>     line = input_map + '|' + timestamp + '\n'
>
>     # write line to the file
>     list_file.write(line)
>
> # close the file
> list_file.close()
>
> # register the maps in grass space_time dataset
> grass.run_command('t.register', input = 'GPM_ZMCM', file = file_name)
>
> #####
>
>
> Hope it will help other peoples having trouble using those data.
>
> Regards,
> Laurent
>
>
> 2014-11-06 5:15 GMT-06:00 maning sambale <emmanuel.sambale at gmail.com>:
>>
>> Thanks Markus.  Already registered and can access ftp.  Unfortunately,
>> processed (L3) rainfall data will be released by Dec 2014.
>> Will just wait then. :)
>>
>> On Thu, Nov 6, 2014 at 4:27 PM, Markus Neteler <neteler at osgeo.org> wrote:
>> > On Mon, Nov 3, 2014 at 10:18 AM, maning sambale
>> > <emmanuel.sambale at gmail.com> wrote:
>> >> Has anyone here able to archive and load GPM dataset into GRASS?  I
>> >> was able to get TRMM netcdf in my area of interest before, but I can't
>> >> find the tools to automate GPM downloads.
>> >>
>> >> Thanks!
>> >>
>> >> [0] http://www.nasa.gov/mission_pages/GPM/main/
>> >
>> > They state
>> >
>> > "All data are freely available through the NASA's Precipitation
>> > Processing System at http://pps.gsfc.nasa.gov"
>> >
>> > --> "Register and search for GPM and TRMM data, order custom subsets
>> > and set up subscriptions using PPS Data Products Ordering Interface
>> > (STORM)."
>> >
>> > At time the server seems to be down?
>> >
>> > Markus
>>
>>
>>
>> --
>> cheers,
>> maning
>> ------------------------------------------------------
>> "Freedom is still the most radical idea of all" -N.Branden
>> wiki: http://esambale.wikispaces.com/
>> blog: http://epsg4253.wordpress.com/
>> ------------------------------------------------------
>> _______________________________________________
>> grass-user mailing list
>> grass-user at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-user
>
>