[GRASS-user] Starspan/GRASS question

[Thomas Adams]

List:

My apologies for the background info. leading-up to my question…

I've seen mention of starspan previously and now I think it's time for 
me to learn more. Let me pose a problem to you to see if starspan would 
be of help. I am working on a modeling project with a couple of other 
people that involves the following:

(1) downloading and decoding gridded fields of numerical weather 
prediction (NWP) model output
(2) ingesting the decoded data into GRASS to calculate basin average 
precipitation & temperature (separate grids) for each subbasin
(3) writing out all calculated basin average values for each grid to 
separate files (one file contains one time step of data for all 
subbasins) for both temperature & precipitation
(4) for data management reasons, the files from (3) are written to a 
PostgreSQL database
(5) once all time steps of gridded precipitation and temperature field 
data are written to the PostgreSQL database, another process collects 
the data and generates individual ascii time series files for each 
subbasin for both temperature & precipitation
(6) once (5) is completed a hydrologic model runs using the temperature 
& precipitation time series as input and hydrologic forecasts are 
generated.

This is suppose to be a *real-time* process. The problem I am having is 
a matter of scale. What I did not say is that there are 12 different 
sets of NWP output covering a period of 168 hours at 6-hour time steps 
for both temperature & precipitation. So, this means I must process 
12*2*(168/6) = 672 grids. Also, I need the mean areal values for 686 
subbasins within the domain for each of the grids.

Steps (2-4) take about 20 seconds total for each of the grids… which is 
~7.5 hours total
Step-5 also takes about 20 seconds for each time series file… which is 
~7.5 hours total

So, about 15 hours total. Now, I can cut this time in half by running 
the processing of the temperature & precipitation grids and generating 
their separate time series files in parallel, rather than sequentially. 
So, I can get to about 7 hours fairly easily — what I am shooting for is 
to get the processing time from 7 hours to about 3 hours (or less).

I need to more efficiently generate the many basin average time series 
files from the numerous grids. Can starspan help by reducing the time to 
calculate the the basin average values faster?

I would also appreciate any/all suggestions on how to efficiently go 
from 'starspan generated basin average values' to my time series files. 
Realize, of course, the the individual grids are only a slice in time, 
so I have to track the grids and their resulting individual basin values 
(in time) to generate the time series files.

To compound the problem, very soon, I need to add model grids from an 
additional 21 models, bringing the total from 12 to 33!

Regards,
Tom

-- 
Thomas E Adams
National Weather Service
Ohio River Forecast Center
1901 South State Route 134
Wilmington, OH 45177

EMAIL:	thomas.adams at noaa.gov

VOICE:	937-383-0528
FAX:	937-383-0033