[STATSGRASS] Raster time series and R - timezones?
Markus Neteler
neteler at itc.it
Wed Jun 21 14:54:22 EDT 2006
Thanks, Roger. I'll sketch up a letter and will show it to
you first for critical inspection.
A small excerpt of the data I have posted here
http://mpa.itc.it/markus/modis/
as CSV + SQL table description for both R and PostgreSQL.
Best wishes,
Markus
On Wed, Jun 21, 2006 at 08:17:24PM +0200, Roger Bivand wrote:
> Markus,
>
> I think a replicable example to R-help is the only option now, possibly
> with a longer input file on your web server. If you'd like me to look at a
> draft before you post, I can.
>
> Best wishes,
>
> Roger
>
>
> On Wed, 21 Jun 2006, Markus Neteler wrote:
>
> > On Wed, Jun 21, 2006 at 10:04:28AM +0200, Roger Bivand wrote:
> > > On Wed, 21 Jun 2006, Hamish wrote:
> > > > Markus wrote:
> > > >
> > > > > > > > strptime(mytime,"%d %b %Y %H:%M:%S")
> > > > > > > [1] "2006-04-07 10:30:00"
> > > > > > > -> UTC lost
> > > > > >
> > > > > > add %Z to include timezone?
> > > > >
> > > > > This is the lacking feature... strptime doesn't care of %Z.
> > > >
> > > >
> > > > Then it's R's strptime() which has a problem.
> > > >
> > > > I found %Z on the glibc <time.h> strptime() help page.
> > > >
> > > > Maybe it is best to write a little bash script using `date --iso` or
> > > > tiny C program using time.h's strptime() to pre-process the data?
> > > > I take it R lets you do system() calls somehow.
> > > > versus fighting R for a feature which doesn't exist.
> > > >
> > > > does R have a gmtime() or localtime() equivalent?
> > > >
> > >
> > > The problems seem to stem from R trying to work cross-platform and
> > > cross-locale.
> >
> > For me it would be sufficient to have R operating in UTC,
> > then I would not have the current problems.
> >
> > > The help page for strptime() does say that %Z and %z are
> > > output only.
> >
> > Right (which is a non-feature IMHO).
> >
> > > It is an interaction of platform (and platform libraries) and
> > > the way locales are represented on that platform and the locale affecting
> > > the R process. There have been threads a number of times on this, also
> > > because locales affect i18n etc. Maybe it is time to draft a short message
> > > to R-help:
> > >
> > > Sys.getlocale()
> > > sessionInfo()
> > > mytime <- "7 May 2006 10:30:00 UTC"
> > > t0 <- strptime(mytime,"%d %b %Y %H:%M:%S")
> >
> > [1] "2006-05-07 10:30:00"
> >
> > -> UTC info lost
> >
> > > attributes(t0)
> > $names
> > [1] "sec" "min" "hour" "mday" "mon" "year" "wday" "yday" "isdst"
> >
> > -> maybe this could be parsed for ISOdatetime?
> >
> > > # how to get mytime into ISOdatetime() ?
> > > t1 <- ISOdatetime(t0$year+1900, t0$mon+1, t0$mday, t0$hour, t0$min,
> > > t0$sec, tz="UTC")
> > > t1
> > > attributes(t1)
> > >
> > > Are we getting anywhere??
> > >
> > > Roger
> >
> > I have now tried another way: Import of CSV into PostgreSQL
> > to later use Rdbi to fetch the data:
> >
> > DROP TABLE modis_lst_meteo;
> > CREATE TABLE modis_lst_meteo (
> > station varchar(5),
> > east double precision,
> > north double precision,
> > LST double precision,
> > obs_data timestamp with time zone,
> > modis_map varchar(40)
> > );
> > SET DATESTYLE To European;
> > COPY modis_lst_meteo FROM stdin USING DELIMITERS ';';
> > [input data ordered by station,date]
> > ...
> >
> >
> > A query result (where station='53') is this:
> > ...
> > 53;2000-03-08 11:30:00+01;modis_lst1km2000038.LST_Day_1km.filt
> > 53;2000-03-08 14:30:00+01;aqua_lst1km2000038.LST_Day_1km.filt
> > 53;2000-03-08 23:30:00+01;modis_lst1km2000038.LST_Night_1km.filt
> > 53;2000-03-08 02:30:00+01;aqua_lst1km2000038.LST_Night_1km.filt
> > 53;2000-03-09 11:30:00+01;modis_lst1km2000039.LST_Day_1km.filt
> > 53;2000-03-09 14:30:00+01;aqua_lst1km2000039.LST_Day_1km.filt
> > 53;2000-03-09 23:30:00+01;modis_lst1km2000039.LST_Night_1km.filt
> > 53;2000-03-09 02:30:00+01;aqua_lst1km2000039.LST_Night_1km.filt
> > 53;2000-03-10 10:30:00+01;modis_lst1km20000310.LST_Day_1km.filt <-switch
> > 53;2000-03-10 14:30:00+01;aqua_lst1km20000310.LST_Day_1km.filt <-no-switch?
> > 53;2000-03-10 22:30:00+01;modis_lst1km20000310.LST_Night_1km.filt <-switch
> > 53;2000-03-10 02:30:00+01;aqua_lst1km20000310.LST_Night_1km.filt <-no-switch?
> > 53;2000-03-11 10:30:00+01;modis_lst1km20000311.LST_Day_1km.filt
> > 53;2000-03-11 14:30:00+01;aqua_lst1km20000311.LST_Day_1km.filt
> > 53;2000-03-11 22:30:00+01;modis_lst1km20000311.LST_Night_1km.filt
> > 53;2000-03-11 02:30:00+01;aqua_lst1km20000311.LST_Night_1km.filt
> > 53;2000-03-12 10:30:00+01;modis_lst1km20000312.LST_Day_1km.filt
> > ...
> >
> > So:
> > - 11:30:00+01 is changed to 10:30:00+01
> > - 14:30:00+01 and 02:30:00+01 remain ?!
> > - 23:30:00+01 is changed to 02:30:00+01
> >
> > and it happens 2000-03-10 (a bit early in the year? Can't find the date
> > of daylight saving switch for 2000 in Europe right now).
> >
> > Aah, a few days later in the DB:
> >
> > 53;2000-03-25 10:30:00+01;modis_lst1km20000325.LST_Day_1km.filt
> > 53;2000-03-25 14:30:00+01;aqua_lst1km20000325.LST_Day_1km.filt
> > 53;2000-03-25 22:30:00+01;modis_lst1km20000325.LST_Night_1km.filt
> > 53;2000-03-25 02:30:00+01;aqua_lst1km20000325.LST_Night_1km.filt
> > 53;2000-03-26 10:30:00+02;modis_lst1km20000326.LST_Day_1km.filt <- +2h
> > 53;2000-03-26 15:30:00+02;aqua_lst1km20000326.LST_Day_1km.filt <-switch
> > 53;2000-03-26 22:30:00+02;modis_lst1km20000326.LST_Night_1km.filt
> > 53;2000-03-26 03:30:00+02;aqua_lst1km20000326.LST_Night_1km.filt <-switch
> >
> > Not really convincing. Moreover frightening that the DST switch
> > isn't happening at the same day in PostgreSQL for different times
> > of the day. 2000-03-26 sounds reasonable to me, but the morning time
> > was already (partially) changes earlier the month. Uffa.
> > Maybe I am missing something essential here (PostgreSQL 8)?
> >
> > Best would be to set R somehow to UTC and to not do anything with
> > daylight saving time since I don't need it.
> >
> > Markus
> >
> > _______________________________________________
> > statsgrass mailing list
> > statsgrass at grass.itc.it
> > http://grass.itc.it/mailman/listinfo/statsgrass
> >
>
> --
> Roger Bivand
> Economic Geography Section, Department of Economics, Norwegian School of
> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
> e-mail: Roger.Bivand at nhh.no
>
--
Markus Neteler <neteler itc it> http://mpa.itc.it/markus/
ITC-irst - Centro per la Ricerca Scientifica e Tecnologica
MPBA - Predictive Models for Biol. & Environ. Data Analysis
Via Sommarive, 18 - 38050 Povo (Trento), Italy
More information about the grass-stats
mailing list