[QGIS-Developer] QGIS, DateTime and TimeZones

[Nyall Dawson]

On Sat, 7 Nov 2020 at 05:50, C Hamilton <adenaculture at gmail.com> wrote:
>
> This was a good discussion that I think is important and wanted to see if it has resulted in any development for better timezone support.
>
> Currently I have a request to provide time conversion tools and coordinate to timezone lookup capability. As I was looking at the python libraries out there it appears that timezonefinder is probably the best for coordinate to timezone support. The only problem is that it is around 50  Mb. My customers generally have great difficulty getting additional libraries installed in QGIS required to run a plugin so I make sure I try to use the libraries already installed with QGIS or else include them with my plugins. I think 50 Mb probably exceeds the maximum size a plugin should be. I could do it for my customers, but it probably would not work as a public plugin.

>
> So is there any current timezone development going on? What are my options for coordinate to timezone lookup? If timezonefinder works for my needs is it possible to have it included as a part of the QGIS distribution? Is there something better?

Looking briefly into this, it seems that all the size of the
timezomefinder library is coming directly from the underlying spatial
layer of timezone boundaries -- that itself is ~50 mb. So ultimately I
don't think you can avoid this cost, unless you simplified the
boundaries and sacrifice accuracy of the results...

But more generally, I'd say you don't need any external library here.
Basically the library is just doing a point in polygon lookup against
the reference boundaries, and that's something which QGIS is already
awesome at ;) I'd suggest you just:
1. find the format which gives smallest file size for the time zone boundaries
2. include this in your plugin
3. use native classes classes to read these boundaries and use
QgsFeatureRequest to do the point in polygon lookups.

Hope that helps!
Nyall



>
> Have a wonderful weekend!!!
>
> Calvin
>
> On Tue, Sep 8, 2020 at 8:20 PM Nyall Dawson <nyall.dawson at gmail.com> wrote:
>>
>> On Tue, 8 Sep 2020 at 23:40, Richard Duivenvoorde <rdmailings at duif.net> wrote:
>> >
>> > Hi Devs,
>> >
>> > I was hitting some timezone issues when I received some csv data in which there are timestamps like:
>> > "2020-07-25 20:21:38 UTC"
>> > Apparently if you read this into QGIS, the UTC IS interpreted, and you get a DateTime in your data which is (in my case) -2 hours(CET) off (well in some parts of QGIS)
>> >
>> > This plays havoc if you for example are going to use this data in the timecontroller or expressions, see
>> > https://github.com/qgis/QGIS/issues/38647
>> >
>> > So I think that now the TimeController is part of QGIS, and people will start to play with Time-based data more and more (from TimeZone aware datasets like csv and postgres) we will run into issues like above.
>> >
>> > Is there something we can do to make QGIS timezone aware?
>> > OR should we stay far away from it (timezones are hell)...
>> > OR should we wait with it after QGIS 4
>>
>> This is a good discussion, and a very complex issue.
>>
>> Here's some summary points from a recent off-list discussion I had
>> regarding this very topic:
>>
>> -  we don't expose time zones anywhere in qgis. while the underlying
>> QDateTime class has good support for timezone handling, there's
>> no-where in QGIS you can "see " the time zone for a datetime value,
>> and there's no functionality present in -the expression engine or
>> processing to work with time zones/manipulate them.
>> - different data providers are handling time zones differently, but
>> most are just giving date time values as a "local time"/"no time zone
>> available" value. As described above, this inconsistency leads to
>> issues when you compare values from different sources. Even though the
>> look the same in QGIS, they aren't!!
>> - some underlying data sources just don't have any time zone
>> capabilities, or have limited capabilities. E.g. GDAL has a flag if a
>> datetime is UTC or "not utc/unknown", and that's it.
>>
>> I think we have no choice but to start exposing time zone information
>> to users everywhere we have dates, and move to treating dates without
>> a timezone as being in "local time" (which may cause issues if
>> projects + data are shared between users in different time zones). We
>> should add timezone functionality to the expression engine, and
>> algorithms for converting time zones to processing. We'd also need to
>> add a "manual time zone override" option to temporal enabled layers --
>> so for those sources with ambiguous time zone information the user has
>> some capacity to manually pick the correct time zone.
>>
>> It's definitely achievable, but it's not a trivial amount of work. I'd
>> estimate we're looking at ~5-7 days to do this properly (and it needs
>> to be done properly, covering all uses of datetime values all
>> throughout QGIS and the different data providers! Hacking in
>> one-by-one fixes in individual data providers will just make the
>> situation worse)...
>>
>> Nyall
>>
>>
>>
>>
>>
>>
>> >
>> > Regards,
>> >
>> > Richard Duivenvoorde
>> >
>> >
>> >
>> > _______________________________________________
>> > QGIS-Developer mailing list
>> > QGIS-Developer at lists.osgeo.org
>> > List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> > Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> _______________________________________________
>> QGIS-Developer mailing list
>> QGIS-Developer at lists.osgeo.org
>> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
>> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer