<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
Hi Richard,<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
I think the underlying issue here is that QGIS does not do data management... It just accesses data (whether not managed, managed or mismanaged).</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
You are correct, the traditional GIS model of a feature with attributes does not map to the real world very well in many use cases, and timeseries data is one. GIS is seldom used to manage data these days, with very powerful spatially enabled databases readily
available. To use your IOT approach where location is just an attribute, in a relational db, the location IS just another attribute. Unlike the GIS model, where the spatial feature is somehow different, here it isn't. A record can have multiple geometries,
times, dates, strings, numerics, etc... like start time, end time, start location, end location, trackline between them, if you want... impossible without nasty hacks in a GIS centric data store.
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
So you have a pretty standard situation where you have a set of sites where data is captured in an ongoing basis.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
I have designed Postgis databases to do this... most recently, when Postgres/Postgis was staring to slow down a bit with 600,000,000 readings I moved to Postgres/Postgis/TimescaleDB which is returning typical query results in 10's of milliseconds with 4 billion
readings.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);" class="elementToProof">
You don't just have sites with readings, you have sites, sensors, calibrations, instruments, readings, personnel, etc... there is a whole lot of metadata/data pertaining to your setup that should probably be managed in a database. I don't know if you will have
enough data to justify using TimescaleDB, but you should be using some sort of db to manage your data effectively & efficiently. And Postgis/Postgres is a hard combination to beat as a tool for doing this well.<br>
</div>
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Then you point QGIS at your database for the mapping, cartography, visualisation, etc. You can use database views to simplify queries, but there are issues with QGIS and views. I find using the QGIS DB manager with a query instantiated as a QGIS layer works
much better than accessing a view in many cases.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
And for an alternative mapping of IOT data to a data model, the Object Relational approach provided by Postgres supports non-relational structures like key/value (hstore) or jsonb data storage.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
52North have a (somewhat complex) Postgis database design to provide a data store for their SOS (Sensor Observation Service) software. Perhaps overkill for your use case, but a very complete and robust solution. See:
<a href="https://www.researchgate.net/figure/The-52-North-SOS-schema_fig14_327124727" id="LPNoLPOWALinkPreview">
https://www.researchgate.net/figure/The-52-North-SOS-schema_fig14_327124727</a><br>
</div>
<div class="_Entity _EType_OWALinkPreview _EId_OWALinkPreview _EReadonly_1"></div>
<br>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Hope this helps, there are certainly effective ways to do what you want out there.
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
Cheers<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div id="Signature">
<div>
<div></div>
<div></div>
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div class="BodyFragment">
<div class="PlainText">Brent Wood<br>
<br>
Principal Technician, Fisheries<br>
NIWA<br>
DDI: +64 (4) 3860529</div>
</div>
</div>
</div>
</div>
</div>
<div id="signature_bookmark"></div>
<div id="appendonsend"></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div class="elementToProof"><font style="font-size: 11pt;" class="elementToProof" face="Calibri, sans-serif" color="#000000"><b></b></font>
<div dir="ltr"><br>
<blockquote type="cite">On 2 Sep 2022, at 13:07, Richard Duivenvoorde via QGIS-Developer <qgis-developer@lists.osgeo.org> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr"><span>Hi All,</span><br>
<span></span><br>
<span>Anybody is aware of a model in which data owns a location, but THAT location has a (growing) set of [timestamp, value] observation data to it....</span><br>
<span></span><br>
<span>In GIS/QGIS everything is tabular (talking about vector here, I know meshes can have more shapes).</span><br>
<span></span><br>
<span>But in the IOT world the 'location' is 'just an attribute', and the 'other' data is actually more of interest (and growing in time).</span><br>
<span></span><br>
</div>
<div dir="ltr"><span>Currently to view measurements, 'we' often replicate the location for every value/measurement: we create a 'Feature' for every time step (eg for example via a WFS)</span><br>
</div>
<div dir="ltr"><span></span><br>
<span>The OGC has a SensorThingsApi (STA) standard, in which for given location you can request all values/observations (of filter a certain sensortype/parameter). So one geometry has a full table of data to it.</span><br>
<span></span><br>
<span>BUT to 'work' with that data in QGIS, you always have to 'flatten' it, one way or another, (I think), for every time step: create a feature with: location, time, value...</span><br>
<span></span><br>
<span>My Question:</span><br>
<span></span><br>
<span>- isn't there some model (or can we come up with one) in which a location can actually have a range of time/value data (like some table join like), and when you use the Temporal Controller, you style your layer by RE-using the one location, but getting
the value for current Timestamp (in the TimeController). Some sort of indirect filtering....?</span><br>
<span></span><br>
<span>- or is this the mesh model (I know netcdf's in which you have often predefined meshes/location and the data/time vector is growing)?</span><br>
<span>And would a solution be to have some kind of in memory vector -> mesh loader or so???</span><br>
<span></span><br>
<span>I hope this makes sense to others. Any input appreciated.</span><br>
<span></span><br>
<span>Regards,</span><br>
<span></span><br>
<span>Richard Duivenvoorde</span><br>
<span></span><br>
<span>PS, if you want to play with SensorThingsApi:</span><br>
<span>- install: https://github.com/AirBreak-UIA/SensorThingsAPI_QGIS-plugin</span><br>
<span>- connect it to: https://airquality-frost.k8s.ilt-dmz.iosb.fraunhofer.de/v1.1/Locations</span><br>
<span>(around 5000 location and 500-million 'observations')</span><br>
<span>- load all locations in one layer, and click on a location</span><br>
<span>My point: the data is then viewable/selectable by table and graph, but NOT loadable in a sense way (yet) in QGIS...</span><br>
<span>_______________________________________________</span><br>
<span>QGIS-Developer mailing list</span><br>
<span>QGIS-Developer@lists.osgeo.org</span><br>
<span>List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer</span><br>
<span>Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer</span><br>
</div>
</blockquote>
</div>
<table style="width: 600px; font-size: 8pt; font-family: Arial" cellspacing="20">
<tbody>
<tr>
<td><a href="https://www.niwa.co.nz"><img src="https://www.niwa.co.nz/static/niwa-2018-horizontal-180.png">
</a></td>
<td>
<table style="width: 500px">
<tbody>
<span>Brent Wood <br>
<font color="#1793d2">Principal Technician - GIS and Spatial Data Management</font><br>
<font color="#1793d2">Programme Leader - Environmental Information Delivery</font><br>
+64-4-386-0529<br>
<br>
National Institute of Water & Atmospheric Research Ltd (NIWA)<br>
301 Evans Bay Parade Hataitai Wellington New Zealand<br>
<b>Connect with NIWA:</b> <a href="https://www.niwa.co.nz">niwa.co.nz</a> <a href="https://www.facebook.com/nzniwa">
Facebook</a> <a href="https://www.linkedin.com/company/niwa">LinkedIn</a> <a href="https://twitter.com/niwa_nz">
Twitter</a> <a href="https://www.instagram.com/niwa_science">Instagram</a> </span>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<font face="Arial" size="1">To ensure compliance with legal requirements and to maintain cyber security standards, NIWA's IT systems are subject to ongoing monitoring, activity logging and auditing. This monitoring and auditing service may be provided by third
parties. Such third parties can access information transmitted to, processed by and stored on NIWA's IT systems</font>
</body>
</html>