<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">I saw a demo of the Crayfish plugin at
the FOSS4G-UK conference earlier in the week, and it provides
'time slider' functionality for netCDF based data, plus additional
tools such as video generation. Worth taking a look?<br>
<br>
Andy<br>
<br>
On 17/06/2016 10:29, Régis Haubourg wrote:<br>
</div>
<blockquote
cite="mid:CABgOYCcX5z+tixPXDyE5QCDd20L2xooJfgqQwedVSuiaHphMSA@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">Hi,
<div>I need the communities lights!</div>
<div><br>
<div>I'm starting to work with huge meteo datasets composed of
a grid of point layers, and hundred of millions of rainfall
/ temperature data. </div>
<div><br>
</div>
<div>Datasets are delivered in a custom text format, so I'm
digging around on what are the best formats for storage, use
in postgis and QGIS. </div>
<div>I would like to be able to :</div>
<div> - run timeManager to generate videos</div>
<div> - display data averaged on day / month / year (or any
other) timeframe</div>
<div> - feed R analyses. </div>
<div><br>
</div>
<div>Up to now, I tried the following paths: </div>
<div> - netcdf / grib: ideal for data storage: </div>
<div> Pros : GDAL and QGIS can view it. R And python
scipy have providers for that</div>
<div> Cons : not easy to generate from exotic
datasources, Current QGIS Netcdf explorer or core date
visualisation (time frame = raster bands) are not handy for
daily data over decades (about 10 000 days available in my
dataset). I didn't manage to build netcdf yet, FME or GDAL
are a bit dry.. </div>
<div><br>
</div>
<div> - load all in postgres / postgis relationnal model: </div>
<div> Pros: available for all clients and fast, if
data is correctly indexed and designed/ </div>
<div> Cons: performance requires a table (not a view
because of lack of estimated metadata for extent computing)
with redondancy over point location. I tried a first
approach with a small geographical table for my point grid
and a value table. With correct indexing and clustering, I
get good performance in psql but very poor in QGIS. First
load is slooow because of the st_extent query, but also
every fetch afterwards, even if I filter on a date frame
(with good index). I didn't expect it to be slow on fetch.. </div>
<div><br>
</div>
<div>Another point with postgis storage, TimeManager plugin
does not like true date datatype, date cast to char truncate
date to first character, so I have to expose my datasets
with a text format in my view, which is not quite efficient.
(I will create a ticket upstream) </div>
<div><br>
</div>
<div><b>Does anyone has any experience and advices on that
field ? </b></div>
<div><b><br>
</b></div>
<div> I saw that postgis has a datacube type, could that be a
way to store data more efficiently? Could QGIS read it?
Should I stick with netcdf ? </div>
<div><br>
</div>
<div>
<div>Thanks a lot </div>
<div><br>
</div>
-- <br>
<div class="gmail_signature"
data-smartmail="gmail_signature">Régis <br>
</div>
</div>
</div>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Qgis-user mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Qgis-user@lists.osgeo.org">Qgis-user@lists.osgeo.org</a>
List info: <a class="moz-txt-link-freetext" href="http://lists.osgeo.org/mailman/listinfo/qgis-user">http://lists.osgeo.org/mailman/listinfo/qgis-user</a>
Unsubscribe: <a class="moz-txt-link-freetext" href="http://lists.osgeo.org/mailman/listinfo/qgis-user">http://lists.osgeo.org/mailman/listinfo/qgis-user</a></pre>
</blockquote>
<br>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Andy Harfoot
GeoData Institute
University of Southampton
Southampton
SO17 1BJ
Tel: +44 (0)23 8059 2719
Fax: +44 (0)23 8059 2849
<a class="moz-txt-link-abbreviated" href="http://www.geodata.soton.ac.uk">www.geodata.soton.ac.uk</a>
</pre>
</body>
</html>