<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div><span></span></div><div><div><span></span></div><div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div><span></span></div><div>Remi, sounds like you're close to finding a pathway that matches your needs.</div><div><br></div><div>there are always several ways to solve every problem. It seems yours is about storage and fast access to the bits that are relevant. You mention circa 1 thousand tiles.</div><div><br></div><div>The only reason I'm pursuing this topic is because I'm sensing a broader solution to an emerging set of challenges. Particularly with the "internet of things", where we will collect seriously BIG spatio-temporal data - eg from sensors and we can't afford to pull everything into PostGIS.</div><div><br></div><div>iOT requires solutions that can deal with datasets many orders of magnitude larger, but can also scale down to suit small projects.</div><div><br></div><div><blockquote type="cite"><font color="#000000"><span style="background-color: rgba(255, 255, 255, 0);">using Big Data /noSQL solutions to do some of the simple heavy lifting</span></font></blockquote></div><div><br></div><div><blockquote type="cite"><span>mongo db has got spatial capabilities And you can use the mongo FDW to return query results into PG.</span><br></blockquote><span></span><div><br></div>Mongo db when I last looked at this only has 2d spatial indexing. Not ideal, but still useful for point cloud storage... I also think the attached article is handy because the author lays out the business problem and how he's tackled it.</div><div><blockquote type="cite"><span></span></blockquote><div><br></div><span style="background-color: rgba(255, 255, 255, 0);"><a href="http://rs.tudelft.nl/~rlindenbergh/workshop/BoehmIQmulus.pdf">http://rs.tudelft.nl/~rlindenbergh/workshop/BoehmIQmulus.pdf</a></span><br><br>I'm not advocating that people jump on the NoSQL bandwagon - I love Postgis and prefer to do as much of my work in postgis - but it's good to know each DBs respective strengths and why..<br><br><blockquote type="cite"><span>IMHO, the quicker we become conversant inPolyglot DB design, the better.</span><br></blockquote><blockquote type="cite"><span></span><br></blockquote><blockquote type="cite"><span>If we think every problem is a nail and therefore postgis is the hammer / answer, then innovation ( which is about using and combining existing technologies in new ways) will leave us behind.</span><br></blockquote><div><br></div>Ok, time to park NoSQL and apply some left field thinking..</div><div><br></div><div>take one of Paul's recent innovations...</div><div><br></div><div><span style="background-color: rgba(255, 255, 255, 0);"><a href="https://github.com/pramsey/pgsql-ogr-fdw">https://github.com/pramsey/pgsql-ogr-fdw</a></span></div><div><span style="background-color: rgba(255, 255, 255, 0);"><br></span></div><div><span style="background-color: rgba(255, 255, 255, 0);">What if we could read straight from file... Or "document" into PG...? And I'm not talking every document - just the ones relevant to us... On an as needs basis.</span></div><div><br></div><div>This is what the mongo_fdw does... In fact we can write back too.</div><div><br></div><div>FDW's go to heart of polyglot design.<br><br></div><div>The common theme for IOT is the need to leverage a cheap distributed file / document storage system to enable fast search and retrieval... With a basic level of spatial awareness that can inform and feed downstream applications incl, PostGIS.</div><div><br></div><div>The idea is that stuff not needed or of no business value downstream gets left behind on the cheapest h/w, s/w before discarding.</div></div><div><br></div><div>I'm keen to hear from anyone about how they are using postgis in combination with NoSQL DBs for their big data pipelines ?</div></div><div><br></div><div>Or if there's another mailing list that covers spatial IT / spatial - temporal big data processing, I'd love to hear about it. </div><div><br></div><div>Thanks</div></body></html>