[gdal-dev] Difficulty opening Safecast GeoJSON data

[Mark Coletti]

On Mon, May 5, 2014 at 4:57 PM, David Fawcett <david.fawcett at gmail.com>wrote:

> On second thought, I see that the .csv file is 2.1 gb!  I would suggest
> not following the .csv driver approach.
>
> I would follow Sean's suggestion and just use some simple Python (or
> whatever language that you want to use to consume the data) to grab the  XY
> values and write out geometries.
>
>
Actually I was able to load the whole shebang into qgis using their CSV
importer.  The problem with that approach is that that file contains
observations for the entire planet, and I just want Japan.  (Particularly
round Fukushima Daiichi.)  Fortunately I can clip to the region of interest
within qgis, though it's admittedly a little sluggish.  ;)

I was interested in their API because I thought that would be an easy way
to do the clipping on their end, while making getting regular updates
viable.  Regularly downloading and processing 2Gb data isn't feasible.

This is the first time I've dealt with "big data", so I'm still getting my
arms round the associated issues.

Cheers!

Mark
-- 
mcoletti at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20140506/e08875ba/attachment.html>