[gdal-dev] Difficulty opening Safecast GeoJSON data

Mon May 5 13:43:22 PDT 2014

Mark,

You could use the GDAL/OGR .csv driver with a VRT to access the data and
spit it out into a different, filtered file.

http://www.gdal.org/ogr/drv_csv.html

You could also use the VSI to pull from a .zip'd file via URL.

You could also likely use args with ogr2ogr to filter by attribute or bbox,
and even reproject the data on the way out.

http://www.gdal.org/ogr2ogr.html

David.

On Mon, May 5, 2014 at 3:29 PM, Mark Coletti <mcoletti at gmail.com> wrote:

> Thank you Even and Sean for your very quick and salient replies!  I'll
> notify the Safecast folks that their data might be better served using
> GeoJSON; meanwhile, they also make their open source data available as a
> single downloadable file in CSV format.  Since qgis can import files of
> that type, I've been using that; it's just that I have to do a lot more
> work on my end to pare the data down to the region of interest.  (And deal
> with the inevitable data corruption.  E.g., I suspect the observations from
> 2020 are bogus unless they have a time traveler volunteering radiation
> readings. =) )
>
> Again, thank you for your help!
>
> Cheers,
>
> Mark
>
>
> On Mon, May 5, 2014 at 4:12 PM, Sean Gillies <sean at mapbox.com> wrote:
>
>> Hi Mark,
>>
>> Safecast's JSON isn't GeoJSON, but wouldn't be too hard to crosswalk over
>> to GeoJSON... make a geometry from its latitude/longitude, stuff all the
>> other attributes in properties.
>>
>>
>> On 5/5/14, 1:56 PM, Mark Coletti wrote:
>>
>>>
>>> Safecast is user sourced radiation data, and has a public API serving
>>> data in GeoJSON format (https://api.safecast.org/en-US/home).  However,
>>> I'm having difficulty in accessing this data via OGR:
>>>
>>> % ogrinfo -so
>>> "https://api.safecast.org/en-US/measurements?distance=10&
>>> latitude=37.42&longitude=141.033"
>>> ERROR 4: Failed to read GeoJSON data
>>> ERROR 4: Failed to read GeoJSON data
>>> FAILURE:
>>> Unable to open datasource
>>> `https://api.safecast.org/measurements.json?distance=10&
>>> latitude=37.42&longitude=141.033'
>>> with the following drivers.
>>>
>>> So OGR sees that the data is formatted as GeoJSON, but is unable to read
>>> it for some reason.
>>>
>>> Using curl to manually make the query I get:
>>>
>>> % curl  -i -H "Accept: application/json"
>>> "https://api.safecast.org/en-US/measurements?distance=10&
>>> latitude=37.42&longitude=141.033"
>>> HTTP/1.1 200 OK
>>> Server: nginx
>>> Date: Mon, 05 May 2014 19:43:55 GMT
>>> Content-Type: application/json; charset=utf-8
>>> Transfer-Encoding: chunked
>>> Connection: keep-alive
>>> Status: 200 OK
>>> Strict-Transport-Security: max-age=31536000
>>> Access-Control-Allow-Origin: safecast.org <http://safecast.org>
>>>
>>> Access-Control-Allow-Methods: POST, GET, OPTIONS
>>> Access-Control-Allow-Headers: *, X-Requested-With
>>> Access-Control-Max-Age: 100000
>>> X-UA-Compatible: IE=Edge,chrome=1
>>> ETag: "e4650539a53d3698d8deb02e912b9ae3"
>>> Cache-Control: max-age=0, private, must-revalidate
>>> X-Request-Id: a30a38247c16374a39ada0a8d4d2fcde
>>> X-Runtime: 0.022855
>>> X-Rack-Cache: miss
>>> X-Powered-By: cloud66
>>>
>>> [{"captured_at":"2014-01-30T05:03:25Z","device_id":
>>> null,"id":30981293,"location_name":null,"original_id":null,
>>> "unit":"cpm","user_id":3,"value":4675.0,"latitude":37.
>>> 41997666666666,"longitude":141.03297166666667},{"
>>> captured_at":"2014-01-30T05:25:26Z","device_id":null,"id":
>>> 30981556,"location_name":null,"original_id":null,"unit":"
>>> cpm","user_id":3,"value":2470.0,"latitude":37.
>>> 42001666666667,"longitude":141.03303666666667},{"
>>> captured_at":"2014-01-30T05:00:54Z","device_id":null,"id":
>>> 30981263,"location_name":null,"original_id":null,"unit":"
>>> cpm","user_id":3,"value":5428.0,"latitude":37.
>>> 41996666666667,"longitude":141.03301666666667},{"
>>> captured_at":"2014-01-30T05:21:20Z","device_id":null,"id":
>>> 30981507,"location_name":null,"original_id":null,"unit":"
>>> cpm","user_id":3,"value":12191.0,"latitude":37.
>>> 420003333333334,"longitude":141.03305333333333},{"
>>> captured_at":"2014-01-30T05:00:49Z","device_id":null,"id":
>>> 30981262,"location_name":null,"original_id":null,"unit":"
>>> cpm","user_id":3,"value":5199.0,"lat
>>>
>> i
>> tude":37.41999666666667,"longitude":141.03306166666667}
>> ,{"captured_at":"2014-01-30T05:25:21Z","device_id":
>> null,"id":30981555,"location_name":null,"original_id":null,
>> "unit":"cpm","user_id":3,"value":2423.0,"latitude":37.
>> 42006833333333,"longitude":141.03301333333334},{"
>> captured_at":"2014-01-30T05:03:30Z","device_id":null,"id":
>> 30981294,"location_name":null,"original_id":null,"unit":"
>> cpm","user_id":3,"value":4456.0,"latitude":37.41992833333333,"longitude":
>> 141.03297333333333},{"captured_at":"2014-01-30T05:
>> 00:59Z","device_id":null,"id":30981264,"location_name":null,
>> "original_id":null,"unit":"cpm","user_id":3,"value":5672.0,"latitude":37.
>> 419963333333335,"longitude":141.03291666666667},{"
>> captured_at":"2014-01-30T05:00:44Z","device_id":null,"id":
>> 30981261,"location_name":null,"original_id":null,"unit":"
>> cpm","user_id":3,"value":5135.0,"latitude":37.420035,"
>> longitude":141.03309},{"captured_at":"2014-01-30T05:
>> 21:14Z","device_id":null,"id":30981506,"location_name":null,
>> "original_id":null,"unit":"c
>> p
>> m","user_id":3,"value":12164.0,"latitude":37.42006333333333,"longitude":
>> 141.03293666666667}]
>>
>>>
>>> Those do look like legitimate observations, but I don't know if they
>>> comply to the GeoJSON format since I'm new to JSON.
>>>
>>> I'm using this version of GDAL/OGR, which was part of the pre-packaged
>>> binaries for qgis for my Mac:
>>> % ogrinfo --version
>>> GDAL 1.10.1, released 2013/08/26
>>>
>>> Eventually I'd like to use GDAL/OGR to conveniently create a local
>>> spatial database from such a query.  However, I need to get over this
>>> hurdle first.
>>>
>>> Cheers,
>>>
>>> Mark
>>> --
>>> mcoletti at gmail.com <mailto:mcoletti at gmail.com>
>>>
>>>
>>>
>>> _______________________________________________
>>> gdal-dev mailing list
>>> gdal-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>
>>>
>> --
>> Sean Gillies
>> sean at mapbox.com
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>
>
>
> --
> mcoletti at gmail.com
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20140505/2bfbf467/attachment.html>