[gdal-dev] Open option for vectors in the cloud

[Rahkonen Jukka (MML)]

Hi,

I was experimenting with accessing some vector files through http (same data as FlatGeoBuffers, GeoPackage, and shapefile). The file size in each format was about 850 MB and the amount of data was about 240000 linestrings. I made ogrinfo request with spatial filter that selects one feature and cheched the number of http requests and amount of requested data.

FlatGeoBuffers
19 http requests
33046509 bytes read

GeoPackage
19 http requests
303085 bytes read

shapefile
52 http requests
1425356 bytes read

I try to add the debug log as an attachment but if it does not go through to the mailing list I can explain that in each case most requests and most transferred bytes for used for reading the spatial index. If somebody really finds a use case for reading vector data from the web it seems obvious that having a possibility to cache and re-use the spatial index would be very beneficial. I can imagine that with shapefile it would mean  downloading the .qix file, with GeoPackage reading the contents of the rtree index table, and with FlatGeoBuffers probably extracting the Static packed Hilbert R-tree index.

In my test I had Apache server running on localhost, and at least the speed of GeoPackage gave quite a good feeling.

-Jukka Rahkonen-

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20191024/2a76f839/attachment-0001.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: Vectors_through_http_debug.txt
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20191024/2a76f839/attachment-0001.txt>