[gdal-dev] Recommended OGR Driver For Very Large Vector Data

[Benjamin Stadin]

Geopackage is not very space efficient, due to the required compatibility with the spec and the „ST*“ methods matching their postgres counterparts. Check the header which is carried over for each geometry:
https://github.com/opengeospatial/geopackage/blob/master/spec/2a_features.adoc

For an example, the  version and SRS is already defined as part of the meta table, but still need to be part of each and every geometry due to technical details.

We do have a proprietary indexed vector tile format which I‘m considering to open source. Though the interest has been rather low in the past to justify putting effort into it. It works like a tile pyramid, using same zoom levels as OSM, but it is projection independent using midpoints of geometries to store vector data into a sinusoidal grid. The grid is used as the tile pyramid storage, the actual projection used by the geometries does not matter other than a projection step between loading and visualization is required. Large line strings like roads are split when writing the data to the storage tiles, and connected dynamically at load time when loading the tiles. Polygons are not split, you‘ll always get a full polygon when querying the tile store (there is additional complexity though for implementing the loading mechanism,into a client app, because it also deals with overlapping polygons. E.g. you could have tiles 1,2,3 in direct „sight“ but the loading mechanism might have to load a tile 5 which overlaps tile 2, which is known as part of the meta info in tile 2).
Storing polygons of different levels of detail for different zoom levels is of course supported. The geometries are wkb-like, but more close to GeoJson (with properties / tags). The storage format for the tiles is described as Google FlatBuffer.

Ben

Am 07.08.2018 um 22:37 schrieb Patrick Young <patrick.mckendree.young at gmail.com<mailto:patrick.mckendree.young at gmail.com>>:

Yeah it's not the geopackage format but that the polygons are just huge, sorry about that.  There is ~1000 polygons but each one is a ton of vertices (~100k), so zoomed out the redraw takes a little while (seconds).  This is true with fileGBD too, which feels slower than the gpkg FWIW.  In practice we have a simplified version of the data we usually deal with and that is plenty fast.

I guess the dream is a format that has "overviews" but for vector data.  Maybe gpkg already supports this idea? I haven't really dug into either format, so apologies for my naivety.

On Sun, Aug 5, 2018 at 11:15 AM, jratike80 <jukka.rahkonen at maanmittauslaitos.fi<mailto:jukka.rahkonen at maanmittauslaitos.fi>> wrote:
Patrick Young wrote
> We also tried a GeoPackage, but it seemed a little slow to read in QGIS.

Hi,

Could you give some more details about your GeoPackage and what you mean
with "little slow"? I have pretty good experiences about using GeoPackage as
data source for MapServer.  I also just tried how a polygon layer with 1.1
million polygons behaves with QGIS and panning and zooming is pretty much
immediate when the map scale is reasonable. With reasonable scale I mean
that roughly 50000 polygons or less fits the map view so that the spatial
index can drop most of the records.

-Jukka Rahkonen-




--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev

_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20180807/15d9373d/attachment.html>