[postgis-devel] [postgis-users] PSC Vote: Keep or drop Flatgeobuf in PostGIS 3.2.0

Wed Nov 24 15:10:02 PST 2021

As mentioned on IRC just reiterated here.  

I would like to see ST_AsFlatGeoBuf in there. Here are my reasons.

1) To Bruce’s point that sure ogr_fdw can read it, I’m more interested in the writing of it (which at least I can’t do from scratch with ogr_fdw) and getting out of the database onto the command line is not one of my great ambitions in life.

2) The fact it is a lighter weight format than ST_AsGeoJSON which has the same 1GB limitation.

Or as mentioned https://flatgeobuf.org/ a better shapefile format (for < 1GB)

And the fact that FlatGeobuf is about 50% lighter than GeoJSON 

>From site:

  (shapefile 1, FlatGeoBuf 0.77,  GeoJSON  1.2)

Means I can stuff much more data in the 1GB limit than I can in GeoJSON.

Though not sure about how that places with the other attribute data if they are compacted as efficiently.

3) Given it is  demonstrated it can work fine with leaflet and openlayers means it is possible to hook it into something like 

https://github.com/CrunchyData/pg_featureserv 

as an alternative format to geojson (not saying you should Martin Davis – just throwing it out there :) )

4) One more reason to fight that 1GB limit – as postgis raster is impacted by that too.

Thanks,

Regina

From: postgis-users [mailto:postgis-users-bounces at lists.osgeo.org] On Behalf Of Björn Harrtell
Sent: Wednesday, November 24, 2021 5:26 PM
To: PostGIS Development Discussion <postgis-devel at lists.osgeo.org>
Cc: PostGIS Users Discussion <postgis-users at lists.osgeo.org>
Subject: Re: [postgis-users] [postgis-devel] PSC Vote: Keep or drop Flatgeobuf in PostGIS 3.2.0

Even though I spent quite a bit of effort on implementing this stuff and I'm sure I can fix the crashers I agree with the arguments to remove it. That is, 1GB limit is really bad and better to use GDAL which has a well maintained impl of it.

If there was a way to stream in and out binary with custom encoding and no size limit (i.e COPY with custom/ext binary format) it could make sense but I don't think that is going to happen any time soon.

Oh well, it was fun. Some of it. 😂

PS. ST_AsGeobuf should be deprecated/removed too - it's even less useful IMHO.

PS2. I do still believe in FlatGeobuf and it is used in production. ;)

/Björn

Den ons 24 nov. 2021 22:26Bruce Rindahl <bruce.rindahl at gmail.com <mailto:bruce.rindahl at gmail.com> > skrev:

FWIW I say remove it and seriously think about not including it at all.  Looks like you can use the format right now via ogr_fdw using GDAL. 

On Wed, Nov 24, 2021 at 12:51 PM Regina Obe <lr at pcorp.us <mailto:lr at pcorp.us> > wrote:

FWIW it’s already in GDAL since 3.1 and yah GDAL is a better home since it doesn’t have the  1GB PostgreSQL limitation

https://gdal.org/drivers/vector/flatgeobuf.html

Also here are OpenLayers and Leaflet examples for those not familiar with the format

OpenLayers: https://flatgeobuf.org/examples/openlayers/

Leaflet: https://flatgeobuf.org/examples/leaflet/

Thanks,

Regina

From: postgis-users [mailto:postgis-users-bounces at lists.osgeo.org <mailto:postgis-users-bounces at lists.osgeo.org> ] On Behalf Of Darafei "Kom?pa" Praliaskouski
Sent: Wednesday, November 24, 2021 3:27 PM
To: PostGIS Development Discussion <postgis-devel at lists.osgeo.org <mailto:postgis-devel at lists.osgeo.org> >
Cc: PostGIS Users Discussion <postgis-users at lists.osgeo.org <mailto:postgis-users at lists.osgeo.org> >
Subject: Re: [postgis-users] [postgis-devel] PSC Vote: Keep or drop Flatgeobuf in PostGIS 3.2.0

Hi,

I have not seen flatgeobuf in the wild, and I believe it can be safely removed. 

The current implementation is impaired by Postgres' life choices of 1GB limit and thus not usable for any data, just size-limited subset. ogr2ogr seems like a better suited place for it to reside.

I'm -0 on adding flatgeobuf to core, and -1 on releasing with known crashers. This would converge to "remove if nobody can fix crashers".

On Wed, Nov 24, 2021 at 11:10 PM Regina Obe <lr at pcorp.us <mailto:lr at pcorp.us> > wrote:

This is a PSC vote, but we would like some feedback on this from packagers
and users as such comments will sway our vote.

We have two blockers that center around the new FlatGeoBuf format.

https://trac.osgeo.org/postgis/ticket/5005  (this one is easily
replicatable)

https://trac.osgeo.org/postgis/ticket/5014 (this one I can only replicate
with the cowbuilder setup Bas Cowenberg provided)

both I think are manifestations of the same problem how the header is
derived and what it's doing with numeric and geometry fields.

I've taken a stab at troubleshooting and fixing, but did not have much luck.
That said, if anyone is willing to help fix that would be great and fix
within a 1 to 2 week time period.

If not I feel that we really need to take it out of our PostGIS 3.2.0
release (which will be going on to 3.2.0beta2).

I'd like to release PostGIS 3.2.0beta2 in about a week or so with flatgeobuf
fixed or removed.  If removed, we'll  push flatgeobuf to PostGIS 3.3.0
cycle.

Thanks,
Regina

_______________________________________________
postgis-devel mailing list
postgis-devel at lists.osgeo.org <mailto:postgis-devel at lists.osgeo.org> 
https://lists.osgeo.org/mailman/listinfo/postgis-devel

_______________________________________________
postgis-devel mailing list
postgis-devel at lists.osgeo.org <mailto:postgis-devel at lists.osgeo.org> 
https://lists.osgeo.org/mailman/listinfo/postgis-devel

_______________________________________________
postgis-devel mailing list
postgis-devel at lists.osgeo.org <mailto:postgis-devel at lists.osgeo.org> 
https://lists.osgeo.org/mailman/listinfo/postgis-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20211124/23455514/attachment-0001.html>