[postgis-devel] [postgis-users] PSC Vote: Keep or drop Flatgeobuf in PostGIS 3.2.0

[Bruce Rindahl]

Sufficiently chastised.  Based on Regina's comments, I might have a use
case for it.

On Wed, Nov 24, 2021 at 3:10 PM Regina Obe <lr at pcorp.us> wrote:

> As mentioned on IRC just reiterated here.
>
>
>
> I would like to see ST_AsFlatGeoBuf in there. Here are my reasons.
>
>
>
> 1) To Bruce’s point that sure ogr_fdw can read it, I’m more interested in
> the writing of it (which at least I can’t do from scratch with ogr_fdw) and
> getting out of the database onto the command line is not one of my great
> ambitions in life.
>
>
>
> 2) The fact it is a lighter weight format than ST_AsGeoJSON which has the
> same 1GB limitation.
>
> Or as mentioned https://flatgeobuf.org/ a better shapefile format (for <
> 1GB)
>
> And the fact that FlatGeobuf is about 50% lighter than GeoJSON
>
> From site:
>
>   (shapefile 1, FlatGeoBuf 0.77,  GeoJSON  1.2)
>
>
>
> Means I can stuff much more data in the 1GB limit than I can in GeoJSON.
>
> Though not sure about how that places with the other attribute data if
> they are compacted as efficiently.
>
>
>
> 3) Given it is  demonstrated it can work fine with leaflet and openlayers
> means it is possible to hook it into something like
>
> https://github.com/CrunchyData/pg_featureserv
>
>
>
> as an alternative format to geojson (not saying you should Martin Davis –
> just throwing it out there J )
>
>
>
> 4) One more reason to fight that 1GB limit – as postgis raster is impacted
> by that too.
>
>
>
>
>
> Thanks,
>
> Regina
>
>
>
> *From:* postgis-users [mailto:postgis-users-bounces at lists.osgeo.org] *On
> Behalf Of *Björn Harrtell
> *Sent:* Wednesday, November 24, 2021 5:26 PM
> *To:* PostGIS Development Discussion <postgis-devel at lists.osgeo.org>
> *Cc:* PostGIS Users Discussion <postgis-users at lists.osgeo.org>
> *Subject:* Re: [postgis-users] [postgis-devel] PSC Vote: Keep or drop
> Flatgeobuf in PostGIS 3.2.0
>
>
>
> Even though I spent quite a bit of effort on implementing this stuff and
> I'm sure I can fix the crashers I agree with the arguments to remove it.
> That is, 1GB limit is really bad and better to use GDAL which has a well
> maintained impl of it.
>
>
>
> If there was a way to stream in and out binary with custom encoding and no
> size limit (i.e COPY with custom/ext binary format) it could make sense but
> I don't think that is going to happen any time soon.
>
>
>
> Oh well, it was fun. Some of it. 😂
>
>
>
> PS. ST_AsGeobuf should be deprecated/removed too - it's even less useful
> IMHO.
>
>
>
> PS2. I do still believe in FlatGeobuf and it is used in production. ;)
>
>
>
> /Björn
>
> Den ons 24 nov. 2021 22:26Bruce Rindahl <bruce.rindahl at gmail.com> skrev:
>
> FWIW I say remove it and seriously think about not including it at all.
> Looks like you can use the format right now via ogr_fdw using GDAL.
>
>
>
> On Wed, Nov 24, 2021 at 12:51 PM Regina Obe <lr at pcorp.us> wrote:
>
> FWIW it’s already in GDAL since 3.1 and yah GDAL is a better home since it
> doesn’t have the  1GB PostgreSQL limitation
>
>
>
> https://gdal.org/drivers/vector/flatgeobuf.html
>
>
>
> Also here are OpenLayers and Leaflet examples for those not familiar with
> the format
>
>
>
> OpenLayers: https://flatgeobuf.org/examples/openlayers/
>
>
>
>
>
> Leaflet: https://flatgeobuf.org/examples/leaflet/
>
>
>
> Thanks,
>
> Regina
>
>
>
> *From:* postgis-users [mailto:postgis-users-bounces at lists.osgeo.org] *On
> Behalf Of *Darafei "Kom?pa" Praliaskouski
> *Sent:* Wednesday, November 24, 2021 3:27 PM
> *To:* PostGIS Development Discussion <postgis-devel at lists.osgeo.org>
> *Cc:* PostGIS Users Discussion <postgis-users at lists.osgeo.org>
> *Subject:* Re: [postgis-users] [postgis-devel] PSC Vote: Keep or drop
> Flatgeobuf in PostGIS 3.2.0
>
>
>
> Hi,
>
> I have not seen flatgeobuf in the wild, and I believe it can be safely
> removed.
>
>
>
> The current implementation is impaired by Postgres' life choices of 1GB
> limit and thus not usable for any data, just size-limited subset. ogr2ogr
> seems like a better suited place for it to reside.
>
>
>
> I'm -0 on adding flatgeobuf to core, and -1 on releasing with known
> crashers. This would converge to "remove if nobody can fix crashers".
>
>
>
>
>
> On Wed, Nov 24, 2021 at 11:10 PM Regina Obe <lr at pcorp.us> wrote:
>
> This is a PSC vote, but we would like some feedback on this from packagers
> and users as such comments will sway our vote.
>
> We have two blockers that center around the new FlatGeoBuf format.
>
> https://trac.osgeo.org/postgis/ticket/5005  (this one is easily
> replicatable)
>
> https://trac.osgeo.org/postgis/ticket/5014 (this one I can only replicate
> with the cowbuilder setup Bas Cowenberg provided)
>
> both I think are manifestations of the same problem how the header is
> derived and what it's doing with numeric and geometry fields.
>
> I've taken a stab at troubleshooting and fixing, but did not have much
> luck.
> That said, if anyone is willing to help fix that would be great and fix
> within a 1 to 2 week time period.
>
> If not I feel that we really need to take it out of our PostGIS 3.2.0
> release (which will be going on to 3.2.0beta2).
>
> I'd like to release PostGIS 3.2.0beta2 in about a week or so with
> flatgeobuf
> fixed or removed.  If removed, we'll  push flatgeobuf to PostGIS 3.3.0
> cycle.
>
> Thanks,
> Regina
>
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/postgis-devel
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/postgis-devel
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/postgis-devel
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/postgis-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20211124/f9bcb76d/attachment.html>