[mapserver-dev] Proposal/request to implement FlatGeobuf as built-in format

[Jeff McKenna]

Great point Even, here is an updated result:

   Shapefile       0.011s
   FlatGeobuf      0.014s
   Shapefile (OGR) 0.024s
   GeoPackage      0.042s
   SpatiaLite      0.045s
   PostGIS         0.053s
   GeoJSON         0.089s


-jeff




On 2022-04-27 2:09 p.m., Even Rouault wrote:
> Jeff,
> 
> as a data point, perhaps you could enhance your bench with shapefiles 
> through OGR ? It would be interesting to have a sense of the OGR 
> overhead (that will be the same for any OGR supported datasource) to 
> have an idea if it is really worth the effort to have a native 
> FlatGeobuf provider for MapServer w.r.t going through OGR (it is hard to 
> beat shapefile, and the current difference between shapefile and 
> flatgeobuf is small)
> 
> Even
> 
> Le 26/04/2022 à 12:57, Jeff McKenna a écrit :
>> (to followup from our chat on IRC yesterday)
>>
>> To clarify: this would be a new native format in MapServer, such as 
>> Shapefile or PostGIS, and the goal would be to not connect to 
>> FlatGeobuf through OGR, but instead directly, such as:
>>
>>   DATA countries.fgb
>>
>> instead of:
>>
>>   CONNECTIONTYPE OGR
>>   CONNECTION "countries.fgb"
>>   DATA "countries"
>>
>> This should improve the performance even more.  And it will be easy to 
>> update the documentation, as the existing files just need to be 
>> updated (see the new page at 
>> https://mapserver.org/input/vector/flatgeobuf.html )
>>
>> Great plan Björn!  I look forward to your Pull Request, ha, and will 
>> tackle the documentation changes for you.
>>
>> This is also great news for MapServer, and will give us some nice 
>> publicity.
>>
>> Thanks so much!
>>
>> -jeff
>>
>>
>>
>> On 2022-04-25 5:57 p.m., Björn Harrtell wrote:
>>> Hi mapserver devs!
>>>
>>> I got interested in the subject because of the tests made recently by 
>>> Jeff that shows the potential of the format. I believe it should be 
>>> possible to get significant additional performance out of FlatGeobuf 
>>> in MapServer if it was built in just like Shapefile support is.
>>>
>>> FlatGeobuf C++ implementation is rather small, doesn't require any 
>>> dependencies, and I don't expect it to require any invasive changes 
>>> to existing code.
>>>
>>> Would love to get started on this ASAP, if there are no objections. :)
>>>
>>> Regards,
>>> Björn Harrtell
>>>