[OSGeo-Standards] An idea about a compressed binary geometry format [SEC=UNCLASSIFIED]

[Nicklas Avén]

Hallo Bruce

I think it is an open question if a format like TWKB shall handle
attribute data. 

Things gets a lot more complicated at once.

What I have thought quite a lot about is the balance between the header
and the coordinates (or other data). 

Since most of my experience is from PostGIS I have thought a lot about
the output from the database. Then I see the need of that every
outputted geometry can live independently from the others. So all
"schema" information needs to be carried by each geometry. 

If the geometry is the detailed border of a country, it doesn't matter
if the header is 2 or 100 bytes. But if the output is 10 million points,
that will make all the difference.

So we need to handle both an output with 10 million points, but also an
output with 1 single geometry, maybe containing 100 sub geometries of
different types.

If we have that geometry format, that can handle this efficient, then
maybe we should be satisfied. Then the format will not hold the more
global data model, but might of course be a part of a data model.

/Nicklas

On Wed, 2013-08-21 at 10:19 +1000, Bruce Bannerman wrote:
> An emerging trend is for federated data services that support a common
> logical data model for the domain in question.
> 
> 
> This is typically done via GML Application Schema. Some examples are
> GeoSciML, WaterML, AvXML etc.
> 
> 
> In many cases, no one organisation can manage and support data
> required, particularly for regional / global analysis.
> 
> 
> We are finding that source data is maintained by many organisations in
> differing native formats, data structures/models, semantic
> representations etc.
> 
> 
> Currently there is considerable time wasted in processing data from
> many organisations to get it in a form that is suitable for analysis.
> 
> 
> The GML Application Schema approach helps to overcome this issue to a
> large extent, with data providers mapping their internal data to an
> agreed logical data model.
> 
> 
> A binary format that supports this concept may be useful.
> 
> 
> Bruce
> 
> 
> From: Jody Garnett <jody.garnett at gmail.com>
> Date: Wednesday, 21 August 2013 10:01 AM
> To: "nicklas.aven at jordogskog.no" <nicklas.aven at jordogskog.no>
> Cc: "standards at lists.osgeo.org (OSGeo)" <standards at lists.osgeo.org>
> Subject: Re: [OSGeo-Standards] An idea about a compressed binary
> geometry format
> 
> 
> 
> Sounds interesting, any alternative to BinaryXML would be welcome. 
> 
> 
> The header of Binary XML was very complicated, extending into
> information such as endian order. I got the impression it was set up
> around with a server bias. The result was a specification which was
> difficult to implement as a client, and has seen only one
> implementation in open source products I am aware of (in the gvSig
> community).
> 
> 
> 
> 
> On Tue, Aug 20, 2013 at 6:27 AM, Nicklas Avén
> <nicklas.aven at jordogskog.no> wrote:
>         Hallo
>         
>         I do not know if this list is the right place so that will be
>         my first
>         question.
>         
>         I have a naive idea about a compressed binary format for
>         geometries.
>         
>         I have done some work to sketch what it is about.
>         
>         The name so far is TWKB (Tiny WKB)
>         
>         Can this list be the right place to discuss this format. It is
>         just an
>         embryo, and my hope is that it can be something designed by
>         and hard
>         connected to the FOSS4G-community.
>         
>         
>         What is done is:
>         
>         A draft of a spec:
>         https://github.com/nicklasaven/TWKB/blob/master/twkb.md
>         
>         including ideas from Evan Rouault and Oliver Tonnhofer
>         
>         
>         Demos:
>         Streaming TWKB from PostGIS to Leaflet via websocket.
>         http://178.79.156.122/twkb_node/
>         
>         Streaming TWKB-aggregates (tiles created on the fly) from
>         PostGIS to
>         Leaflet
>         http://178.79.156.122/twkb_agg
>         
>         A page to compare twkb in timing and size with geoJSON
>         directly from
>         PostGIS.
>         http://178.79.156.122/twkb_test
>         
>         
>         Examples of implementation.
>         The PostGIS part is in the trunk:
>         http://svn.osgeo.org/postgis/trunk/
>         
>         The client parts is exemplified here:
>         https://github.com/nicklasaven/twkb_web
>         
>         
>         So, first, can this list host discussions about TWKB (or
>         whatever name
>         it gets)?
>         If not where should it go?
>         
>         Second, any feedback is very welcome
>         
>         
>         
>         Best Regards
>         
>         Nicklas Avén
>         
>         _______________________________________________
>         Standards mailing list
>         Standards at lists.osgeo.org
>         http://lists.osgeo.org/mailman/listinfo/standards
> 
>