[postgis-devel] About formats and PostGIS 3

Sat May 11 08:12:02 PDT 2019

пт, 10 мая 2019, 5:31 PM карыстальнік Regina Obe <lr at pcorp.us> напісаў:

> Nicklas,
>
>
>
> I've added additional stuff I know is done, which was not even slated.
>
>
>
> I moved the serialization stuff to In Progress because I think Paul had
> something cooking on that.
>

I believe most of the needed gains were done by Postgres 12 partial
detoasting patch by Paul, so the main driver of "we need format change" -
absence of header-only detoasting - got phased down.

Also moved the GEOS stuff to in progress – some of that might be done.  I
> know there was work being done on palloc but that might be to fix some
> issues rather than as enhancements.
>
>
>
> Others should update as I've been a bit distracted lately myself so
> haven't been following the gyrations of the PostGIS project in the past
> couple of months.
>
>
>
>
>
> *From:* postgis-devel [mailto:postgis-devel-bounces at lists.osgeo.org] *On
> Behalf Of *Nicklas Avén
> *Sent:* Wednesday, May 08, 2019 3:09 AM
> *To:* postgis-devel at lists.osgeo.org
> *Subject:* [postgis-devel] About formats and PostGIS 3
>
>
>
> Hi
>
> I haven't followed the discussions close enough.
>
> Is this page up to date?
>
> http://trac.osgeo.org/postgis/wiki/PostGIS3
> <http://trac.osgeo.org/postgis/wiki/PostGIS3?action=history>
>
>
>
> What ideas have been abandoned and what is getting real?
>
>
>
> I am specifically thinking about the serialization. I think the ideas in
> that wiki page about a generic header and differnt underlying coordinate
> storages, is very interesting.
>
> There is also a bullet point about coordinate precision. This also came up
> in this thread:
>
> https://lists.osgeo.org/pipermail/postgis-devel/2019-May/027895.html
>
>
>
> With a coordinate storage that can gain from less precision we get both
> more compact data and control over the precision of the data.
>
> If the user knows that the accuracy of the data set is only 1 dm it should
> be a win win to not store a lot of random numbers after the first decimal.
>
>
>
> I guess I might be to late to the party, since it is not that many months
> to release, so I am mostly asking for what the status is.
>
>
>
> As I have mentioned in some mails earlier I also have a crazy idea. Since
> a lot of the PostGIS users use it more or less for storage and to render a
> map from that storage it makes sense to optimize for rendering.
>
> In my TilelessMap-app I am now using this technique (will push the new
> code to F-droid quite soon) :
>
> The polygon geometries is stored as twkb with triangulation information.
>
> I have extended the twkb geometry and use one of the unused flags to
> indicate that it includes triangels.
>
> Then, after the coordinates it is a list of indexes into the coordinates
> describing the triangels.
>
> By ordering the triangles in that list and store the indexes as delta
> values it is close to 3 byte storage per triangle also for very big
> polygons.
>
> Those twkb-geometries with triangles is very well suited for GPU-rendering.
>
> Whendecoding the twkb, I get 2 arrays.
>
> First is a list of coordinates and second is an array of triangle indexes.
> Those arrays is exactly what the GPU wants to draw polygons.
>
> So, in short what needs to be done to render those polygons in openGL is:
>
>
>
> load the arrays to the GPU:
>
>    glBufferData(GL_ARRAY_BUFFER, sizeof(GLfloat) *
> number_of_coordinate_values, coordinate_array, GL_STATIC_DRAW);
>
>     glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(GLshort) *
> number_of_index_values, index_array, GL_STATIC_DRAW);
>
> draw the triangles:
>
> glDrawElements(GL_TRIANGLES, n_vals,GL_UNSIGNED_SHORT,(GLvoid*)
> index_offset);
>
>
>
> colors and z-values and so on is sent to the gpu as separate values and
> handled in the shader code.
>
> To recalculate to screen pixels a Matrix is also needded that gets pushed
> to the gpu also handled in the shader code.
>
>
>
> The point is that a format like that is extremely efficient for rendering.
> It can of course be stored also now in PostGIS as a bytea.
>
> But if we had that possibility as a inbuilt type it would be easier for
> something like Qgis to take advantage from that.
>
> As mentioned before, if the user is aware of the accuracy of the data set
> it is a win win win.
>
> 1) Smaller storage
>
> 2) faster rendering
>
> 3) coordinates that don't lie about their accuracy
>
>
>
>
>
> If this model of splitting header and coordiante format as described in
> the wiki page gets real, maybe things like this can be added between major
> releases?
>
> That would mean that a geometry working in PostGIS 3.5 might not work in
> 3.0. That is not a good thing.
>
> But if the header includes enough information it is at least easy to avoid
> a crash and handle things correct. In the above example for instance, the
> added triangles wouldn't stop PostGIS from reading the coordiante part of
> the geometry. But it doesn't understand the triangle part.
>
> I have most of the code for this, but I guess it is too close to release
> for talking this through and get things tested properly.
>
> But the idea to store data as a gpu wants it is appealing.
>
>
>
>
>
> Thanks
>
>
>
> Nicklas
>
>
>
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/postgis-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20190511/864d4579/attachment.html>