[postgis-devel] lwgeom_*_in_place()

[Daniel Baston]

For profiling, I run postgres in single-user mode, through valgrind
--tool=callgrind. It works really well, though it slows execution down so
much that it is sometimes challenging to profile realistic queries.

Kcachegrind is a nice tool for viewing the callgrind output.

Dan

On Mon, Oct 2, 2017 at 8:19 PM, Darafei "Komяpa" Praliaskouski <
me at komzpa.net> wrote:

> Hello,
>
> Can you please share the way you are using profiler on postgis?
> I'd like to profile sorting by geometry. For now I've found for myself
>
> objdump -S --disassemble /usr/lib/liblwgeom-2.5.so.0 |less
>
> ... but that doesn't scale well enough. Still allows to see that
> geometry/geography if's reading bits of header with something hidden three
> files away under #define aren't a simple thing.
>
> For _in_place suggestion, I think it's enough to make sure it's done for
> the critical path of MVT and codified somewhere as a guideline.
>
> вт, 3 окт. 2017 г. в 0:53, Paul Ramsey <pramsey at cleverelephant.ca>:
>
>> Hey devs,
>> So, in pursuit of spare CPU cycles for the good folks at Carto, I've been
>> putting various workloads through a profiler. Some of the results you've
>> seen in recent tweaks committed.
>>
>> One thing I found as a general rule was that as the number of simple
>> objects goes up, the more noticeable the overhead of things like memory
>> allocation / deallocation becomes. So the overhead of just constructing an
>> LWPOINT on a point, when you're slamming through 1M of them, is not nothing.
>>
>> For some workloads, like generating MVT (there it is) we call numerous
>> geometry processing functions in a row. For some functions, that expect
>> const LWGEOM inputs, that can result in a lot of cloning of objects, and
>> hence, memory management. For other functions, that expect to modify things
>> in place, the coordinates are just altered right there.
>>
>> This has always been a little ugliness in our API, and for a while we
>> settled on defaulting to making copies, so the API was at least getting
>> cleaner. But also getting slower.
>>
>> I built a prototype for generating MVT that does all the coordinate
>> processing in place, and for bulk simple objects it ran about 30% faster
>> than the existing code. There's a benefit to be had to in working in place.
>>
>> I propose that we... cut the baby in half.
>>
>> * All liblwgeom functions that do in place coordinate modification should
>> be re-named to *_in_place() and a bare version that takes a const LWGEOM
>> and returns a copy as necessary be added.
>> * All liblwgeom functions that already take const LWGEOM be supplemented
>> with *_in_place() variants so that it's possible to build processing chains
>> that are capable of doing processing with a minimal allocation/deallocation
>> footprint.
>>
>> Functions on my list immediately:
>>
>>   * remove_repeated_points
>>   * simplify
>>   * affine
>>   * grid
>>   * transform
>>
>> I will, however, do a full review and attempt to ensure all non-GEOS
>> functions have in_place variants available in liblwgeom.
>>
>> Thoughts?
>>
>> P.
>>
>>
>> _______________________________________________
>> postgis-devel mailing list
>> postgis-devel at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/postgis-devel
>
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/postgis-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20171002/c453ccd1/attachment.html>