[pdal] Proposal: Remove SQLite reader/writer for PDAL 2.0

[Jed Frechette]

On Thu, Aug 1, 2019 at 2:06 PM Norman Barker <norman.barker at gmail.com> wrote:
> Without meaning to hijack this thread, which toolchain (e.g. mingw64) are you using when you build from source and are not using conda?

Not being able to control the toolchain was the problem I was hoping
to bypass. Ideally, by having something we can use in environments
where we have Python, but aren't guaranteed to have much else. To give
concrete examples, two of the applications we wanted to extend
initially were Agisoft and Houdini.

Agisoft includes a Python interpreter that sys.version reports as
"3.5.2 (default, Aug 28 2018, 15:41:10) [MSC v.1600 64 bit (AMD64)]".
The embedded Python doesn't ship with Numpy or much else in the way of
useful modules, and other than Python there is no other real extension
mechanism for the application.

Houdini also ships with Python, "2.7.15 (default, Apr  8 2019,
15:38:59) [MSC v.1916 64 bit (AMD64)]" in that case. Despite still
being on 2.7 (for ecosystem reasons) Houdini is a lot better. It does
at least include Numpy and also offers a bunch of other ways to extend
the application including a C++ SDK and even their own method of
making C++ functions accessible from Python [1].

In both cases, I know it is possible to add external Python modules to
the embedded distributions, but the process seems very brittle and
unsustainable, especially when you start getting in to modules that
mix Python and C++ e.g. [2].

I don't know that this is necessarily a problem that PDAL needs to
solve. Although having a portable format that could easily be handled
by pure Python would be convenient, there are certainly downsides to
working within that restriction. If the current SQLite implementation
isn't a good solution, I'm not necessarily arguing to keep it. More
than anything I'm just glad this thread happened the week before we
started working on it. ;-)

TileDB does seem like a good working format for situations where we
have more control over the toolchain.

Best,

[1] https://www.sidefx.com/docs/houdini/hom/extendingwithcpp.html
[2] https://www.sidefx.com/forum/topic/58442/

-- 
Jed Frechette