[QGIS-Developer] Processing: questions on temporary files

[Richard Duivenvoorde]

On 05/12/2019 01.01, Nyall Dawson wrote:
> What's needed/missing is a method for the memory data provider to
> detect when a layer is growing too large, and start writing this out
> to disk. (Important note: we CAN'T use any of the standard spatial
> formats for this (as they are lossy when compared to the memory
> provider support), so we'd need to directly serialize the memory layer
> features to a custom binary format, with indexing support for rapid
> retrieval of features.

If I understand correct, FME is also working with (indexed) temp files?
I'm not an FME user, so I can bu wrong, but I always wondered what kind
of format that is...

IF we (as FOSS community) could come up with some (new?) indexed file
format, it (besides being a in-between product for transformations) also
a potential data exchange format (maybe some zipped version of file+index?)?

Anybody into this?

Regards,

Richard Duivenvoorde