[Qgis-developer] Memory data provider persistence

[Chris Crook]

Hi Martin and list readers

Interesting idea!  This would certainly achieve my main goal, which is to have the project reopen with the same information in as when I closed it.  

I agree that saving the the XML file could be cumbersome - even though these are only memory provider layers I guess they could get quite big.  And very good point about loading from the DOM.

Another way to achieve a similar result could be to embed the project file in the spatialite database?  That would make the memory provider data easier to use - basically it just becomes a spatialite provided layer.  The project itself becomes harder to manually edit, if you wanted to do so.  However it could easily be exported to a separate file if that is required.  And the whole thing could still be zipped if you wanted to.

I don't see these options as exclusive.  

Look forward to comments, 
Chris  

________________________________________
From: Martin Dobias [wonder.sk at gmail.com]
Sent: 03 December 2010 11:01
To: Chris Crook
Cc: qgis-developer at lists.osgeo.org
Subject: Re: [Qgis-developer] Memory data provider persistence

Hi Chris

On Mon, Nov 29, 2010 at 8:54 PM, Chris Crook <ccrook at linz.govt.nz> wrote:
> Hi All
>
> I'm looking for ideas/suggestions/comments on the memory data provider.
>
> A while ago I raised a ticket http://trac.osgeo.org/qgis/ticket/2487, the essence of which is that if you save and reload a project nothing should have changed, but if you have a layer supported by a memory data provider then its contents get lost.
>
> In the ticket I suggested that the user should be warned on saving, and the layers not included in the project file.  That way at least you don't end up with empty layers.
>
> I'm now coming to a different point of view, which is that it would make much more sense to save the data for the memory provider into the project file.

Having a possibility to save the data into a project would be a nice
feature. However I think that saving the features directly into the
XML project file wouldn't scale well with the amount of data. It can
work well for small datasets, though I'm afraid that it will fail with
a lot of data. One reason is that the project file would get pretty
big due the verbosity of XML, another problem would be with the memory
footprint and parsing speed - we're using DOM.

I have started to think about a more radical step: to allow the
projects to be not only XML files, but additionally they could be e.g.
ZIP files with project XML file inside and optionally some more files.
AFAIK OpenDocument format (used in OpenOffice) does this too.
Internally QGIS could ask the user and convert his data from memory
provider (or any other provider) to e.g. spatialite database that
would be included in that project ZIP file.

This would involve the following changes:
- implement reading and writing of compressed project files (using zlib)
- write a generic QgsVectorLayer -> SpatiaLite export class (similar
to QgsVectorFileWriter or an extension of it)
- add support to read/write "embedded" layers

This would also enable a nice feature to let QGIS pack all data into
one (compressed) file and pass just that file around!

What do you (and others) think of such solution?

Regards
Martin
______________________________________________________________________________________________________

This message contains information, which is confidential and may be subject to legal privilege. 
If you are not the intended recipient, you must not peruse, use, disseminate, distribute or copy this message.
If you have received this message in error, please notify us immediately (Phone 0800 665 463 or info at linz.govt.nz) and destroy the original message.
LINZ accepts no responsibility for changes to this email, or for any attachments, after its transmission from LINZ.

Thank you.
______________________________________________________________________________________________________