[GeoNode-devel] [GeoNode-users] GeoNode Backup and Restore: first draft

Simone Dalmasso simone.dalmasso at gmail.com
Fri Feb 26 05:07:45 PST 2016 

Great work!

2016-02-26 13:07 GMT+01:00 Alessio Fabiani <alessio.fabiani at geo-solutions.it
>:

> Dear all,
> as a follow up on this you can find here
>
>
> https://github.com/GeoNode/geonode/commit/cf1c7e7c7a90239f5171a6f83608d4d40360c262
>
> The GeoNode backup and restore utilities converted into GeoNode Base
> Management Commands.
>
> They can be invoked now as:
>
>
> *python manage.py backup --backup-dir=/tmp/*
>
> *python manage.py restore --backup-file=/tmp/2016-02-26_054445.zip*
>
>
>
>
>
>
>
>
> Best Regards,
> Alessio Fabiani.
>
> ==
> GeoServer Professional Services from the experts!
> Visit http://goo.gl/it488V for more information.
> ==
>
> Ing. Alessio Fabiani
> @alfa7691
> Founder/Technical Lead
>
> GeoSolutions S.A.S.
> Via di Montramito 3/A
> 55054  Massarosa (LU)
> Italy
> phone: +39 0584 962313
> fax:     +39 0584 1660272
> mob:   +39 331 6233686
>
> http://www.geo-solutions.it
> http://twitter.com/geosolutions_it
>
> -------------------------------------------------------
>
> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>
> Le informazioni contenute in questo messaggio di posta elettronica e/o
> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
> darcene notizia via e-mail e di procedere alla distruzione del messaggio
> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
> principi dettati dal D.Lgs. 196/2003.
>
>
>
> The information in this message and/or attachments, is intended solely for
> the attention and use of the named addressee(s) and may be confidential or
> proprietary in nature or covered by the provisions of privacy act
> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
> Code).Any use not in accord with its purpose, any disclosure, reproduction,
> copying, distribution, or either dissemination, either whole or partial, is
> strictly forbidden except previous formal approval of the named
> addressee(s). If you are not the intended recipient, please contact
> immediately the sender by telephone, fax or e-mail and delete the
> information in this message that has been received in error. The sender
> does not give any warranty or accept liability as the content, accuracy or
> completeness of sent messages and accepts no responsibility  for changes
> made after they were sent or for other risks which arise as a result of
> e-mail transmission, viruses, etc.
>
> ---------------------------------------------------------------------
>
> On Thu, Feb 25, 2016 at 4:22 PM, Alessio Fabiani <
> alessio.fabiani at geo-solutions.it> wrote:
>
>> Dear all,
>>
>> please find here
>>
>> *https://github.com/GeoNode/geonode/tree/backup_restore/scripts/backup-restore
>> <https://github.com/GeoNode/geonode/tree/backup_restore/scripts/backup-restore>*
>>
>> a first implementation of the backup and restore scripts using JSON
>> fixtures.
>>
>> The scripts allow to automatically store into a ZIP archive all the
>> GeoNode Objects as JSON files along with the static resources.
>>
>> A restore command takes the zip file as input and restores everything. At
>> the end you will have an exact copy of the old GeoNode replicated into the
>> new instance independently from the datastore.
>>
>> *NOTICE 1:* I'm still working on the GeoServer side. GeoServer will be
>> enriched with a plugin allowing an administrator to store on a zip file the
>> whole data and configuration and having the ability to restore it at any
>> time.
>>
>> *NOTICE 2: *In order to make things working you'll need to apply also
>> the "patch" attached to the email to the GeoNode source code *OR* clone
>> the *backup_restore *repository
>>
>> How the GeoNode scripts work:
>>
>> 1. Backup (on source instance)
>> ----------------------------------------------
>>
>> Just go to the geonode/scripts/backup-restore folder and run the command
>>
>>     *python backup.py*
>>
>> The command will output few lines on the console
>>
>> geonode at geonode:~/geonode/scripts/backup-restore$ python backup.py
>> Dumping 'people' into 'people.json'.
>> Dumping 'account' into 'accounts.json'.
>> Dumping 'avatar.avatar' into 'avatars.json'.
>> Dumping 'base.license' into 'licenses.json'.
>> Dumping 'base.topiccategory' into 'topiccategories.json'.
>> Dumping 'base.region' into 'regions.json'.
>> Dumping 'base.resourcebase' into 'resourcebases.json'.
>> Dumping 'base.contactrole' into 'contactroles.json'.
>> Dumping 'base.link' into 'links.json'.
>> Dumping 'base.restrictioncodetype' into 'restrictioncodetypes.json'.
>> Dumping 'base.spatialrepresentationtype' into
>> 'spatialrepresentationtypes.json'.
>> Dumping 'guardian.userobjectpermission' into 'useropermissions.json'.
>> Dumping 'guardian.groupobjectpermission' into 'groupopermissions.json'.
>> Dumping 'layers.uploadsession' into 'uploadsessions.json'.
>> Dumping 'layers.layer' into 'layers.json'.
>> Dumping 'layers.attribute' into 'attributes.json'.
>> Dumping 'maps.map' into 'maps.json'.
>> Dumping 'maps.maplayer' into 'maplayers.json'.
>> Dumping 'documents.document' into 'documents.json'.
>> Dumping 'taggit' into 'tags.json'.
>> Saved Media Files from '/home/geonode/geonode/geonode/uploaded'.
>> Saved Static Root from '/home/geonode/geonode/geonode/static_root'.
>> Saved Static Files from '/home/geonode/geonode/geonode/static'.
>> Saved Template Files from '/home/geonode/geonode/geonode/templates'.
>> Saved Locale Files from '/home/geonode/geonode/geonode/locale'.
>> Backup Finished. Archive generated 'backup/*2016-02-25_180915.zip*'.
>> None
>> geonode at geonode:~/geonode/scripts/backup-restore$
>>
>>
>> and will store a timestamped ZIP file into the "backup" subfolder.
>>
>> Notice that the scripts gets the locations and settings automatically
>> from the "geonode.settings" file.
>>
>> 2. Restore (on destination instance)
>> ----------------------------------------------------
>>
>> *WARNING: **This procedure will completely purge and overwrite your
>> destination GeoNode instance. BE CAREFUL.*
>>
>> Go to the geonode/scripts/backup-restore folder and edit the
>> *settings.ini* file, providing the connection parameter of the *TARGET
>> DB*.
>>
>> [targetdb]
>> dbname = geonode_rst
>> host   = localhost
>> port   = 5432
>> user   = geonode
>> passwd = geonode
>>
>>
>> This is necessary in case you have a PostgreSQL Backend. The Restore
>> procedures tries to apply some patches to the DB in order to make things
>> working.
>>
>> If you don't have a PostgreSQL backend, just ignore this passage.
>>
>> Now run the Restore command providing the ZIP file to restore
>>
>>    *python restore.py backup/2016-02-25_180915.zip*
>>
>> The command will ask you confirmation to proceed.
>> By inserting *"y"* it will wipe out the target instance DB and folders
>> and restore the old ones.
>>
>> geonode at geonode:~/geonode/scripts/backup-restore$ python restore.py
>> backup/2016-02-25_180915.zip
>> WARNING: The restore will overwrite all your GeoNode data and files. Are
>> you sure you want to proceed? [n]|y: y
>> Creating tables ...
>> Installing custom SQL ...
>> Installing indexes ...
>> Deserializing restore/2016-02-25_180915/people.json
>> Installed 3 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/accounts.json
>> Installed 5 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/avatars.json
>> WARNING: No valid fixture data found for 'avatars'.
>> Deserializing restore/2016-02-25_180915/licenses.json
>> Installed 7 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/topiccategories.json
>> Installed 19 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/regions.json
>> Installed 259 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/resourcebases.json
>> Installed 2 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/contactroles.json
>> Installed 4 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/links.json
>> Installed 27 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/restrictioncodetypes.json
>> Installed 8 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/spatialrepresentationtypes.json
>> Installed 6 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/useropermissions.json
>> Installed 18 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/groupopermissions.json
>> Installed 4 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/uploadsessions.json
>> Installed 3 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/layers.json
>> Installed 1 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/attributes.json
>> Installed 3 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/maps.json
>> Installed 1 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/maplayers.json
>> Installed 5 object(s) from 1 fixture(s)
>> Deserializing restore/2016-02-25_180915/documents.json
>> WARNING: No valid fixture data found for 'documents'.
>> Deserializing restore/2016-02-25_180915/tags.json
>> WARNING: No valid fixture data found for 'tags'.
>> Media Files Restored into '/home/geonode/geonode/geonode/uploaded'.
>> Static Root Restored into '/home/geonode/geonode/geonode/static_root'.
>> Static Files Restored into '/home/geonode/geonode/geonode/static'.
>> Template Files Restored into '/home/geonode/geonode/geonode/templates'.
>> Locale Files Restored into '/home/geonode/geonode/geonode/locale'.
>> Restore finished. Please find restored files and dumps into:
>> 'restore/2016-02-25_180915'.
>> geonode at geonode:~/geonode/scripts/backup-restore$
>>
>>
>>
>> That's all for now.
>> I'm moving forward with the Backup&Restore plugin for GeoServer and more
>> improvements on the GeoNode backup/restore scripts.
>>
>> Notice that having JSON fixtures it should be possible with the next
>> improvements to:
>>
>> 1. Filter out only some features to restore. As an instance you may want
>> to restore just a subset of the Layers and or Maps
>>
>> 2. Remap the Layers (or part of them) on another instance
>>
>> For the moment if you want/need to do some experiments with this, just do
>> a full backup and manually edit the layers.json file inside the ZIP archive.
>>
>>
>> Best Regards,
>> Alessio Fabiani.
>>
>> ==
>> GeoServer Professional Services from the experts!
>> Visit http://goo.gl/it488V for more information.
>> ==
>>
>> Ing. Alessio Fabiani
>> @alfa7691
>> Founder/Technical Lead
>>
>> GeoSolutions S.A.S.
>> Via di Montramito 3/A
>> 55054  Massarosa (LU)
>> Italy
>> phone: +39 0584 962313
>> fax:     +39 0584 1660272
>> mob:   +39 331 6233686
>>
>> http://www.geo-solutions.it
>> http://twitter.com/geosolutions_it
>>
>> -------------------------------------------------------
>>
>> *AVVERTENZE AI SENSI DEL D.Lgs. 196/2003*
>>
>> Le informazioni contenute in questo messaggio di posta elettronica e/o
>> nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il
>> loro utilizzo è consentito esclusivamente al destinatario del messaggio,
>> per le finalità indicate nel messaggio stesso. Qualora riceviate questo
>> messaggio senza esserne il destinatario, Vi preghiamo cortesemente di
>> darcene notizia via e-mail e di procedere alla distruzione del messaggio
>> stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso,
>> divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od
>> utilizzarlo per finalità diverse, costituisce comportamento contrario ai
>> principi dettati dal D.Lgs. 196/2003.
>>
>>
>>
>> The information in this message and/or attachments, is intended solely
>> for the attention and use of the named addressee(s) and may be confidential
>> or proprietary in nature or covered by the provisions of privacy act
>> (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection
>> Code).Any use not in accord with its purpose, any disclosure, reproduction,
>> copying, distribution, or either dissemination, either whole or partial, is
>> strictly forbidden except previous formal approval of the named
>> addressee(s). If you are not the intended recipient, please contact
>> immediately the sender by telephone, fax or e-mail and delete the
>> information in this message that has been received in error. The sender
>> does not give any warranty or accept liability as the content, accuracy or
>> completeness of sent messages and accepts no responsibility  for changes
>> made after they were sent or for other risks which arise as a result of
>> e-mail transmission, viruses, etc.
>>
>> ---------------------------------------------------------------------
>>
>
>
> _______________________________________________
> geonode-users mailing list
> geonode-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/geonode-users
>
>


-- 
Simone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-devel/attachments/20160226/ee2187e7/attachment-0001.html>

More information about the geonode-devel mailing list