<div dir="ltr">Alesio, thanks for this work, looks very exciting.<div><br></div><div>Very excited to see the GeoServer plugin side afterwards.</div><div><br></div><div>One question, what's the strategy for migrating between different versions of GeoNode? In particular between GeoNode forks and between future (and db incompatible) GeoNode versions.</div><div><br></div><div>One of the main uses for these scripts would be to help users backup their GeoNode before attempting an upgrade, either between major releases or between heavily modified dev versions.</div><div><br></div><div>-a</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Feb 26, 2016 at 7:07 AM, Alessio Fabiani <span dir="ltr"><<a href="mailto:alessio.fabiani@geo-solutions.it" target="_blank">alessio.fabiani@geo-solutions.it</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dear all,<div>as a follow up on this you can find here</div><div><br></div><div><a href="https://github.com/GeoNode/geonode/commit/cf1c7e7c7a90239f5171a6f83608d4d40360c262" target="_blank">https://github.com/GeoNode/geonode/commit/cf1c7e7c7a90239f5171a6f83608d4d40360c262</a><br></div><div><br></div><div>The GeoNode backup and restore utilities converted into GeoNode Base Management Commands.</div><div><br></div><div>They can be invoked now as:</div><div><br></div><div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><font face="monospace, monospace"><b>python manage.py backup --backup-dir=/tmp/<br></b></font></div><div><font face="monospace, monospace"><b><br></b></font></div><div><font face="monospace, monospace"><b>python manage.py restore --backup-file=/tmp/2016-02-26_054445.zip</b></font><br></div><div><br></div><div><br></div></blockquote></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><span class=""><br clear="all"><div><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div style="font-size:12.8px">Best Regards,</div><div style="font-size:12.8px">Alessio Fabiani.</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>==</div><div>GeoServer Professional Services from the experts!</div><div>Visit <a href="http://goo.gl/it488V" target="_blank">http://goo.gl/it488V</a> for more information.</div><div>==</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Ing. Alessio Fabiani</div><div style="font-size:12.8px">@alfa7691</div><div style="font-size:12.8px">Founder/Technical Lead</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>GeoSolutions S.A.S.</div><div>Via di Montramito 3/A</div><div>55054 Massarosa (LU)</div><div>Italy</div><div>phone: <a href="tel:%2B39%200584%20962313" value="+390584962313" target="_blank">+39 0584 962313</a></div><div>fax: <a href="tel:%2B39%200584%201660272" value="+3905841660272" target="_blank">+39 0584 1660272</a></div><div>mob: <a href="tel:%2B39%20331%206233686" value="+393316233686" target="_blank">+39 331 6233686</a></div><div><br></div><div><a href="http://www.geo-solutions.it" target="_blank">http://www.geo-solutions.it</a></div><div><a href="http://twitter.com/geosolutions_it" target="_blank">http://twitter.com/geosolutions_it</a></div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">-------------------------------------------------------</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><p><span lang="IT"><font size="1"><b>AVVERTENZE AI SENSI DEL D.Lgs. 196/2003</b></font></span></p><p><span lang="IT"><font size="1">Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.</font></span></p><p><span lang="IT"><font size="1"> </font></span></p><p><font size="1">The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.</font></p><p><font size="1">---------------------------------------------------------------------</font></p></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
<br></span><div><div class="h5"><div class="gmail_quote">On Thu, Feb 25, 2016 at 4:22 PM, Alessio Fabiani <span dir="ltr"><<a href="mailto:alessio.fabiani@geo-solutions.it" target="_blank">alessio.fabiani@geo-solutions.it</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Dear all,<div><br></div><div>please find here </div><div><br></div><div><b><a href="https://github.com/GeoNode/geonode/tree/backup_restore/scripts/backup-restore" target="_blank">https://github.com/GeoNode/geonode/tree/backup_restore/scripts/backup-restore</a></b><br></div><div><br></div><div>a first implementation of the backup and restore scripts using JSON fixtures.</div><div><br></div><div>The scripts allow to automatically store into a ZIP archive all the GeoNode Objects as JSON files along with the static resources.</div><div><br></div><div>A restore command takes the zip file as input and restores everything. At the end you will have an exact copy of the old GeoNode replicated into the new instance independently from the datastore.</div><div><br></div><div><b>NOTICE 1:</b> I'm still working on the GeoServer side. GeoServer will be enriched with a plugin allowing an administrator to store on a zip file the whole data and configuration and having the ability to restore it at any time.</div><div><br></div><div><b>NOTICE 2: </b>In order to make things working you'll need to apply also the "patch" attached to the email to the GeoNode source code <b>OR</b> clone the <b>backup_restore </b>repository</div><div><br></div><div>How the GeoNode scripts work:</div><div><br></div><div>1. Backup (on source instance)</div><div>----------------------------------------------</div><div><br></div><div>Just go to the geonode/scripts/backup-restore folder and run the command</div><div><br></div><div> <font face="monospace, monospace"><b>python backup.py</b></font></div><div><br></div><div>The command will output few lines on the console</div><div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><font face="monospace, monospace" size="1">geonode@geonode:~/geonode/scripts/backup-restore$ python backup.py</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'people' into 'people.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'account' into 'accounts.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'avatar.avatar' into 'avatars.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.license' into 'licenses.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.topiccategory' into 'topiccategories.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.region' into 'regions.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.resourcebase' into 'resourcebases.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.contactrole' into 'contactroles.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.link' into 'links.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.restrictioncodetype' into 'restrictioncodetypes.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'base.spatialrepresentationtype' into 'spatialrepresentationtypes.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'guardian.userobjectpermission' into 'useropermissions.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'guardian.groupobjectpermission' into 'groupopermissions.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'layers.uploadsession' into 'uploadsessions.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'layers.layer' into 'layers.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'layers.attribute' into 'attributes.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'maps.map' into 'maps.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'maps.maplayer' into 'maplayers.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'documents.document' into 'documents.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Dumping 'taggit' into 'tags.json'.</font></div></div><div><div><font face="monospace, monospace" size="1">Saved Media Files from '/home/geonode/geonode/geonode/uploaded'.</font></div></div><div><div><font face="monospace, monospace" size="1">Saved Static Root from '/home/geonode/geonode/geonode/static_root'.</font></div></div><div><div><font face="monospace, monospace" size="1">Saved Static Files from '/home/geonode/geonode/geonode/static'.</font></div></div><div><div><font face="monospace, monospace" size="1">Saved Template Files from '/home/geonode/geonode/geonode/templates'.</font></div></div><div><div><font face="monospace, monospace" size="1">Saved Locale Files from '/home/geonode/geonode/geonode/locale'.</font></div></div><div><div><font face="monospace, monospace" size="1">Backup Finished. Archive generated 'backup/<b>2016-02-25_180915.zip</b>'.</font></div></div><div><div><font face="monospace, monospace" size="1">None</font></div></div><div><div><font face="monospace, monospace" size="1">geonode@geonode:~/geonode/scripts/backup-restore$</font></div></div></blockquote><div><br></div><div>and will store a timestamped ZIP file into the "backup" subfolder.</div><div><br></div><div>Notice that the scripts gets the locations and settings automatically from the "geonode.settings" file.</div><div><br></div><div><div>2. Restore (on destination instance)</div><div>----------------------------------------------------</div></div><div><br></div><div><b>WARNING: </b><i>This procedure will completely purge and overwrite your destination GeoNode instance. <b>BE CAREFUL</b>.</i></div><div><br></div><div>Go to the geonode/scripts/backup-restore folder and edit the <b>settings.ini</b> file, providing the connection parameter of the <b>TARGET DB</b>. </div><div><br></div><blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px"><div><div><font face="monospace, monospace">[targetdb]</font></div></div><div><div><font face="monospace, monospace">dbname = geonode_rst</font></div></div><div><div><font face="monospace, monospace">host = localhost</font></div></div><div><div><font face="monospace, monospace">port = 5432</font></div></div><div><div><font face="monospace, monospace">user = geonode</font></div></div><div><div><font face="monospace, monospace">passwd = geonode</font></div></div></blockquote><div><br></div><div>This is necessary in case you have a PostgreSQL Backend. The Restore procedures tries to apply some patches to the DB in order to make things working. </div><div><br></div><div>If you don't have a PostgreSQL backend, just ignore this passage.<br></div><div><br></div><div>Now run the Restore command providing the ZIP file to restore</div><div><br></div><div> <font face="monospace, monospace"><b>python restore.py backup/2016-02-25_180915.zip</b></font><br></div><div><br></div><div>The command will ask you confirmation to proceed.</div><div>By inserting <b>"y"</b> it will wipe out the target instance DB and folders and restore the old ones.</div><div><br></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div><font face="monospace, monospace" size="1">geonode@geonode:~/geonode/scripts/backup-restore$ python restore.py backup/2016-02-25_180915.zip</font></div></div><div><div><font face="monospace, monospace" size="1">WARNING: The restore will overwrite all your GeoNode data and files. Are you sure you want to proceed? [n]|y: y</font></div></div><div><div><font face="monospace, monospace" size="1">Creating tables ...</font></div></div><div><div><font face="monospace, monospace" size="1">Installing custom SQL ...</font></div></div><div><div><font face="monospace, monospace" size="1">Installing indexes ...</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/people.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 3 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/accounts.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 5 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/avatars.json</font></div></div><div><div><font face="monospace, monospace" size="1">WARNING: No valid fixture data found for 'avatars'.</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/licenses.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 7 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/topiccategories.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 19 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/regions.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 259 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/resourcebases.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 2 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/contactroles.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 4 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/links.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 27 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/restrictioncodetypes.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 8 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/spatialrepresentationtypes.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 6 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/useropermissions.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 18 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/groupopermissions.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 4 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/uploadsessions.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 3 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/layers.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 1 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/attributes.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 3 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/maps.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 1 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/maplayers.json</font></div></div><div><div><font face="monospace, monospace" size="1">Installed 5 object(s) from 1 fixture(s)</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/documents.json</font></div></div><div><div><font face="monospace, monospace" size="1">WARNING: No valid fixture data found for 'documents'.</font></div></div><div><div><font face="monospace, monospace" size="1">Deserializing restore/2016-02-25_180915/tags.json</font></div></div><div><div><font face="monospace, monospace" size="1">WARNING: No valid fixture data found for 'tags'.</font></div></div><div><div><font face="monospace, monospace" size="1">Media Files Restored into '/home/geonode/geonode/geonode/uploaded'.</font></div></div><div><div><font face="monospace, monospace" size="1">Static Root Restored into '/home/geonode/geonode/geonode/static_root'.</font></div></div><div><div><font face="monospace, monospace" size="1">Static Files Restored into '/home/geonode/geonode/geonode/static'.</font></div></div><div><div><font face="monospace, monospace" size="1">Template Files Restored into '/home/geonode/geonode/geonode/templates'.</font></div></div><div><div><font face="monospace, monospace" size="1">Locale Files Restored into '/home/geonode/geonode/geonode/locale'.</font></div></div><div><div><font face="monospace, monospace" size="1">Restore finished. Please find restored files and dumps into: 'restore/2016-02-25_180915'.</font></div></div><div><div><font face="monospace, monospace" size="1">geonode@geonode:~/geonode/scripts/backup-restore$</font></div></div></blockquote><div><br></div><div><br></div><div>That's all for now.</div><div>I'm moving forward with the Backup&Restore plugin for GeoServer and more improvements on the GeoNode backup/restore scripts.</div><div><br></div><div>Notice that having JSON fixtures it should be possible with the next improvements to:</div><div><br></div><div>1. Filter out only some features to restore. As an instance you may want to restore just a subset of the Layers and or Maps</div><div><br></div><div>2. Remap the Layers (or part of them) on another instance</div><div><br></div><div>For the moment if you want/need to do some experiments with this, just do a full backup and manually edit the layers.json file inside the ZIP archive.</div><div><br></div><div><br clear="all"><div><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><div style="font-size:12.8px">Best Regards,</div><div style="font-size:12.8px">Alessio Fabiani.</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>==</div><div>GeoServer Professional Services from the experts!</div><div>Visit <a href="http://goo.gl/it488V" target="_blank">http://goo.gl/it488V</a> for more information.</div><div>==</div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Ing. Alessio Fabiani</div><div style="font-size:12.8px">@alfa7691</div><div style="font-size:12.8px">Founder/Technical Lead</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><div>GeoSolutions S.A.S.</div><div>Via di Montramito 3/A</div><div>55054 Massarosa (LU)</div><div>Italy</div><div>phone: <a href="tel:%2B39%200584%20962313" value="+390584962313" target="_blank">+39 0584 962313</a></div><div>fax: <a href="tel:%2B39%200584%201660272" value="+3905841660272" target="_blank">+39 0584 1660272</a></div><div>mob: <a href="tel:%2B39%20331%206233686" value="+393316233686" target="_blank">+39 331 6233686</a></div><div><br></div><div><a href="http://www.geo-solutions.it" target="_blank">http://www.geo-solutions.it</a></div><div><a href="http://twitter.com/geosolutions_it" target="_blank">http://twitter.com/geosolutions_it</a></div></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">-------------------------------------------------------</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><p><span lang="IT"><font size="1"><b>AVVERTENZE AI SENSI DEL D.Lgs. 196/2003</b></font></span></p><p><span lang="IT"><font size="1">Le informazioni contenute in questo messaggio di posta elettronica e/o nel/i file/s allegato/i sono da considerarsi strettamente riservate. Il loro utilizzo è consentito esclusivamente al destinatario del messaggio, per le finalità indicate nel messaggio stesso. Qualora riceviate questo messaggio senza esserne il destinatario, Vi preghiamo cortesemente di darcene notizia via e-mail e di procedere alla distruzione del messaggio stesso, cancellandolo dal Vostro sistema. Conservare il messaggio stesso, divulgarlo anche in parte, distribuirlo ad altri soggetti, copiarlo, od utilizzarlo per finalità diverse, costituisce comportamento contrario ai principi dettati dal D.Lgs. 196/2003.</font></span></p><p><span lang="IT"><font size="1"> </font></span></p><p><font size="1">The information in this message and/or attachments, is intended solely for the attention and use of the named addressee(s) and may be confidential or proprietary in nature or covered by the provisions of privacy act (Legislative Decree June, 30 2003, no.196 - Italy's New Data Protection Code).Any use not in accord with its purpose, any disclosure, reproduction, copying, distribution, or either dissemination, either whole or partial, is strictly forbidden except previous formal approval of the named addressee(s). If you are not the intended recipient, please contact immediately the sender by telephone, fax or e-mail and delete the information in this message that has been received in error. The sender does not give any warranty or accept liability as the content, accuracy or completeness of sent messages and accepts no responsibility for changes made after they were sent or for other risks which arise as a result of e-mail transmission, viruses, etc.</font></p><p><font size="1">---------------------------------------------------------------------</font></p></div></div></div></div></div></div></div></div></div></div></div></div></div></div>
</div></div>
</blockquote></div><br></div></div></div>
<br>_______________________________________________<br>
geonode-users mailing list<br>
<a href="mailto:geonode-users@lists.osgeo.org">geonode-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/geonode-users" rel="noreferrer" target="_blank">http://lists.osgeo.org/mailman/listinfo/geonode-users</a><br>
<br></blockquote></div><br></div>