[GeoNode-users] migration suggestions

Sat Mar 21 09:44:33 PDT 2020

Dear Eugenio,

> I don't know what is the quickest. If you say that migration will keep me more time than the insert I proceed with the insert one by one.
> For now I thought it was better the migration.

I would say this depends on your maps. As you can upload several layers in one shot further use importlayers
management command to import for example all shapes of a folder I tend to say: You might be faster by just rebuilding
your data.

> Regarding Multisites in the past I chose this option because I thought it was the best solution to manage different project that each one needs a different frontpage and provide specific layers and maps.
> What do you could recommend me instead of multi-sites?

I've never tried multisites. But from what I know it's currently not maintained.
Well I think I would fire up several docker stacks and bind them to specific domains.

> Regarding the command 'migrate' I run the command you suggest me, it finished without errors in the command line, but I don't know how and if the table in the db have been affected. Could you give me an example of a possible difference between the 2.4 and the 2.10 schema, so I can check it out?

The database is actually build by Django`s model definition. For example have a look
at the monitoring model: 
https://github.com/GeoNode/geonode/blob/master/geonode/monitoring/models.py#L125-L137 <https://github.com/GeoNode/geonode/blob/master/geonode/monitoring/models.py#L125-L137>

This did not exist in 2.4 means you should not find tables for monitoring_ in a 2.4 database.
In a 2.10 database there should exist monitoring_ tables and most likely one called something
with _service having the fields as marked by above link. 

Long story short. The models should allow you to spot check your migration.

> Eventually I didn't understand what you mean with 
> "
> Further yes, a wrong settings file could lead to a wrong database usage.
> (Just rename the old and you will see ;) "

Sorry for being unclear. I just confirmed your guess. As settings.py and local_settings.py define
database connection it could lead to strange behaviour when using the wrong file.

> 
> Actually I copied local_settings.py.geoserver.sample into the same folder with the name local_settings.py and I tried to modify there ALLOWED_HOSTS without result (I pout it on settings.py to have geonode running without error) and db properties.
> What I see is that in the old instance I had even local_settins.pyc while in the new it doesn't exist. In the old version local_settings.py is owned by eugenio:root while in the new by eugenio:eugenio. Moroover in the old local_settings.pyc is owned by eugenio:www-data.
> Could the problem be related to the grants on these files?

This depends on which user runs the apache process (I think you are using apache right)?
But most likely not. More likely there is some confusing with your used settings files.
In case you have followed the installation guide on docs.geonode.org <http://docs.geonode.org/> you would use nginx
and uwsgi. Uwsgi set s all configuration through environment variables. I would strongly suggest
to follow this setup guide ;)

Cheers,

Toni

> 
> Cheers,
> 
> Eugenio
> 
> Da: Toni Schönbuchner <toni.schoenbuchner at csgis.de>
> Inviato: sabato 21 marzo 2020 14:30
> A: Eugenio Trumpy <frippe12573 at hotmail.com>
> Cc: geonode-users <geonode-users at lists.osgeo.org>
> Oggetto: Re: migration suggestions
>  
> Dear Eugenio,
> 
> first of all, please be prepared that a migration is not a walk in the park.
> You might need some background knowledge about the db scheme and how 
> is everything is wired together. Further I think the whole multisites idea is not
> maintained since long time and hence I would suggest that you first try to port your
> data to a single site 2.10.3 instance. 
> 
> You're saying them amount of tables changed after running migrate? That
> is a good sign. The most interesting here is if the manage command threw errors?
> 
> Further yes, a wrong settings file could lead to a wrong database usage.
> (Just rename the old and you will see ;) 
> 
> In case of geonode_project you can try to set local_settings here.
> 
> https://github.com/GeoNode/geonode-project/blob/2.10.3/project_name/wsgi.py#L38 <https://github.com/GeoNode/geonode-project/blob/2.10.3/project_name/wsgi.py#L38>
> 
> 
> Best regards
> 
> Toni
> 
>> Am 21.03.2020 um 14:20 schrieb Eugenio Trumpy <frippe12573 at hotmail.com <mailto:frippe12573 at hotmail.com>>:
>> 
>> Dear Toni,
>> 
>> thank you to have answered to mail last email.
>> My situation is the following:
>> 
>> I have a geonode 2.4 running on a server 'A' configured to run as multi-site which embeds something like 120 layers, 20 maps and 3 users. In server 'A' is running ubuntu 14.04, which is obsolete.
>> I have also server 'B' where ubuntu is at the version 18.04.
>> I want to migrate my genode 2.4 running instance as multisites from server 'A' to 'B'.
>> I have asked some weeks ago here in the list what would have been the best solution, consider I'm not a developer and I never used the virtualenv. I installed the 2.4 version for production.
>> For the migration I was following some suggestions from Alessio: 
>> Setup a new GeoServer 2.15.2 ( https://build.geo-solutions.it/geonode/geoserver/latest/geoserver-2.15.2.war <https://build.geo-solutions.it/geonode/geoserver/latest/geoserver-2.15.2.war> ) 
>> Transfer the Data Dir from the old GeoServer to the new one (I strongly suggest to backport **only** the workspace, datasets and styles, not the security subsystem)
>> Setup the new GeoNode instance
>> Run the "updatelayers" management command (see http://docs.geonode.org/en/master/admin/mgmt_commands/index.html#management-command-updatelayers <http://docs.geonode.org/en/master/admin/mgmt_commands/index.html#management-command-updatelayers> )
>> Update manually the missing permissions and metadata
>> I completed step 1 with geoserver 2.16.2 - I installed the data_dir outside geoserver - and it runs fine
>> I completed step 2 and I'm able to see all the layers I had in server 'A'
>> I set up a fresh geonode instance, even thanks to you help. I installed geonode from GIT, so I should have the master version (2.10).
>> Within the step 2 I restored in server 'A' the two postgresql db I had inserver 'A'
>> Since I remembered some steps of the custom configuration I did for version 2.4 (and I never used NGINX) I configured APACHE2.
>> Currently in server 'B' I'm able to see the geonode home page, but I cannot see my layers as well as I cannot enter as administrator.
>> I followed you suggestions and I 
>> ALTER TABLE public.layers_layer DROP COLUMN service_id
>> successfully
>> and I gave the command 
>> django python manage.py migrate --fake-initial
>> but I'm still not able to enter as administrator
>> I realized even that the db schema apparently didn't change (e.g. I had 75 tables in the old version and I still have 75 tables in the newer version, I don't if the fields are modified instead)
>> to have the version 2.10 running I followed your suggestion to insert my domain in the ALLOWED_HOSTS variable inside local_settings.py. I did it but it didn't work as long as I insert the same in settings.py. For this reason I'm supposing that local_settings.py is not active in some way and for the same reason it doesn't allow to connect to the db. Am I right? 
>> Once the administrator login will be solved: 
>> I should configure this new running geonode instance for multisites - Could I restore in someway the look & feels I had in the old instance?
>> check the authorization between geonode and geoserver
>> I hope this can help you to have a clearer vision of 'whats going on'...
>> 
>> Best regards,
>> 
>> Eugenio
>> 
>> Da: Toni Schönbuchner <toni.schoenbuchner at csgis.de <mailto:toni.schoenbuchner at csgis.de>>
>> Inviato: sabato 21 marzo 2020 13:45
>> A: Eugenio Trumpy <frippe12573 at hotmail.com <mailto:frippe12573 at hotmail.com>>
>> Cc: geonode-users <geonode-users at lists.osgeo.org <mailto:geonode-users at lists.osgeo.org>>
>> Oggetto: Re: migration suggestions
>>  
>> Hi Eugenio,
>> 
>> I would need to have a look what is going on. But yes local_settings should override settings.
>> The correct command for migrations is python manage.py migrate ... (instead of syncdb)
>> 
>> best regards
>> 
>> toni
>> 
>>> Am 20.03.2020 um 19:58 schrieb Eugenio Trumpy <frippe12573 at hotmail.com <mailto:frippe12573 at hotmail.com>>:
>>> 
>>> Dear Toni,
>>> 
>>> I followed your hints.
>>> I put the domain in the ALLOWED_HOSTS and now I visualise the home page.
>>> Just a question, I had to put the domain the settings.py file because it didn't work in local_setting.py, why? Don't local_setting override setting.py?
>>> Then I modify local_setting.py to set credential to access geonode db.
>>> Then I alter the table you suggested and I run python manage.py --syncdb but the number of tables didn't change, is that correct?
>>> The geonode home page is visualised but I'm not able to enter with the old administrator credential.
>>> I presume there is no connection with the db. Should I use again setting.py instead of local_setting.py?
>>> 
>>> Thanks in advance,
>>> 
>>> Eugenio
>>> 
>>> Da: Toni Schönbuchner <toni.schoenbuchner at csgis.de <mailto:toni.schoenbuchner at csgis.de>>
>>> Inviato: giovedì 19 marzo 2020 20:37
>>> A: Eugenio Trumpy <frippe12573 at hotmail.com <mailto:frippe12573 at hotmail.com>>
>>> Cc: geonode-users <geonode-users at lists.osgeo.org <mailto:geonode-users at lists.osgeo.org>>
>>> Oggetto: Re: migration suggestions
>>>  
>>> Dear Eugenio,
>>> 
>>> 
>>> >        • I set the wgsi module in apache2. The geonode is served now with the 'allowed_hosts' error. I think the issue could be related to the settings. In the 2.4 version I remember I set SITE_URL, should I set something similar even in this version?
>>> 
>>> Allowed hosts is a security mechanism by Django  to define Domains that serve your instance.
>>> https://docs.djangoproject.com/en/3.0/ref/settings/#allowed-hosts <https://docs.djangoproject.com/en/3.0/ref/settings/#allowed-hosts>
>>> 
>>> To overcome an 504 error you can add something like
>>> 
>>> ALLOWED_HOSTS = ['localhost' '127.0.0.1' 'your domain.com <http://domain.com/>']
>>> 
>>> to your settings.py or local_settings.p However, we all more and more avoid touching the settings file and try to configure everything
>>> with environment variables. So with a manual setup you can try to add it to your uwsgi config:
>>> https://docs.geonode.org/en/master/install/core/index.html#serving-geonode-geoserver-via-nginx <https://docs.geonode.org/en/master/install/core/index.html#serving-geonode-geoserver-via-nginx>
>>> 
>>> >        • I created a geonode postgresql empty db, while I restored the backup from that I was using in 2.4 version. How I should populate with the new data-schema the geonode metadata db? I remember there was the command python manage.py --syncdb, does still exist? I tried but I got error.
>>> 
>>> I'm unsure If I correctly understand. But the rough flow looks like:
>>> 
>>>         • Export your geonode 2.4 Database
>>>         • Import it to a fresh Database
>>>         • Be sure to correctly set permissions for your User
>>>         • Change your settings.py for using the new database and in case the correct user
>>>         • Fix some inconsistency within your new database. Run f.e. with pgadmin or by psql from console
>>>                 • ALTER TABLE public.layers_layer DROP COLUMN service_id
>>>         • Run the migrations, this should update your database scheme from 2.4 to 2.10
>>>                 • DJANGO_SETTINGS_MODULE=YOURGEONODE.local_settings django python manage.py migrate --fake-initial
>>> 
>>> You will need to change those commands to fit your setup further run into errors here and
>>> there but hopefully this steps help you to find out of the forest.
>>> 
>>> Cheers,
>>> 
>>> Toni
>>> 
>>> 
>>> -----------------------------------------------
>>> CSGIS
>>> 
>>> -----------------------------------------------
>>> Spinnereistraße 7
>>> Halle 18 04179 Leipzig
>>> -----------------------------------------------
>>> Web             https://csgis.de <https://csgis.de/>
>>> -----------------------------------------------
>>> 
>>> Hinweis gemäß § 33 BDSG
>>> Daten der Verfahrensbeteiligten werden gespeichert. Dieses Dokument ist ausschließlich für den 
>>> Adressaten bestimmt. Der Inhalt der E-Mail ist vertraulich. Falls Sie diese E-Mail versehentlich 
>>> erhalten haben, rufen Sie uns unter obiger Rufnummer umgehend an und löschen Sie diese Nachricht 
>>> von Ihrem Computer. Jegliche Art von Reproduktionen, Verbreitung, Vervielfältigung, Veränderung, 
>>> Verteilung und/oder Veröffentlichung dieser E-Mail ist verboten.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-users/attachments/20200321/356acfd6/attachment-0001.html>