[GeoNode-users] Migration from 2.4

Fri Oct 19 11:33:40 PDT 2018

It would definitely make sense to have it as part of the updatelayers
command, though that is slower when only the sync of permissions is
needed.
In our case, with a very large number of layers to sync, it is
definitely much faster
Paolo
On Fri, Oct 19, 2018 at 2:04 PM Toni Schönbuchner
<toni.schoenbuchner at csgis.de> wrote:
>
> Thanks Paolo.
>
> But with World map the title ‚master of migration‘
> Is yours :D
>
> > remember, therefore you may need to run the sync_geofence command.
>
>
> A small note: I darkly remember that this command will be removed.
> (If I’m not wrong said at some GitHub issue by Alessio).
>
> Cheers,
>
> Toni
>
>
> > Am 19.10.2018 um 19:56 schrieb Paolo Corti <pcorti at gmail.com>:
> >
> > Great instructions, thanks to both of you.
> > One thing to add: I think 2.4 didn't use GeoFence as far as I can
> > remember, therefore you may need to run the sync_geofence command.
> > Also updatelayers should sync the permissions in GeoFence database
> > tough.
> > best
> > Paolo
> > On Thu, Oct 18, 2018 at 4:23 AM Toni Schönbuchner
> > <toni.schoenbuchner at csgis.de> wrote:
> >>
> >> A small addition for the record. I more or less followed Olivers path for migrations.
> >> But needed to alter layers as well like:
> >>
> >> sudo -u postgres psql -d geonode_new -c 'UPDATE base_resourcebase SET is_approved = TRUE'
> >>
> >> # might be specific to my instance
> >> sudo -u postgres psql -d geonode_new -c "DELETE FROM maps_maplayer WHERE maps_maplayer.name = 'AerialWithLabels'"
> >>
> >> Cheers,
> >>
> >> Toni
> >>
> >> Am 18.10.2018 um 08:45 schrieb Olivier Dalang <olivier.dalang at gmail.com>:
> >>
> >> Hi Toni,
> >>
> >> Thanks for the link, I somehow missed your issue when searching... I had the same issue than you at layers.0029_layer_service
> >>
> >> For reference, here's the procedure I used to successfully complete the migrations : https://github.com/olivierdalang/SPCgeonode#how-to-migrate-from-an-existing-standard-geonode-install (specific to SPCgeonode but can be transposed).
> >>
> >> Cheers,
> >>
> >> Olivier
> >>
> >>
> >>
> >> On Thu, Oct 18, 2018 at 6:26 PM Toni Schönbuchner <toni.schoenbuchner at csgis.de> wrote:
> >>>
> >>> Hi Olivier,
> >>>
> >>> I´ve migrated 2.4 -> 2.8 some time ago and found problems with db migrations as well.
> >>> See Issue: https://github.com/GeoNode/geonode/issues/3870
> >>> Did you go to 2.10? And can you share your errortrace somewhere?
> >>>
> >>>
> >>> Chees,
> >>>
> >>> Toni
> >>>
> >>> Today's Topics:
> >>>
> >>>  1. Migration from 2.4 (Olivier Dalang)
> >>>
> >>>
> >>> ----------------------------------------------------------------------
> >>>
> >>> Message: 1
> >>> Date: Thu, 18 Oct 2018 13:34:29 +1200
> >>> From: Olivier Dalang <olivier.dalang at gmail.com>
> >>> To: geonode-users <geonode-users at lists.osgeo.org>
> >>> Subject: [GeoNode-users] Migration from 2.4
> >>> Message-ID:
> >>> <CAExk7p3RqhCDCg2B6cz0ZMSQigFuv5WxrsEkjZjFCGwULdUwpA at mail.gmail.com>
> >>> Content-Type: text/plain; charset="utf-8"
> >>>
> >>> Hi !
> >>>
> >>> I'm currently trying to migrate an old geonode install (2.4) to the current
> >>> version.
> >>> I managed to get the migrations to run after some tweaks (details below),
> >>> but the database is not consistent with the models.
> >>>
> >>> Is this expected ? Or is there something wrong with the previous install ?
> >>> (it's old and I have no details about how it was installed). I searched a
> >>> bit about migrating old geonodes, but didn't find any details past
> >>> migrating with --fake-initial.
> >>>
> >>> I can fix the schema with sqldiff (from django-extensions) but would like
> >>> to know if this is supposed to happen or not.
> >>>
> >>> Thanks !!
> >>>
> >>> Olivier
> >>>
> >>>
> >>>
> >>> Here are the steps I did :
> >>>
> >>> 1) Run migrations with --fake-initial. This fails at :
> >>>
> >>> Applying layers.0029_layer_service...
> >>> ...
> >>> django.db.utils.ProgrammingError: column "service_id" of relation
> >>> "layers_layer" already exists
> >>>
> >>> 2) Manually fix this :
> >>> ALTER TABLE public.layers_layer DROP COLUMN service_id;
> >>>
> >>> 3) Run the migrations (they complete)
> >>>
> >>> 4) Check the consistency with python manage.py sqldiff -ae (from
> >>> django-extensions). There are inconsistencies, some of which will cause
> >>> problems (NOT NULL constraints, missing columns, etc.). Using this, I can
> >>> fix the schema (need to drop tables with no models).
> >>>
> >>> BEGIN;
> >>> -- Model missing for table: account_accountdeletion
> >>> -- Model missing for table: account_account
> >>> -- Model missing for table: account_signupcoderesult
> >>> -- Model missing for table: account_signupcode
> >>> -- Model missing for table: celery_taskmeta
> >>> -- Model missing for table: celery_tasksetmeta
> >>> -- Model missing for table: djcelery_intervalschedule
> >>> -- Model missing for table: djcelery_crontabschedule
> >>> -- Model missing for table: djcelery_periodictasks
> >>> -- Model missing for table: djcelery_workerstate
> >>> -- Model missing for table: account_signupcodeextended
> >>> -- Model missing for table: djcelery_periodictask
> >>> -- Model missing for table: djcelery_taskstate
> >>> -- Application: actstream
> >>> -- Model: Follow
> >>> CREATE INDEX "actstream_follow_object_id"
> >>> ON "actstream_follow" ("object_id");
> >>> CREATE INDEX "actstream_follow_object_id_like"
> >>> ON "actstream_follow" ("object_id" varchar_pattern_ops);
> >>> CREATE INDEX "actstream_follow_started"
> >>> ON "actstream_follow" ("started");
> >>> -- Model: Action
> >>> CREATE INDEX "actstream_action_actor_object_id"
> >>> ON "actstream_action" ("actor_object_id");
> >>> CREATE INDEX "actstream_action_actor_object_id_like"
> >>> ON "actstream_action" ("actor_object_id" varchar_pattern_ops);
> >>> CREATE INDEX "actstream_action_verb"
> >>> ON "actstream_action" ("verb");
> >>> CREATE INDEX "actstream_action_verb_like"
> >>> ON "actstream_action" ("verb" varchar_pattern_ops);
> >>> CREATE INDEX "actstream_action_target_object_id"
> >>> ON "actstream_action" ("target_object_id");
> >>> CREATE INDEX "actstream_action_target_object_id_like"
> >>> ON "actstream_action" ("target_object_id" varchar_pattern_ops);
> >>> CREATE INDEX "actstream_action_action_object_object_id"
> >>> ON "actstream_action" ("action_object_object_id");
> >>> CREATE INDEX "actstream_action_action_object_object_id_like"
> >>> ON "actstream_action" ("action_object_object_id" varchar_pattern_ops);
> >>> CREATE INDEX "actstream_action_timestamp"
> >>> ON "actstream_action" ("timestamp");
> >>> CREATE INDEX "actstream_action_public"
> >>> ON "actstream_action" ("public");
> >>> ALTER TABLE "actstream_action"
> >>> ALTER "data" TYPE jsonb;
> >>> -- Application: tastypie
> >>> -- Model: ApiKey
> >>> ALTER TABLE "tastypie_apikey"
> >>> ALTER "key" TYPE varchar(128);
> >>> -- Application: base
> >>> -- Model: ContactRole
> >>> ALTER TABLE "base_contactrole"
> >>> ALTER COLUMN "resource_id" DROP NOT NULL;
> >>> -- Model: ResourceBase
> >>> ALTER TABLE "base_resourcebase"
> >>> DROP COLUMN "distribution_url";
> >>> ALTER TABLE "base_resourcebase"
> >>> DROP COLUMN "distribution_description";
> >>> -- Model: Link
> >>> ALTER TABLE "base_link"
> >>> ALTER COLUMN "resource_id" DROP NOT NULL;
> >>> -- Application: layers
> >>> -- Model: Layer
> >>> ALTER TABLE "layers_layer"
> >>> DROP COLUMN "distribution_description_en";
> >>> -- Application: maps
> >>> -- Model: Map
> >>> ALTER TABLE "maps_map"
> >>> DROP COLUMN "distribution_description_en";
> >>> -- Application: documents
> >>> -- Model: Document
> >>> ALTER TABLE "documents_document"
> >>> DROP COLUMN "distribution_description_en";
> >>> ALTER TABLE "documents_document"
> >>> ALTER "doc_file" TYPE varchar(255);
> >>> ALTER TABLE "documents_document"
> >>> ALTER "doc_url" TYPE varchar(255);
> >>> -- Application: people
> >>> -- Model: Profile
> >>> ALTER TABLE "people_profile"
> >>> ALTER "email" TYPE varchar(254);
> >>> ALTER TABLE "people_profile"
> >>> ALTER COLUMN "last_login" DROP NOT NULL;
> >>> -- Application: groups
> >>> -- Model: GroupProfile
> >>> ALTER TABLE "groups_groupprofile"
> >>> ALTER "email" TYPE varchar(254);
> >>> -- Application: upload
> >>> -- Model: Upload
> >>> ALTER TABLE "upload_upload"
> >>> ADD COLUMN "mosaic_time_value" varchar(128);
> >>> ALTER TABLE "upload_upload"
> >>> ADD COLUMN "mosaic_time_regex" varchar(128);
> >>> ALTER TABLE "upload_upload"
> >>> ADD COLUMN "mosaic_elev_regex" varchar(128);
> >>> ALTER TABLE "upload_upload"
> >>> ADD COLUMN "mosaic_elev_value" varchar(128);
> >>> COMMIT;
> >>> -------------- next part --------------
> >>> An HTML attachment was scrubbed...
> >>> URL: <http://lists.osgeo.org/pipermail/geonode-users/attachments/20181018/e836c8ca/attachment.html>
> >>>
> >>> ------------------------------
> >>>
> >>> Subject: Digest Footer
> >>>
> >>> _______________________________________________
> >>> geonode-users mailing list
> >>> geonode-users at lists.osgeo.org
> >>> https://lists.osgeo.org/mailman/listinfo/geonode-users
> >>>
> >>>
> >>> ------------------------------
> >>>
> >>> End of geonode-users Digest, Vol 45, Issue 69
> >>> *********************************************
> >>>
> >>>
> >>
> >> _______________________________________________
> >> geonode-users mailing list
> >> geonode-users at lists.osgeo.org
> >> https://lists.osgeo.org/mailman/listinfo/geonode-users
> >
> >
> >
> > --
> > Paolo Corti
> > Geospatial software developer
> > web: http://www.paolocorti.net
> > twitter: @capooti
> > skype: capooti
> >
>


-- 
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti