[GeoNode-users] failed migration from 2.0 to 2.4

Mon Apr 25 00:46:38 PDT 2016

Thank you. We checked, and it seems it is pointing in the right direction.

When running the migrate2.sh we observe that the underlying data for each
layer is not transferred to the new server with geonode 2.4 and the new
geoserver. No store is created in the new geoserver during the migration
process. Could that be the root cause of the migration issue? Should a
store be automatically created in the new geoserver during the migration,
or should we create one before running the migration?

thank you,

Levi

On Tue, Apr 19, 2016 at 4:33 PM, Paolo Corti <pcorti at gmail.com> wrote:

> Yes, it looks like you are failing to migrate the GeoServer data directory.
> Check if it is pointing to the correct location.
> Please keep the list in copy, other people may benefit from those
> answers in future
> p
>
> On Tue, Apr 19, 2016 at 9:33 AM, Levi Westerveld
> <levi.westerveld at gmail.com> wrote:
> > I attached a screenshot of the geoserver and the geonode. As you see, no
> > layers show up in the geoserver but they do in the geonode.. I hope this
> > helps!
> >
> > On Tue, Apr 19, 2016 at 3:28 PM, Levi Westerveld <
> levi.westerveld at gmail.com>
> > wrote:
> >>
> >> Thank you Paolo for your quick answer,
> >>
> >> In the thread you mentioned I saw you made this comment particular
> >> comment:
> >>
> >> please note that, as suggested in the updated instruction, now the
> >> migration process consists of the following steps:
> >>
> >> run migrate.py
> >> restore geoserver
> >> run migrate2.py
> >>
> >> it looks like you are on step 1? Therefore you don't see your layers, as
> >> they have been loaded in db with default permissions: in this case only
> if
> >> you are authenticated you will be able to see them.
> >>
> >> Basically, you need to login in GeoNode :)
> >>
> >> After running migrate2.py script, permissions will be correctly setup.
> >>
> >> Let me know if this works.
> >>
> >> By the way, I did only step 1 with your backup, I cannot process further
> >> because I would need the geoserver data directory to run step 2.
> >>
> >>
> >> Would you suggest us to first restore geoserver and then try running the
> >> second migration script again? It seems from reading the thread that the
> >> internal server error issue and the layers not migrating correctly
> issue are
> >> related.
> >>
> >> Thank you,
> >>
> >> Levi
> >>
> >>
> >> On Tue, Apr 19, 2016 at 3:03 PM, Paolo Corti <pcorti at gmail.com> wrote:
> >>>
> >>> Hi
> >>> this won't help, you will loose all the style, metadata, tags,
> >>> permissions doing so.
> >>>
> >>> Checking your logs, first script ran successfully.
> >>> You have a problem at the beginning of the second script: before
> >>> running it make sure that GeoServer is communicating correctly with
> >>> GeoNode: you should be able to see the layers in the GeoServer
> >>> preview, for example.
> >>> The fact that links are broken, this is expected, as updating layers
> >>> failed at the beginning of the second script.
> >>> You are exactly in the same problem as a couple of users here (they
> >>> managed to fix it): https://github.com/GeoNode/geonode/issues/2067
> >>>
> >>> Hope it helps
> >>> p
> >>>
> >>> On Tue, Apr 19, 2016 at 7:31 AM, Levi Westerveld
> >>> <levi.westerveld at gmail.com> wrote:
> >>> > As a follow up to this email.
> >>> >
> >>> > It appears that for the second error none of the GIS data was
> actually
> >>> > migrated but just a description of the layers was. We will now try to
> >>> > import
> >>> > the layers that did not migrate using the sudo geonode importalyers
> >>> > <dat_dir> to the new geonode and I just hope that the imported layers
> >>> > will
> >>> > link back with the descriptive data that was imported for each layer.
> >>> >
> >>> > The first issue remains unresolved..
> >>> >
> >>> > Thank you,
> >>> >
> >>> > levi
> >>> >
> >>> > On Tue, Apr 19, 2016 at 12:48 PM, Levi Westerveld
> >>> > <levi.westerveld at gmail.com> wrote:
> >>> >>
> >>> >> Dear Simone and Paolo,
> >>> >>
> >>> >> We have reinstalled ubuntu and ran the migration files successfully
> >>> >> again
> >>> >> from our geonode 2.0 server to our geonode 2.4 instance.
> >>> >>
> >>> >> We have two issues
> >>> >>
> >>> >> 1. we are able to access the geonode 2.4 if we are signed in as a
> user
> >>> >> at
> >>> >> http://opengeo.grida.no/ but if not signed in then an internal
> server
> >>> >> issues
> >>> >> shows up
> >>> >> 2. when we are in the layers section and click on a layer it seems
> >>> >> that
> >>> >> the link is broken, and we are not redirected to that layer, but
> stay
> >>> >> on the
> >>> >> same page
> >>> >>
> >>> >> I attached the two log files for the migration.
> >>> >>
> >>> >> Thank you!
> >>> >>
> >>> >> Levi
> >>> >>
> >>> >> On Fri, Apr 15, 2016 at 9:14 AM, Simone Dalmasso
> >>> >> <simone.dalmasso at gmail.com> wrote:
> >>> >>>
> >>> >>> Levi,
> >>> >>>
> >>> >>> that is definitely not a good error but can be solved. Linux puts
> the
> >>> >>> file system in read only mode when there are inconsistencies on the
> >>> >>> disk.I
> >>> >>> suggest you to google for that error, you will find tons of
> >>> >>> suggestions on
> >>> >>> how to check and fix the disk errors.
> >>> >>>
> >>> >>> 2016-04-14 22:36 GMT+02:00 Levi Westerveld
> >>> >>> <levi.westerveld at gmail.com>:
> >>> >>>>
> >>> >>>> Hello Paolo and Simone,
> >>> >>>>
> >>> >>>> We just ran the script again and we got a new error message in the
> >>> >>>> terminal. Have you seen this before? Is it maybe because we had
> >>> >>>> already ran
> >>> >>>> the scripts at an earlier stage? Thank you for your help!
> >>> >>>>
> >>> >>>> Levi
> >>> >>>>
> >>> >>>> Traceback (most recent call last):
> >>> >>>>   File
> >>> >>>>
> >>> >>>>
> "/home/administrator/geonode-migration_from20_to_24/migrate_contactroles.py",
> >>> >>>> line 4, in <module>
> >>> >>>>     src = utils.get_src()
> >>> >>>>   File
> >>> >>>> "/home/administrator/geonode-migration_from20_to_24/utils.py",
> >>> >>>> line 21, in get_src
> >>> >>>>     "dbname='%s' user='%s' port='5432' host='%s' password='%s'" %
> >>> >>>> (gn20_dbname, gn20_user, gn20_host, gn20_password)
> >>> >>>>   File "/usr/lib/python2.7/dist-packages/psycopg2/__init__.py",
> line
> >>> >>>> 179, in connect
> >>> >>>>     connection_factory=connection_factory, async=async)
> >>> >>>> psycopg2.OperationalError: FATAL:  could not open file
> >>> >>>> "base/16384/11757": Read-only file system
> >>> >>>>
> >>> >>>> On Mon, Apr 11, 2016 at 4:52 PM, Paolo Corti <pcorti at gmail.com>
> >>> >>>> wrote:
> >>> >>>>>
> >>> >>>>> Looks like the migrations scripts didn't run successfully.
> >>> >>>>> Levi, could you re-run the process and provide the full log of
> the
> >>> >>>>> migration scripts?
> >>> >>>>> Also, make sure to read comments here:
> >>> >>>>> https://github.com/GeoNode/geonode/issues/2067
> >>> >>>>> thanks
> >>> >>>>> p
> >>> >>>>>
> >>> >>>>> On Mon, Apr 11, 2016 at 8:03 AM, Simone Dalmasso
> >>> >>>>> <simone.dalmasso at gmail.com> wrote:
> >>> >>>>> > The issue seems to be originating here
> >>> >>>>> >
> >>> >>>>> >
> >>> >>>>> >
> https://github.com/GeoNode/geonode/blob/master/geonode/base/templatetags/base_tags.py#L54
> .
> >>> >>>>> > And the final error is: ValueError: invalid literal for int()
> >>> >>>>> > with
> >>> >>>>> > base 10:
> >>> >>>>> > 'None'
> >>> >>>>> >  @capooti any hints?
> >>> >>>>> >
> >>> >>>>> > 2016-04-11 13:59 GMT+02:00 Levi Westerveld
> >>> >>>>> > <levi.westerveld at gmail.com>:
> >>> >>>>> >>
> >>> >>>>> >> Hello,
> >>> >>>>> >>
> >>> >>>>> >> We recently moved our data from a 2.0 geonode server
> >>> >>>>> >> (http://geonode.grida.no/) to a new server with geonode 2.4
> >>> >>>>> >> (http://opengeo.grida.no/). geonode 2.4 was successfully
> >>> >>>>> >> installed
> >>> >>>>> >> before we
> >>> >>>>> >> ran the migration using the scripts provided here:
> >>> >>>>> >>
> >>> >>>>> >>
> >>> >>>>> >>
> https://github.com/capooti/geonode/tree/migration_from20_to_24/scripts/migrations/migrate20to24
> .
> >>> >>>>> >>
> >>> >>>>> >> After running the two migration scripts, users, layers, and
> maps
> >>> >>>>> >> data were
> >>> >>>>> >> transferred successfully, but each time I clicked on a layer
> >>> >>>>> >> nothing
> >>> >>>>> >> would
> >>> >>>>> >> happen. When we tried to access the geoserver after logging in
> >>> >>>>> >> we
> >>> >>>>> >> also
> >>> >>>>> >> received an error message.
> >>> >>>>> >>
> >>> >>>>> >> Now we have an Internal Server Error after we tried to open up
> >>> >>>>> >> to
> >>> >>>>> >> the
> >>> >>>>> >> client side so that the server could be accessed externally
> from
> >>> >>>>> >> a
> >>> >>>>> >> browser.
> >>> >>>>> >>
> >>> >>>>> >> The error log file from /var/logs/apache2/error.log is
> attached
> >>> >>>>> >> to
> >>> >>>>> >> this
> >>> >>>>> >> email.
> >>> >>>>> >>
> >>> >>>>> >> perhaps someone here has an idea on what is the best way
> >>> >>>>> >> forward?
> >>> >>>>> >>
> >>> >>>>> >> Many thanks for the help,
> >>> >>>>> >>
> >>> >>>>> >> Levi
> >>> >>>>> >>
> >>> >>>>> >>
> >>> >>>>> >>
> >>> >>>>> >>
> >>> >>>>> >> _______________________________________________
> >>> >>>>> >> geonode-users mailing list
> >>> >>>>> >> geonode-users at lists.osgeo.org
> >>> >>>>> >> http://lists.osgeo.org/mailman/listinfo/geonode-users
> >>> >>>>> >>
> >>> >>>>> >
> >>> >>>>> >
> >>> >>>>> >
> >>> >>>>> > --
> >>> >>>>> > Simone
> >>> >>>>> >
> >>> >>>>> > _______________________________________________
> >>> >>>>> > geonode-users mailing list
> >>> >>>>> > geonode-users at lists.osgeo.org
> >>> >>>>> > http://lists.osgeo.org/mailman/listinfo/geonode-users
> >>> >>>>> >
> >>> >>>>>
> >>> >>>>>
> >>> >>>>>
> >>> >>>>> --
> >>> >>>>> Paolo Corti
> >>> >>>>> Geospatial software developer
> >>> >>>>> web: http://www.paolocorti.net
> >>> >>>>> twitter: @capooti
> >>> >>>>> skype: capooti
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>> --
> >>> >>>> Levi Westerveld
> >>> >>>>
> >>> >>>>
> >>> >>>>
> >>> >>>
> >>> >>>
> >>> >>>
> >>> >>> --
> >>> >>> Simone
> >>> >>
> >>> >>
> >>> >>
> >>> >>
> >>> >> --
> >>> >> Levi Westerveld
> >>> >>
> >>> >>
> >>> >>
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > Levi Westerveld
> >>> >
> >>> >
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Paolo Corti
> >>> Geospatial software developer
> >>> web: http://www.paolocorti.net
> >>> twitter: @capooti
> >>> skype: capooti
> >>
> >>
> >>
> >>
> >> --
> >> Levi Westerveld
> >>
> >>
> >>
> >
> >
> >
> > --
> > Levi Westerveld
> >
> >
> >
>
>
>
> --
> Paolo Corti
> Geospatial software developer
> web: http://www.paolocorti.net
> twitter: @capooti
> skype: capooti
>

-- 
Levi Westerveld
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-users/attachments/20160425/31249a92/attachment-0001.html>