[GeoNode-users] failed migration from 2.0 to 2.4

Paolo Corti pcorti at gmail.com
Wed Apr 27 08:57:35 PDT 2016 

You need to manually migrate the GeoServer data directory, as indicated here:
https://github.com/capooti/geonode/blob/migration_from20_to_24/scripts/migrations/migrate20to24/index.rst

"Copy GeoServer data directory to the GeoServer instance included with
GeoNode 2.4. Login with the GeoServer admin user to check if
everything is working properly."

p

On Mon, Apr 25, 2016 at 3:46 AM, Levi Westerveld
<levi.westerveld at gmail.com> wrote:
> Thank you. We checked, and it seems it is pointing in the right direction.
>
> When running the migrate2.sh we observe that the underlying data for each
> layer is not transferred to the new server with geonode 2.4 and the new
> geoserver. No store is created in the new geoserver during the migration
> process. Could that be the root cause of the migration issue? Should a store
> be automatically created in the new geoserver during the migration, or
> should we create one before running the migration?
>
> thank you,
>
> Levi
>
>
> On Tue, Apr 19, 2016 at 4:33 PM, Paolo Corti <pcorti at gmail.com> wrote:
>>
>> Yes, it looks like you are failing to migrate the GeoServer data
>> directory.
>> Check if it is pointing to the correct location.
>> Please keep the list in copy, other people may benefit from those
>> answers in future
>> p
>>
>> On Tue, Apr 19, 2016 at 9:33 AM, Levi Westerveld
>> <levi.westerveld at gmail.com> wrote:
>> > I attached a screenshot of the geoserver and the geonode. As you see, no
>> > layers show up in the geoserver but they do in the geonode.. I hope this
>> > helps!
>> >
>> > On Tue, Apr 19, 2016 at 3:28 PM, Levi Westerveld
>> > <levi.westerveld at gmail.com>
>> > wrote:
>> >>
>> >> Thank you Paolo for your quick answer,
>> >>
>> >> In the thread you mentioned I saw you made this comment particular
>> >> comment:
>> >>
>> >> please note that, as suggested in the updated instruction, now the
>> >> migration process consists of the following steps:
>> >>
>> >> run migrate.py
>> >> restore geoserver
>> >> run migrate2.py
>> >>
>> >> it looks like you are on step 1? Therefore you don't see your layers,
>> >> as
>> >> they have been loaded in db with default permissions: in this case only
>> >> if
>> >> you are authenticated you will be able to see them.
>> >>
>> >> Basically, you need to login in GeoNode :)
>> >>
>> >> After running migrate2.py script, permissions will be correctly setup.
>> >>
>> >> Let me know if this works.
>> >>
>> >> By the way, I did only step 1 with your backup, I cannot process
>> >> further
>> >> because I would need the geoserver data directory to run step 2.
>> >>
>> >>
>> >> Would you suggest us to first restore geoserver and then try running
>> >> the
>> >> second migration script again? It seems from reading the thread that
>> >> the
>> >> internal server error issue and the layers not migrating correctly
>> >> issue are
>> >> related.
>> >>
>> >> Thank you,
>> >>
>> >> Levi
>> >>
>> >>
>> >> On Tue, Apr 19, 2016 at 3:03 PM, Paolo Corti <pcorti at gmail.com> wrote:
>> >>>
>> >>> Hi
>> >>> this won't help, you will loose all the style, metadata, tags,
>> >>> permissions doing so.
>> >>>
>> >>> Checking your logs, first script ran successfully.
>> >>> You have a problem at the beginning of the second script: before
>> >>> running it make sure that GeoServer is communicating correctly with
>> >>> GeoNode: you should be able to see the layers in the GeoServer
>> >>> preview, for example.
>> >>> The fact that links are broken, this is expected, as updating layers
>> >>> failed at the beginning of the second script.
>> >>> You are exactly in the same problem as a couple of users here (they
>> >>> managed to fix it): https://github.com/GeoNode/geonode/issues/2067
>> >>>
>> >>> Hope it helps
>> >>> p
>> >>>
>> >>> On Tue, Apr 19, 2016 at 7:31 AM, Levi Westerveld
>> >>> <levi.westerveld at gmail.com> wrote:
>> >>> > As a follow up to this email.
>> >>> >
>> >>> > It appears that for the second error none of the GIS data was
>> >>> > actually
>> >>> > migrated but just a description of the layers was. We will now try
>> >>> > to
>> >>> > import
>> >>> > the layers that did not migrate using the sudo geonode importalyers
>> >>> > <dat_dir> to the new geonode and I just hope that the imported
>> >>> > layers
>> >>> > will
>> >>> > link back with the descriptive data that was imported for each
>> >>> > layer.
>> >>> >
>> >>> > The first issue remains unresolved..
>> >>> >
>> >>> > Thank you,
>> >>> >
>> >>> > levi
>> >>> >
>> >>> > On Tue, Apr 19, 2016 at 12:48 PM, Levi Westerveld
>> >>> > <levi.westerveld at gmail.com> wrote:
>> >>> >>
>> >>> >> Dear Simone and Paolo,
>> >>> >>
>> >>> >> We have reinstalled ubuntu and ran the migration files successfully
>> >>> >> again
>> >>> >> from our geonode 2.0 server to our geonode 2.4 instance.
>> >>> >>
>> >>> >> We have two issues
>> >>> >>
>> >>> >> 1. we are able to access the geonode 2.4 if we are signed in as a
>> >>> >> user
>> >>> >> at
>> >>> >> http://opengeo.grida.no/ but if not signed in then an internal
>> >>> >> server
>> >>> >> issues
>> >>> >> shows up
>> >>> >> 2. when we are in the layers section and click on a layer it seems
>> >>> >> that
>> >>> >> the link is broken, and we are not redirected to that layer, but
>> >>> >> stay
>> >>> >> on the
>> >>> >> same page
>> >>> >>
>> >>> >> I attached the two log files for the migration.
>> >>> >>
>> >>> >> Thank you!
>> >>> >>
>> >>> >> Levi
>> >>> >>
>> >>> >> On Fri, Apr 15, 2016 at 9:14 AM, Simone Dalmasso
>> >>> >> <simone.dalmasso at gmail.com> wrote:
>> >>> >>>
>> >>> >>> Levi,
>> >>> >>>
>> >>> >>> that is definitely not a good error but can be solved. Linux puts
>> >>> >>> the
>> >>> >>> file system in read only mode when there are inconsistencies on
>> >>> >>> the
>> >>> >>> disk.I
>> >>> >>> suggest you to google for that error, you will find tons of
>> >>> >>> suggestions on
>> >>> >>> how to check and fix the disk errors.
>> >>> >>>
>> >>> >>> 2016-04-14 22:36 GMT+02:00 Levi Westerveld
>> >>> >>> <levi.westerveld at gmail.com>:
>> >>> >>>>
>> >>> >>>> Hello Paolo and Simone,
>> >>> >>>>
>> >>> >>>> We just ran the script again and we got a new error message in
>> >>> >>>> the
>> >>> >>>> terminal. Have you seen this before? Is it maybe because we had
>> >>> >>>> already ran
>> >>> >>>> the scripts at an earlier stage? Thank you for your help!
>> >>> >>>>
>> >>> >>>> Levi
>> >>> >>>>
>> >>> >>>> Traceback (most recent call last):
>> >>> >>>>   File
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> "/home/administrator/geonode-migration_from20_to_24/migrate_contactroles.py",
>> >>> >>>> line 4, in <module>
>> >>> >>>>     src = utils.get_src()
>> >>> >>>>   File
>> >>> >>>> "/home/administrator/geonode-migration_from20_to_24/utils.py",
>> >>> >>>> line 21, in get_src
>> >>> >>>>     "dbname='%s' user='%s' port='5432' host='%s' password='%s'" %
>> >>> >>>> (gn20_dbname, gn20_user, gn20_host, gn20_password)
>> >>> >>>>   File "/usr/lib/python2.7/dist-packages/psycopg2/__init__.py",
>> >>> >>>> line
>> >>> >>>> 179, in connect
>> >>> >>>>     connection_factory=connection_factory, async=async)
>> >>> >>>> psycopg2.OperationalError: FATAL:  could not open file
>> >>> >>>> "base/16384/11757": Read-only file system
>> >>> >>>>
>> >>> >>>> On Mon, Apr 11, 2016 at 4:52 PM, Paolo Corti <pcorti at gmail.com>
>> >>> >>>> wrote:
>> >>> >>>>>
>> >>> >>>>> Looks like the migrations scripts didn't run successfully.
>> >>> >>>>> Levi, could you re-run the process and provide the full log of
>> >>> >>>>> the
>> >>> >>>>> migration scripts?
>> >>> >>>>> Also, make sure to read comments here:
>> >>> >>>>> https://github.com/GeoNode/geonode/issues/2067
>> >>> >>>>> thanks
>> >>> >>>>> p
>> >>> >>>>>
>> >>> >>>>> On Mon, Apr 11, 2016 at 8:03 AM, Simone Dalmasso
>> >>> >>>>> <simone.dalmasso at gmail.com> wrote:
>> >>> >>>>> > The issue seems to be originating here
>> >>> >>>>> >
>> >>> >>>>> >
>> >>> >>>>> >
>> >>> >>>>> > https://github.com/GeoNode/geonode/blob/master/geonode/base/templatetags/base_tags.py#L54.
>> >>> >>>>> > And the final error is: ValueError: invalid literal for int()
>> >>> >>>>> > with
>> >>> >>>>> > base 10:
>> >>> >>>>> > 'None'
>> >>> >>>>> >  @capooti any hints?
>> >>> >>>>> >
>> >>> >>>>> > 2016-04-11 13:59 GMT+02:00 Levi Westerveld
>> >>> >>>>> > <levi.westerveld at gmail.com>:
>> >>> >>>>> >>
>> >>> >>>>> >> Hello,
>> >>> >>>>> >>
>> >>> >>>>> >> We recently moved our data from a 2.0 geonode server
>> >>> >>>>> >> (http://geonode.grida.no/) to a new server with geonode 2.4
>> >>> >>>>> >> (http://opengeo.grida.no/). geonode 2.4 was successfully
>> >>> >>>>> >> installed
>> >>> >>>>> >> before we
>> >>> >>>>> >> ran the migration using the scripts provided here:
>> >>> >>>>> >>
>> >>> >>>>> >>
>> >>> >>>>> >>
>> >>> >>>>> >> https://github.com/capooti/geonode/tree/migration_from20_to_24/scripts/migrations/migrate20to24.
>> >>> >>>>> >>
>> >>> >>>>> >> After running the two migration scripts, users, layers, and
>> >>> >>>>> >> maps
>> >>> >>>>> >> data were
>> >>> >>>>> >> transferred successfully, but each time I clicked on a layer
>> >>> >>>>> >> nothing
>> >>> >>>>> >> would
>> >>> >>>>> >> happen. When we tried to access the geoserver after logging
>> >>> >>>>> >> in
>> >>> >>>>> >> we
>> >>> >>>>> >> also
>> >>> >>>>> >> received an error message.
>> >>> >>>>> >>
>> >>> >>>>> >> Now we have an Internal Server Error after we tried to open
>> >>> >>>>> >> up
>> >>> >>>>> >> to
>> >>> >>>>> >> the
>> >>> >>>>> >> client side so that the server could be accessed externally
>> >>> >>>>> >> from
>> >>> >>>>> >> a
>> >>> >>>>> >> browser.
>> >>> >>>>> >>
>> >>> >>>>> >> The error log file from /var/logs/apache2/error.log is
>> >>> >>>>> >> attached
>> >>> >>>>> >> to
>> >>> >>>>> >> this
>> >>> >>>>> >> email.
>> >>> >>>>> >>
>> >>> >>>>> >> perhaps someone here has an idea on what is the best way
>> >>> >>>>> >> forward?
>> >>> >>>>> >>
>> >>> >>>>> >> Many thanks for the help,
>> >>> >>>>> >>
>> >>> >>>>> >> Levi
>> >>> >>>>> >>
>> >>> >>>>> >>
>> >>> >>>>> >>
>> >>> >>>>> >>
>> >>> >>>>> >> _______________________________________________
>> >>> >>>>> >> geonode-users mailing list
>> >>> >>>>> >> geonode-users at lists.osgeo.org
>> >>> >>>>> >> http://lists.osgeo.org/mailman/listinfo/geonode-users
>> >>> >>>>> >>
>> >>> >>>>> >
>> >>> >>>>> >
>> >>> >>>>> >
>> >>> >>>>> > --
>> >>> >>>>> > Simone
>> >>> >>>>> >
>> >>> >>>>> > _______________________________________________
>> >>> >>>>> > geonode-users mailing list
>> >>> >>>>> > geonode-users at lists.osgeo.org
>> >>> >>>>> > http://lists.osgeo.org/mailman/listinfo/geonode-users
>> >>> >>>>> >
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>>
>> >>> >>>>> --
>> >>> >>>>> Paolo Corti
>> >>> >>>>> Geospatial software developer
>> >>> >>>>> web: http://www.paolocorti.net
>> >>> >>>>> twitter: @capooti
>> >>> >>>>> skype: capooti
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>> --
>> >>> >>>> Levi Westerveld
>> >>> >>>>
>> >>> >>>>
>> >>> >>>>
>> >>> >>>
>> >>> >>>
>> >>> >>>
>> >>> >>> --
>> >>> >>> Simone
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >> --
>> >>> >> Levi Westerveld
>> >>> >>
>> >>> >>
>> >>> >>
>> >>> >
>> >>> >
>> >>> >
>> >>> > --
>> >>> > Levi Westerveld
>> >>> >
>> >>> >
>> >>> >
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> Paolo Corti
>> >>> Geospatial software developer
>> >>> web: http://www.paolocorti.net
>> >>> twitter: @capooti
>> >>> skype: capooti
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> Levi Westerveld
>> >>
>> >>
>> >>
>> >
>> >
>> >
>> > --
>> > Levi Westerveld
>> >
>> >
>> >
>>
>>
>>
>> --
>> Paolo Corti
>> Geospatial software developer
>> web: http://www.paolocorti.net
>> twitter: @capooti
>> skype: capooti
>
>
>
>
> --
> Levi Westerveld
>
>
>



-- 
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti

More information about the geonode-users mailing list