[GeoNode-devel] GNIP-64 Status and Plan

Wed Nov 13 13:47:13 PST 2019

Hello all,

I have updated the GNIP with appropriate links as suggested. I've also
created a new document per Alessio's suggestion. Feel free to update it or
let me know how I could make it better. I will be filling it out more as I
go and probably things will get changed over time.

By the way, please tell me if the link does not work or you cannot edit it.

GNIP-64 tasks spreadsheet:
https://docs.google.com/spreadsheets/d/1gr4ISkFBarY-7i_DPi0Vt_FnA5_uF30dGUgYpcIZ-xw/edit#gid=0

Cheers,
Travis

On Tue, Nov 5, 2019 at 11:18 AM Travis Brundage <travislbrundage at gmail.com>
wrote:

> Thank you all for your feedback!
>
> Toni: No actually, I am still having that issue. :/ That is a good hint
> though - I will debug some more, but if anyone knows why that is happening,
> it would be much appreciated...
>
> Alessio: Thanks so much for your feedback. I think your suggestions make
> sense. I'll work on that today and update the GNIP main post with the
> relevant links and post a shorter notice as a comment about the update.
>
> Cheers,
> Travis
>
> On Tue, Nov 5, 2019 at 1:55 AM Toni Schönbuchner <
> toni.schoenbuchner at csgis.de> wrote:
>
>> A W E S O M E ~ T R A V I S !
>>
>> Many many thanks!
>>
>> (
>> Have you already found a solution for the celery path problem? If I
>> remember
>> corectly there have been some updates on celery-beat (maybe celery as
>> well) which might
>> cause issues with docker ...
>> )
>>
>> – toni
>>
>>
>> *Von: *Travis Brundage <travislbrundage at gmail.com>
>> *Betreff: **[GeoNode-devel] GNIP-64 Status and Plan*
>> *Datum: *5. November 2019 um 05:22:59 MEZ
>> *An: *geonode-devel at lists.osgeo.org
>>
>>
>> Hi all,
>>
>> Just wanted to write a status report with regards to GNIP-64. I am
>> excited to hear that help is on the way too :)
>>
>> Most of what Coop wrote in the GNIP is still accurate. Broadly speaking,
>> I see this broken down into the following steps:
>> 1. Python library analysis
>> 2. Test suite improvements
>> 3a. Python library updates
>> 3b. GeoNode codebase update
>> 3c. django-osgeo-importer update
>> 4. Testing and release
>> 5. Repeat for Django 2
>>
>> Coop had already started a spreadsheet detailing the libraries GeoNode
>> uses and their Python 3 compatibility. I have updated this recently. You
>> can find it here:
>> https://docs.google.com/spreadsheets/d/1buUuIeGPL8zfv5xFotfGMpxHt2i8njoCEh4Y4kna3rY/edit#gid=0
>>
>> From this you can see a few categories.
>> 1. Some libraries are completely outdated and inactive and require
>> removal and if needed, replacement.
>> 2. Some libraries do not yet have Python 3 compatibility, but we could
>> update them for the community. Otherwise, we’ll need to remove and possibly
>> replace them.
>> 3. Some libraries do not yet have Python 3 compatibility, but the
>> community is working on it or their github still has activity. We could
>> save these for last or offer to help their community upgrade.
>> 4. Some libraries do not officially support Python 3 or 3.7 specifically,
>> but seem to build fine as is in a Python 3.7 environment.
>> 5. Some libraries are GeoNode community’s. :)
>> 6. Some libraries have newer versions that are Python 3+ compatible, but
>> not backwards compatible with Python 2.7. These need to be delayed until
>> GeoNode core is updated.
>> 7. The majority thankfully - some libraries are fully Python 2.7 and 3.7+
>> compatible.
>>
>> Next is improving GeoNode’s test suite. I don’t have a precise goal here,
>> but want to improve the speed and coverage of GeoNode’s tests to prepare
>> for the Python 3 upgrade, so we can more easily identify any functionality
>> that gets broken. I have already run into a few issues and made another
>> thread about this.
>>
>> For upgrading the codebase, I planned to branch off the next release,
>> 2.10.1.
>>
>> From what I have researched, I don’t know if there’s a “correct” way to
>> update a codebase to Python 3, but seems like there are a few strategies.
>> My plan was to create a new branch and brute force the 2to3.py script, and
>> hopefully with good test coverage, get a useful coverage report to see what
>> has broken.
>>
>> Possibly that is too much and won’t prove fruitful, in which case I would
>> start a new branch and do it app by app instead. I have not tested 2to3.py
>> in the wild before, so I don’t know how helpful it is, but from what I’ve
>> read it is a good starting point usually.
>>
>> As a side note, it will be helpful to configure your IDE of choice with a
>> Python 3 interpreter. I use PyCharm, so some instructions for that here:
>> https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html
>>
>> In general my plan of attack is:
>> 1. Run Python 2to3 script
>> 2. Clean up anything breaking GeoNode build entirely
>> 3. Run tests for coverage report to see what breaks
>> 4. Clean up until tests pass
>> 5. Check visually in GeoNode that the app seems to function
>> 6. Clean up until interactions work as expected
>>
>> Simultaneously we’ll need to update the remaining Python libraries. This
>> will be a bit tricky as some provide Python 3 compatibility on later
>> versions, but not with Python 2 compatibility. Also, some still do not have
>> Python 3 compatibility at all, so the work is on us or drop those libraries.
>>
>> We'll also need to make sure GeoNode's libraries are updated, and it was
>> planned to do django-osgeo-importer as part of the work too.
>>
>> If all goes well, we should have a functioning GeoNode running on Python
>> 3. :)
>> I think Alessio’s testing document will be very helpful again here:
>> https://docs.google.com/spreadsheets/d/13aaZ1l_2_SHzdmz9DwExEO_jxwgNuXfMa6oF5eWIv_s/edit#gid=376642658
>>
>> I don’t know exactly how this work should be merged in (or if it would
>> be), as I assume development will continue on GeoNode master, so we will
>> need to resolve changes along the way. This won’t be a problem for a little
>> while though.
>>
>> Finally, repeat for Django 2. My hope is this part is easier, as I think
>> there are less compatibility conflicts there, but we’ll see.
>>
>> I think that about wraps it up. Let me know your thoughts or if you have
>> any advice for me, and feel free to message me here or on Gitter to
>> coordinate on this effort. Thank you all for helping out on this!
>>
>> Cheers,
>> Travis
>>
>>
>>
>> *Von: *Alessio Fabiani <alessio.fabiani at geo-solutions.it>
>> *Betreff: **Aw: [GeoNode-devel] GNIP-64 Status and Plan*
>> *Datum: *5. November 2019 um 10:04:57 MEZ
>> *An: *Travis Brundage <travislbrundage at gmail.com>, Travis Brundage <
>> tbrundage at boundlessgeo.com>
>> *Kopie: *geonode-devel <geonode-devel at lists.osgeo.org>
>>
>>
>> @Travis Brundage <tbrundage at boundlessgeo.com> awesome plan, thanks.
>>
>> Just a couple of comments, for now:
>>
>> 1. After the release, the "master" branch will become GeoNode 3.0 and
>> officially it will start the Python 3 upgrade.
>>
>> 2. I would keep the issue (https://github.com/GeoNode/geonode/issues/4276)
>> updated with the references to discussions and decisions.
>> We should edit the GNIP description and add to the bottom all the links
>> to ml discussion also, that can be retrieved from here
>> http://osgeo-org.1560.x6.nabble.com/GeoNode-devel-f5185388.html
>>
>> Also, at a certain point, we would need (at least this is my suggestion)
>> to translate the plan into a Google spreadheest and a list of tasks so that
>> other people can easily keep them a start working in parallel. What do you
>> think?
>>
>> Thanks again,
>> Alessio.
>>
>>
>>
>> Il giorno mar 5 nov 2019 alle ore 05:26 Travis Brundage <
>> travislbrundage at gmail.com> ha scritto:
>>
>>> Hi all,
>>>
>>> Just wanted to write a status report with regards to GNIP-64. I am
>>> excited to hear that help is on the way too :)
>>>
>>> Most of what Coop wrote in the GNIP is still accurate. Broadly speaking,
>>> I see this broken down into the following steps:
>>> 1. Python library analysis
>>> 2. Test suite improvements
>>> 3a. Python library updates
>>> 3b. GeoNode codebase update
>>> 3c. django-osgeo-importer update
>>> 4. Testing and release
>>> 5. Repeat for Django 2
>>>
>>> Coop had already started a spreadsheet detailing the libraries GeoNode
>>> uses and their Python 3 compatibility. I have updated this recently. You
>>> can find it here:
>>> https://docs.google.com/spreadsheets/d/1buUuIeGPL8zfv5xFotfGMpxHt2i8njoCEh4Y4kna3rY/edit#gid=0
>>>
>>> From this you can see a few categories.
>>> 1. Some libraries are completely outdated and inactive and require
>>> removal and if needed, replacement.
>>> 2. Some libraries do not yet have Python 3 compatibility, but we could
>>> update them for the community. Otherwise, we’ll need to remove and possibly
>>> replace them.
>>> 3. Some libraries do not yet have Python 3 compatibility, but the
>>> community is working on it or their github still has activity. We could
>>> save these for last or offer to help their community upgrade.
>>> 4. Some libraries do not officially support Python 3 or 3.7
>>> specifically, but seem to build fine as is in a Python 3.7 environment.
>>> 5. Some libraries are GeoNode community’s. :)
>>> 6. Some libraries have newer versions that are Python 3+ compatible, but
>>> not backwards compatible with Python 2.7. These need to be delayed until
>>> GeoNode core is updated.
>>> 7. The majority thankfully - some libraries are fully Python 2.7 and
>>> 3.7+ compatible.
>>>
>>> Next is improving GeoNode’s test suite. I don’t have a precise goal
>>> here, but want to improve the speed and coverage of GeoNode’s tests to
>>> prepare for the Python 3 upgrade, so we can more easily identify any
>>> functionality that gets broken. I have already run into a few issues and
>>> made another thread about this.
>>>
>>> For upgrading the codebase, I planned to branch off the next release,
>>> 2.10.1.
>>>
>>> From what I have researched, I don’t know if there’s a “correct” way to
>>> update a codebase to Python 3, but seems like there are a few strategies.
>>> My plan was to create a new branch and brute force the 2to3.py script, and
>>> hopefully with good test coverage, get a useful coverage report to see what
>>> has broken.
>>>
>>> Possibly that is too much and won’t prove fruitful, in which case I
>>> would start a new branch and do it app by app instead. I have not tested
>>> 2to3.py in the wild before, so I don’t know how helpful it is, but from
>>> what I’ve read it is a good starting point usually.
>>>
>>> As a side note, it will be helpful to configure your IDE of choice with
>>> a Python 3 interpreter. I use PyCharm, so some instructions for that here:
>>> https://www.jetbrains.com/help/pycharm/configuring-python-interpreter.html
>>>
>>> In general my plan of attack is:
>>> 1. Run Python 2to3 script
>>> 2. Clean up anything breaking GeoNode build entirely
>>> 3. Run tests for coverage report to see what breaks
>>> 4. Clean up until tests pass
>>> 5. Check visually in GeoNode that the app seems to function
>>> 6. Clean up until interactions work as expected
>>>
>>> Simultaneously we’ll need to update the remaining Python libraries. This
>>> will be a bit tricky as some provide Python 3 compatibility on later
>>> versions, but not with Python 2 compatibility. Also, some still do not have
>>> Python 3 compatibility at all, so the work is on us or drop those libraries.
>>>
>>> We'll also need to make sure GeoNode's libraries are updated, and it was
>>> planned to do django-osgeo-importer as part of the work too.
>>>
>>> If all goes well, we should have a functioning GeoNode running on Python
>>> 3. :)
>>> I think Alessio’s testing document will be very helpful again here:
>>> https://docs.google.com/spreadsheets/d/13aaZ1l_2_SHzdmz9DwExEO_jxwgNuXfMa6oF5eWIv_s/edit#gid=376642658
>>>
>>> I don’t know exactly how this work should be merged in (or if it would
>>> be), as I assume development will continue on GeoNode master, so we will
>>> need to resolve changes along the way. This won’t be a problem for a little
>>> while though.
>>>
>>> Finally, repeat for Django 2. My hope is this part is easier, as I think
>>> there are less compatibility conflicts there, but we’ll see.
>>>
>>> I think that about wraps it up. Let me know your thoughts or if you have
>>> any advice for me, and feel free to message me here or on Gitter to
>>> coordinate on this effort. Thank you all for helping out on this!
>>>
>>> Cheers,
>>> Travis
>>> _______________________________________________
>>> geonode-devel mailing list
>>> geonode-devel at lists.osgeo.org
>>> https://lists.osgeo.org/mailman/listinfo/geonode-devel
>>>
>>
>>
>> --
>> ==
>> GeoServer Professional Services from the experts! Visit
>> http://goo.gl/it488V for more information.
>> ==
>> Ing. Alessio Fabiani
>> @alfa7691
>> Founder/Technical Lead
>>
>> GeoSolutions S.A.S.
>> Via di Montramito 3/A - 55054  Massarosa (LU) - Italy
>> phone: +39 0584 962313
>> fax:     +39 0584 1660272
>> mob:   +39 331 6233686
>>
>> http://www.geo-solutions.it
>> http://twitter.com/geosolutions_it
>> -------------------------------------------------------
>> Con riferimento alla normativa sul trattamento dei dati personali (Reg.
>> UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si
>> precisa che ogni circostanza inerente alla presente email (il suo
>> contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è
>> riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il
>> messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra
>> operazione è illecita. Le sarei comunque grato se potesse darmene notizia.
>>
>> This email is intended only for the person or entity to which it is
>> addressed and may contain information that is privileged, confidential or
>> otherwise protected from disclosure. We remind that - as provided by
>> European Regulation 2016/679 “GDPR” - copying, dissemination or use of this
>> e-mail or the information herein by anyone other than the intended
>> recipient is prohibited. If you have received this email by mistake, please
>> notify us immediately by telephone or e-mail.
>>
>>
>> _______________________________________________
>> geonode-devel mailing list
>> geonode-devel at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/geonode-devel
>>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/geonode-devel/attachments/20191113/302ab56e/attachment-0001.html>