<div dir="ltr">Fyi. My reply was held for moderator's approval before the delivery to this mailing list container <br><br><div class="gmail_quote"><div dir="ltr">---------- Forwarded message ---------<br>From: francesco bartoli <<a href="mailto:xbartolone@gmail.com" target="_blank">xbartolone@gmail.com</a>><br>Date: lun 28 mag 2018 alle ore 08:41<br>Subject: Re: [GeoNode-devel] Docker Strategy Chat<br>To: Olivier Dalang <<a href="mailto:olivier.dalang@gmail.com" target="_blank">olivier.dalang@gmail.com</a>><br>Cc: geonode-devel <<a href="mailto:geonode-devel@lists.osgeo.org" target="_blank">geonode-devel@lists.osgeo.org</a>>, Cristiano Giovando <<a href="mailto:giovand@gmail.com" target="_blank">giovand@gmail.com</a>>, Frédéric Jacon <<a href="mailto:frederic.jacon@camptocamp.com" target="_blank">frederic.jacon@camptocamp.com</a>>, G. Allegri <<a href="mailto:giovanni.allegri@geo-solutions.it" target="_blank">giovanni.allegri@geo-solutions.it</a>>, Gavin Fleming <<a href="mailto:gavin@kartoza.com" target="_blank">gavin@kartoza.com</a>>, Vivien Deparday <<a href="mailto:vdeparday@worldbank.org" target="_blank">vdeparday@worldbank.org</a>>, Alessio Fabiani <<a href="mailto:alessio.fabiani@geo-solutions.it" target="_blank">alessio.fabiani@geo-solutions.it</a>>, Alessandro Parma <<a href="mailto:alessandro.parma@geo-solutions.it" target="_blank">alessandro.parma@geo-solutions.it</a>>, Ariel Nunez <<a href="mailto:ariel@terranodo.io" target="_blank">ariel@terranodo.io</a>><br></div><br><br><div dir="ltr">Hi Olivier,<div><br></div><div>thanks, my comments inline.</div><div><br></div><div>Cheers,</div><div>Francesco<br><br><div class="gmail_quote"><div dir="ltr">Il giorno lun 28 mag 2018 alle ore 06:54 Olivier Dalang <<a href="mailto:olivier.dalang@gmail.com" target="_blank">olivier.dalang@gmail.com</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
<div>Hi !<br><br>I made some progress on my Docker setup, it's now based on 2.7.x (future 2.8.1) :
<a href="https://github.com/olivierdalang/SPCgeonode/tree/master-2.8.1" target="_blank">https://github.com/olivierdalang/SPCgeonode/tree/master-2.8.1</a>
<br><br>Most importantly, I removed my infamous postgres hack to use
OAuth2 instead, so that it's more similar to a typical Geonode
installation.<br><br></div>I also made several other smaller improvements (see changelog/git history), including easy to configure backups using rClone (with sample configuration for amazon s3).<br><div><br>I didn't test it much yet, but basic functionalities seem to work again (uploading layers, setting thumbnails, changing style, wms...).<br><br>I
suggest you guys have a look. If this makes sense, I suggest following next steps :<br>- 1/ To move it somewhere under
<a href="http://github.com/Geonode" target="_blank">github.com/Geonode</a> <br>- 2/ I can work on a PR to document this in Geonode's documentation<br>- 3/ If ready before 2.8.1 (is there a date already ?), we could advertise this as an experimental deployment method for some time, and promote it to supported deployment method
for 2.8.2 release
(if feedback is good)
<br><br></div><div>Help/input still needed on :<br></div><div>-
Geoserver Dockerfile. I don't know much about deploying java web
apps, so I just used the vanilla GeoServer binaries which includes a
Jetty
webserver and added Geonode's geoserver.war on top of it. It looks like
it's working, but I've no idea if there's a better way to do it. Also
there are a lot of warnings on GeoServer startup, not sure how serious they
are.<br></div></div></blockquote><div><br></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_quote">[FB] The recommended java container should be Tomcat. If you are going to work on PR the best approach would be you start from the current repo geoserver-docker</div></blockquote><div><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>- Add the QGIS backend option<br></div><div>- Your thoughts about the idea of hosting this under the main
Geonode repo instead of a separate repo (to avoid replication between
dev setup and production setup), probably to discuss in detail with Francesco.<br></div></div></blockquote></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div class="gmail_quote"><div><br></div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div class="gmail_quote"><div>[FB] I've the impression here that an optimal solution would be to keep separate dev and prod setup which means we might create an unique repo for the prod one where GeoNode is managed as a library (relying on stable pip versions) so we can address also geonode-project under the Ariel's vision and keep a docker-compose for dev on the geonode repo as it is now.</div></div></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div class="gmail_quote"><div> </div></div></div></blockquote><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>- See if there's a way to run the integration tests on the docker-compose stack<br></div></div></blockquote><div><br></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div class="gmail_quote"><div>[FB] This should be very important imho and at least should cover minimal use cases like the basic functionalities you mentioned above</div></div></div></blockquote><div><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div><div>-
There are some problems with the monitoring module (I disabled it, it crashes during
migrations, looks like it's trying to reach GeoServer which isn't
available during migrations in my setup)<br></div><div>- Discuss if there's a possible strategy for long term migration path of the geonode data directory (to avoid data loss/starting from scratch for upcoming updates)<br></div><div>- General review/comments<br></div></div></blockquote><div><br></div></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><div class="gmail_quote"><div>[FB] If working on a PR my general suggestion would be to accommodate your work as much as possible with that already in place under the *-docker repos on the Geonode github organization</div></div></div></blockquote><div><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>Cheers,<br><br></div><div>Olivier</div>
<br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 20, 2018 at 1:40 AM, Ariel Nunez <span dir="ltr"><<a href="mailto:ariel@terranodo.io" target="_blank">ariel@terranodo.io</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Olivier,<br><br>I suscribe to the vision of geonode as a library and that a geonode-project should be a simple django project that does not import anything from geonode.settings. The reason for this is that geonode becomes a starting point and what people start to build are their own projects.<br><br>I hope at some point in the near future we deprecate geonode.settings at all and instead let an ecosystem of different starter projects built on the same docker base images flourish.<span class="m_1896988230684334013m_1814266692921434279m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668HOEnZb"><font color="#888888"><br><br>-a</font></span><div class="m_1896988230684334013m_1814266692921434279m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668HOEnZb"><div class="m_1896988230684334013m_1814266692921434279m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668h5"><br><div class="gmail_quote"><div dir="ltr">On Thu, Apr 19, 2018 at 5:52 AM Olivier Dalang <<a href="mailto:olivier.dalang@gmail.com" target="_blank">olivier.dalang@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div dir="ltr"><div>Hello all !</div><div><br></div><div>Thank you all for the great discussion !! I very much appreciated how much you all took into account my inputs and I felt part of the discussion despite being deeply asleep :-) <span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Thank you very much for the recording Cristiano !</span></div><div><br></div><div><br></div><div>I think there are two different needs that were discussed, the need for stable docker images (typically for providers to build their custom stacks on reliable/secure images), and the need for a supported deployment through docker-compose (typically for smaller users without skills/ressources to do a proper deployment on their own).</div><div><br></div><div>So far I focused only on the docker-compose for deployment (as it's the case we need to support here).</div><div>
<br class="m_1896988230684334013m_1814266692921434279m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668m_4549730285531974548m_-2510610479178620297m_8234380312452560400m_7216416946810614236gmail-m_765127961121936531gmail-Apple-interchange-newline"><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">But I agree having those stable images would be great too. I think that even if the needs are different, in the end they may very well align in one setup, where the docker-compose setup for production could even serve as an integration test of the individual docker files.</span>
<br></div><div><br></div><div>It's not completely clear to me what the scope of those images would be. I think at the very least, they would include all dependencies (system and python).</div><div>Would they also include build steps (instead of pulling external builds from <a href="http://builds.geonode.org" rel="noreferrer noreferrer" target="_blank">builds.geonode.org</a>) ? (may be good in terms of build stability and make it easier to contribute, also with multistage dockerfiles it's quite easy to keep images lightweight despite having the whole build process in there).</div><div>Would they be though as final images (that you use as is, just by configuring env vars), or base images (that are supposed to serve as base to user's custom dockerfiles in a composition) ?</div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Also it will be hard to find the sweet spot between <span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">too little opinionated (it would become a pain to configure and loose stability) and too much (you wouldn't be able to use it anymore in a custom stack).</span><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span> </span></span>
</span></div><div><br></div><div><br></div><div>I had another thought about docker-compose that I don't think was raised : with the current geonode-project approach, there will be some duplication, because there is already another docker-compose file in Geonode's main repo (for development and testing of geonode itself - which is obviously also needed).</div><div><br></div><div>Could it be that the docker-compose setup for deployment is built on top of the docker-compose for development ? In the end, the geonode repo already contains a django project. It's just a matter of being able to override the settings. I'm not sure if this would end up being overly complicated for nothing, or if it would be a win because of increased similarity between dev setup and deployment setup. Going that way would be a step towards a continuous delivery model (a big incentive for devs to upstream their features). <span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I also believe that with Alessio's recent commits for UI customization, around 50% of the use cases for geonode-project just vanished (thanks for that great feature :-) ).</span>
</div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I also agree with Tim on the idea of supporting both the QGIS and Geoserver backends. I'm not familiar enough with the differences that implies so far to tell how much work that would be.</span></div><div><br></div><div><br></div><div>And about the smaller scale technical questions :</div><div><br></div><div>- Containers restarting VS. wait-for-postgres.sh : I have no strong opinion about that. It's true that it would look a little bit more elegant at first startup, but it also adds a little bit of complexity in the entrypoint, and we must make sure we don't create artificial blockages (e.g. waiting for geoserver would be a bad idea, as we should be able to start even if geoserver is down).</div><div><br></div><div>- Additional requirements (pip) : I'll try to make a PR to upstream those fixes including some other to make pip install more stable, as discussed here : <a href="https://github.com/GeoNode/geonode/issues/3227" rel="noreferrer noreferrer" target="_blank">https://github.com/GeoNode/geonode/issues/3227</a></div><div><br></div><div>- A<span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">bout the more minimal docker-compose (without letsencrypt and other optional components), so far I use<span> </span></span><font color="#222222" size="2"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">docker-compose up nginx django geoserver postgres</span></font><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span> </span>for that purpose, but we could indeed also use docker-compose overrides to achieve similar result. I would vote for the option that looks the most clear in the command line.</span></div><div dir="auto"><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><br></div><div>I'll continue to push my setup a bit further anyway (to 2.8) and agree with idea of another talk at some point.</div><div dir="auto"><br></div><div dir="auto">Thanks again!</div><div><br></div><div>Cheers,</div><div><br></div><div>Olivier</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 19, 2018 at 2:45 AM, Cristiano Giovando <span dir="ltr"><<a href="mailto:giovand@gmail.com" rel="noreferrer noreferrer" target="_blank">giovand@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks to everyone who joined the docker chat today - here's the<br>
recording in case you missed it:<br>
<br>
<a href="https://drive.google.com/open?id=1voxdexJQLPdikLAW6hLsXk7TwYkv9KJT" rel="noreferrer noreferrer noreferrer" target="_blank">https://drive.google.com/open?id=1voxdexJQLPdikLAW6hLsXk7TwYkv9KJT</a><br>
<br>
Please share any feedback here or directly in the GNIP:<br>
<a href="https://github.com/GeoNode/geonode/issues/3707" rel="noreferrer noreferrer noreferrer" target="_blank">https://github.com/GeoNode/geonode/issues/3707</a><br>
<br>
Looking forward to a successful and coordinated approach! ;)<br>
<div><div class="m_1896988230684334013m_1814266692921434279m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668m_4549730285531974548m_-2510610479178620297m_8234380312452560400m_7216416946810614236h5"><br>
On Tue, Apr 17, 2018 at 9:44 PM, Cristiano Giovando <<a href="mailto:giovand@gmail.com" rel="noreferrer noreferrer" target="_blank">giovand@gmail.com</a>> wrote:<br>
> Hey all,<br>
><br>
> Following some good conversations around the topic at the summit and the<br>
> recent GNIP by Olivier, we're planning a call tomorrow at 15:00 CEST / 13:00<br>
> UTC to chat about a common strategy. We're inviting anything who's<br>
> interested to join.<br>
><br>
> Here's the GNIP for reference:<br>
> <a href="https://github.com/GeoNode/geonode/issues/3707" rel="noreferrer noreferrer noreferrer" target="_blank">https://github.com/GeoNode/geonode/issues/3707</a><br>
><br>
> Meeting link:<br>
> <a href="https://worldbankgroup.webex.com/worldbankgroup/j.php?MTID=mab1ab3faf3942146706111bc9eaad51f" rel="noreferrer noreferrer noreferrer" target="_blank">https://worldbankgroup.webex.com/worldbankgroup/j.php?MTID=mab1ab3faf3942146706111bc9eaad51f</a><br>
><br>
> Join by phone<br>
> 1-650-479-3207 Call-in toll number (US/Canada)<br>
> Access code: 733 296 941<br>
><br>
> Global call-in numbers:<br>
> <a href="https://worldbankgroup.webex.com/cmp3200/webcomponents/widget/globalcallin/globalcallin.do?siteurl=worldbankgroup&serviceType=MC&ED=529265077&tollFree=0" rel="noreferrer noreferrer noreferrer" target="_blank">https://worldbankgroup.webex.com/cmp3200/webcomponents/widget/globalcallin/globalcallin.do?siteurl=worldbankgroup&serviceType=MC&ED=529265077&tollFree=0</a><br>
><br>
> See you tomorrow!<br>
><br>
> Cristiano<br>
</div></div>_______________________________________________<br>
geonode-devel mailing list<br>
<a href="mailto:geonode-devel@lists.osgeo.org" rel="noreferrer noreferrer" target="_blank">geonode-devel@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/geonode-devel" rel="noreferrer noreferrer noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/geonode-devel</a><br>
</blockquote></div><br></div>
</blockquote></div>
</div></div></blockquote></div><br></div></div>
</blockquote></div></div></div></div></div>