<div dir="ltr">Hi Olivier,<div><br></div><div>well actually the main problem with the current deployment is that we haven't a maintainer and despite the fact GeoNode is with no doubt complex I believe the issue #3722 is not totally relevant (I would have been expected at least a PR for the docs from that guy) and I don't feel very reasonable all the complains from that one. Often sysadmins are complaining with piece of software just because they don't want to learn how to configure them and still more often they are forced to install them by their boss who just wants the cheapest solution without any investment into resources (which can be trainings, support, expertise, etc). I know this is philosophy but it's also the reality.</div><div><br></div><div>Come back to our plan for the new organization of docker artifacts I had understood from the GNIP and all the discussions that there was consensus to:</div><div><br></div><div>1. Still support the current working use cases (docker native and docker-machine both out of the box/Windows, Mac Os, Linux) regardless whether to keep the current approach with magic ip stuff or not </div><div>2. Converge all the current repositories (*-docker) from GeoNode organization into a new central one</div><div>3. Converge all the goods which can make the deployment more robust and reliable (as you explained in the GNIP) from the SPCgeonode repository</div><div>4. Achieve a CI/CD strategy for docker deployment which can make developers aware when they are gone to break it</div><div><br></div><div>I'd rather we did move 4. between 2. and 3. so we could make sure the deployment would be ever safe while we move forward it with the improvements.</div><div><br></div><div>Cheers,</div><div>Francesco</div><div><br><div class="gmail_quote"><div dir="ltr">Il giorno ven 7 set 2018 alle ore 05:13 Olivier Dalang <<a href="mailto:olivier.dalang@gmail.com">olivier.dalang@gmail.com</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>@Francesco<br></div><div><br></div><div>Indeed, this does not necessarily need to be tied with next release. But I think it is urgent still. We have a lot of problems with the current deployment method (well described in <a href="https://github.com/GeoNode/geonode/issues/3722" target="_blank">#3722</a> ) and it seems with the resources we have, it won't get better over time.</div><div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>It's likely in the next couple of months I can
dedicate a bit of time to achieve this which is my top priority at the
moment for docker deployments. Obviously your or somebody's else help
would be very useful if you are available.</div></blockquote>
</div><div><br></div><div>We have a <a href="https://github.com/GeoNode/geonode/issues/3707" target="_blank">GNIP</a> about this, had lots of discussions including a skype call with a lot of us, and the consensus was basically to wait until the current PSC to validate the GNIP and see how we move on. If you're currently working on docker for deployment, it would be good not to ignore those completely, especially if you're hoping for help or collaboration.<br></div><div><br></div><div>
<div>As for why I suggest to use the SPCgeonode setup to
start with, it's just that it's relatively close to tick the boxes that
we discussed in the GNIP, and that it could realistically reach beta
state very soon.</div>
</div><div><br></div><div>My idea is not to impose my solution, hence the 2 steps proposition with first step outside of the main repo. Once we've solved the urging need of having another more sustainable deployment solution, we can slowly work to merge the implementations in the main repo.<br></div><div><br></div><div>@Toni</div><div><br></div><div>It would be for the name of the repository. SPCgeonode wouldn't really make sense outside of my organization.</div><div>Maybe something like GeoNode/docker-deployment...</div><div><br></div><div>Cheers,</div><div><br></div><div>Olivier<br></div><div><br></div><div><br></div><div><br></div><div><br><div class="gmail_quote"><div dir="ltr">Le jeu. 6 sept. 2018 à 22:22, francesco bartoli <<a href="mailto:xbartolone@gmail.com" target="_blank">xbartolone@gmail.com</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Olivier,<div><br></div><div>I would hold this one until the next meeting and vote your proposal with the two phases. Personally I don't like to have 2 parallel deployment methods and even regarding 1/ I don't see the urgency to have this additional intermediate step for the release. I have the feeling that this would be confusing for the users.</div><div><br></div><div>If we want to take on something which is one of the most important issue than I would try to setup a CI for docker deployments which can be then still used when we'll achieve the step 2/ imho.</div><div>I'll give you an example: you reported the issue regarding allowed_hosts for a Windows environment but yesterday I realized together with Alessio that actually it was a regression affecting all the deployments regardless of the OS and nobody was aware...</div><div><br></div><div>It's likely in the next couple of months I can dedicate a bit of time to achieve this which is my top priority at the moment for docker deployments. Obviously your or somebody's else help would be very useful if you are available.</div><div><br></div><div>Cheers,</div><div>Francesco </div></div><div dir="ltr"><div><br><br><div class="gmail_quote"><div dir="ltr">Il giorno gio 6 set 2018 alle ore 03:49 Olivier Dalang <<a href="mailto:olivier.dalang@gmail.com" target="_blank">olivier.dalang@gmail.com</a>> ha scritto:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div dir="ltr"><div>Hi all !</div><div><br></div><div>Some time has passed since we last discussed deployment with Docker.
I'm still willing to work on officially supporting docker deployment for Geonode.
</div><div></div><div><br></div><div>
To move forward on the docker deployment, I suggest a two phase strategy.</div><div><br></div><div>1/ copy over the SPCgeonode repo to the geonode organization (more or less as is) and document that deployment method as an official one
l (but in beta state). We'd need to find a good name for it - suggestions welcome.
</div><div>2/ work on making the SPCgeonode setup and the setup in the main repo converge, with the end goal of removing the docker deployment repo altogether and have everything in the main repo, using the same docker-compose, with a docker-compose.deploy.yml that adds deployment specific elements (https, backups)</div><div><br></div></div><div dir="ltr"><div>1/ could happen very soon (maybe even before next release), while 2/ would take more time, as it requires to go carefully through both implementations and take the best of both.</div></div><div dir="ltr"><br></div><div>I think this would accommodate both the goal of having a ready deployment method soon,
and the final goal of having the deployment method use the same docker
images than dev, without being disruptive for users currently relying on the docker images.<br></div><div><br></div><div>The reason why I think we must do it in 2 steps is that IMO, while good for development, the current setup in the main repo still requires significant work until it's usable as a
base for deployment (docker images are too heavy, split across many repos, images not properly versioned, non-production ready elements such as static files being served by django, no ci, etc.). That's not mentioning the additional steps needed for a deployment meeting basic good practice requirements (https, backups...).</div><div><br></div><div>I would definitely be able to support the work on this as part of my current activities. I also believe it meets a high demand from the community (this is more or less my only serious contribution to Geonode, so that I take my election in PSC a clear sign of that demand). <br></div><div><div><br></div><div>Francesco, would that 2 steps strategy be OK as you
were keen to have the spcgeonode setup reuse as much as possible
existing docker setups ?</div>
</div><div><br></div><div>Let me know what you think. Let's discuss this, and if needed vote during next PSC (though if we reach consensus before that it's even better).</div><div dir="ltr"><div><br></div><div>Cheers,</div><div><br></div><div>Olivier<br></div><div dir="ltr"><br><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Apr 20, 2018 at 1:40 AM, Ariel Nunez <span dir="ltr"><<a href="mailto:ariel@terranodo.io" target="_blank">ariel@terranodo.io</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Olivier,<br><br>I suscribe to the vision of geonode as a library and that a geonode-project should be a simple django project that does not import anything from geonode.settings. The reason for this is that geonode becomes a starting point and what people start to build are their own projects.<br><br>I hope at some point in the near future we deprecate geonode.settings at all and instead let an ecosystem of different starter projects built on the same docker base images flourish.<span class="m_8796136840773523281m_8304454529060246799m_-7802435813222838024m_-8841901270468820228m_2500548522515209897m_1799600292067883780m_-1550389143515942797m_-2751589179940224409m_7327341835245374799m_-915754548127605620m_8655918908424057203m_6525704897634662602m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668HOEnZb"><font color="#888888"><br><br>-a</font></span><div class="m_8796136840773523281m_8304454529060246799m_-7802435813222838024m_-8841901270468820228m_2500548522515209897m_1799600292067883780m_-1550389143515942797m_-2751589179940224409m_7327341835245374799m_-915754548127605620m_8655918908424057203m_6525704897634662602m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668HOEnZb"><div class="m_8796136840773523281m_8304454529060246799m_-7802435813222838024m_-8841901270468820228m_2500548522515209897m_1799600292067883780m_-1550389143515942797m_-2751589179940224409m_7327341835245374799m_-915754548127605620m_8655918908424057203m_6525704897634662602m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668h5"><br><div class="gmail_quote"><div dir="ltr">On Thu, Apr 19, 2018 at 5:52 AM Olivier Dalang <<a href="mailto:olivier.dalang@gmail.com" target="_blank">olivier.dalang@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="auto"><div dir="ltr"><div>Hello all !</div><div><br></div><div>Thank you all for the great discussion !! I very much appreciated how much you all took into account my inputs and I felt part of the discussion despite being deeply asleep :-) <span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Thank you very much for the recording Cristiano !</span></div><div><br></div><div><br></div><div>I think there are two different needs that were discussed, the need for stable docker images (typically for providers to build their custom stacks on reliable/secure images), and the need for a supported deployment through docker-compose (typically for smaller users without skills/ressources to do a proper deployment on their own).</div><div><br></div><div>So far I focused only on the docker-compose for deployment (as it's the case we need to support here).</div><div>
<br class="m_8796136840773523281m_8304454529060246799m_-7802435813222838024m_-8841901270468820228m_2500548522515209897m_1799600292067883780m_-1550389143515942797m_-2751589179940224409m_7327341835245374799m_-915754548127605620m_8655918908424057203m_6525704897634662602m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668m_4549730285531974548m_-2510610479178620297m_8234380312452560400m_7216416946810614236gmail-m_765127961121936531gmail-Apple-interchange-newline"><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">But I agree having those stable images would be great too. I think that even if the needs are different, in the end they may very well align in one setup, where the docker-compose setup for production could even serve as an integration test of the individual docker files.</span>
<br></div><div><br></div><div>It's not completely clear to me what the scope of those images would be. I think at the very least, they would include all dependencies (system and python).</div><div>Would they also include build steps (instead of pulling external builds from <a href="http://builds.geonode.org" rel="noreferrer noreferrer" target="_blank">builds.geonode.org</a>) ? (may be good in terms of build stability and make it easier to contribute, also with multistage dockerfiles it's quite easy to keep images lightweight despite having the whole build process in there).</div><div>Would they be though as final images (that you use as is, just by configuring env vars), or base images (that are supposed to serve as base to user's custom dockerfiles in a composition) ?</div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">Also it will be hard to find the sweet spot between <span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">too little opinionated (it would become a pain to configure and loose stability) and too much (you wouldn't be able to use it anymore in a custom stack).</span><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span> </span></span>
</span></div><div><br></div><div><br></div><div>I had another thought about docker-compose that I don't think was raised : with the current geonode-project approach, there will be some duplication, because there is already another docker-compose file in Geonode's main repo (for development and testing of geonode itself - which is obviously also needed).</div><div><br></div><div>Could it be that the docker-compose setup for deployment is built on top of the docker-compose for development ? In the end, the geonode repo already contains a django project. It's just a matter of being able to override the settings. I'm not sure if this would end up being overly complicated for nothing, or if it would be a win because of increased similarity between dev setup and deployment setup. Going that way would be a step towards a continuous delivery model (a big incentive for devs to upstream their features). <span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I also believe that with Alessio's recent commits for UI customization, around 50% of the use cases for geonode-project just vanished (thanks for that great feature :-) ).</span>
</div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">I also agree with Tim on the idea of supporting both the QGIS and Geoserver backends. I'm not familiar enough with the differences that implies so far to tell how much work that would be.</span></div><div><br></div><div><br></div><div>And about the smaller scale technical questions :</div><div><br></div><div>- Containers restarting VS. wait-for-postgres.sh : I have no strong opinion about that. It's true that it would look a little bit more elegant at first startup, but it also adds a little bit of complexity in the entrypoint, and we must make sure we don't create artificial blockages (e.g. waiting for geoserver would be a bad idea, as we should be able to start even if geoserver is down).</div><div><br></div><div>- Additional requirements (pip) : I'll try to make a PR to upstream those fixes including some other to make pip install more stable, as discussed here : <a href="https://github.com/GeoNode/geonode/issues/3227" rel="noreferrer noreferrer" target="_blank">https://github.com/GeoNode/geonode/issues/3227</a></div><div><br></div><div>- A<span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">bout the more minimal docker-compose (without letsencrypt and other optional components), so far I use<span> </span></span><font size="2" color="#222222"><span style="background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">docker-compose up nginx django geoserver postgres</span></font><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><span> </span>for that purpose, but we could indeed also use docker-compose overrides to achieve similar result. I would vote for the option that looks the most clear in the command line.</span></div><div dir="auto"><span style="text-align:start;text-indent:0px;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><br></span></div><div><br></div><div>I'll continue to push my setup a bit further anyway (to 2.8) and agree with idea of another talk at some point.</div><div dir="auto"><br></div><div dir="auto">Thanks again!</div><div><br></div><div>Cheers,</div><div><br></div><div>Olivier</div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Apr 19, 2018 at 2:45 AM, Cristiano Giovando <span dir="ltr"><<a href="mailto:giovand@gmail.com" rel="noreferrer noreferrer" target="_blank">giovand@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Thanks to everyone who joined the docker chat today - here's the<br>
recording in case you missed it:<br>
<br>
<a href="https://drive.google.com/open?id=1voxdexJQLPdikLAW6hLsXk7TwYkv9KJT" rel="noreferrer noreferrer noreferrer" target="_blank">https://drive.google.com/open?id=1voxdexJQLPdikLAW6hLsXk7TwYkv9KJT</a><br>
<br>
Please share any feedback here or directly in the GNIP:<br>
<a href="https://github.com/GeoNode/geonode/issues/3707" rel="noreferrer noreferrer noreferrer" target="_blank">https://github.com/GeoNode/geonode/issues/3707</a><br>
<br>
Looking forward to a successful and coordinated approach! ;)<br>
<div><div class="m_8796136840773523281m_8304454529060246799m_-7802435813222838024m_-8841901270468820228m_2500548522515209897m_1799600292067883780m_-1550389143515942797m_-2751589179940224409m_7327341835245374799m_-915754548127605620m_8655918908424057203m_6525704897634662602m_-3823682970190089128m_-1038740136096827641m_-1136954947812398580m_1084826330633004668m_4549730285531974548m_-2510610479178620297m_8234380312452560400m_7216416946810614236h5"><br>
On Tue, Apr 17, 2018 at 9:44 PM, Cristiano Giovando <<a href="mailto:giovand@gmail.com" rel="noreferrer noreferrer" target="_blank">giovand@gmail.com</a>> wrote:<br>
> Hey all,<br>
><br>
> Following some good conversations around the topic at the summit and the<br>
> recent GNIP by Olivier, we're planning a call tomorrow at 15:00 CEST / 13:00<br>
> UTC to chat about a common strategy. We're inviting anything who's<br>
> interested to join.<br>
><br>
> Here's the GNIP for reference:<br>
> <a href="https://github.com/GeoNode/geonode/issues/3707" rel="noreferrer noreferrer noreferrer" target="_blank">https://github.com/GeoNode/geonode/issues/3707</a><br>
><br>
> Meeting link:<br>
> <a href="https://worldbankgroup.webex.com/worldbankgroup/j.php?MTID=mab1ab3faf3942146706111bc9eaad51f" rel="noreferrer noreferrer noreferrer" target="_blank">https://worldbankgroup.webex.com/worldbankgroup/j.php?MTID=mab1ab3faf3942146706111bc9eaad51f</a><br>
><br>
> Join by phone<br>
> 1-650-479-3207 Call-in toll number (US/Canada)<br>
> Access code: 733 296 941<br>
><br>
> Global call-in numbers:<br>
> <a href="https://worldbankgroup.webex.com/cmp3200/webcomponents/widget/globalcallin/globalcallin.do?siteurl=worldbankgroup&serviceType=MC&ED=529265077&tollFree=0" rel="noreferrer noreferrer noreferrer" target="_blank">https://worldbankgroup.webex.com/cmp3200/webcomponents/widget/globalcallin/globalcallin.do?siteurl=worldbankgroup&serviceType=MC&ED=529265077&tollFree=0</a><br>
><br>
> See you tomorrow!<br>
><br>
> Cristiano<br>
</div></div>_______________________________________________<br>
geonode-devel mailing list<br>
<a href="mailto:geonode-devel@lists.osgeo.org" rel="noreferrer noreferrer" target="_blank">geonode-devel@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/geonode-devel" rel="noreferrer noreferrer noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/geonode-devel</a><br>
</blockquote></div><br></div>
</blockquote></div>
</div></div></blockquote></div><br></div></div><div dir="ltr"><div><div><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div class="m_8796136840773523281m_8304454529060246799m_-7802435813222838024m_-8841901270468820228m_2500548522515209897m_1799600292067883780m_-1550389143515942797m_-2751589179940224409m_7327341835245374799m_-915754548127605620m_8655918908424057203h5"><div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
</blockquote></div></div></div></div></div>
</blockquote></div></div></div></div></div></div></div>
</blockquote></div></div></div></blockquote></div></div></div>
</blockquote></div></div></div>