<div dir="ltr"><br><div class="gmail_extra">On Thu, Jul 20, 2017 at 9:29 AM, Matthias Kuhn <span dir="ltr"><<a href="mailto:matthias@opengis.ch" target="_blank">matthias@opengis.ch</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div text="#000000" bgcolor="#FFFFFF"><span class="">
<p>On 7/20/17 8:56 AM, Alessandro Pasotti wrote:<br>
</p>
<blockquote type="cite">
<div dir="ltr">That's very interesting, Alessandro<br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> <br>
How are dependencies built, using some packaging system
or a set of scripts? Are the scripts or recipes around
somewhere?<br>
</div>
</blockquote>
<div><br>
</div>
<div>Oh, sorry for not being clear, we are not building
dependencies but only QGIS itself, but I think that the
general workflow could be used for building the
dependencies in the docker instead of QGIS and push the
image to docker HUB.<br>
</div>
</div>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>So, when Travis starts, it will pull the pre-built
docker image with all dependencies already built and
either install them in Travis and build QGIS or build QGIS
inside the docker "inside" Travis.<br>
</div>
</div>
</div>
</div>
</blockquote>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<div>BTW, the recipes are here: <a href="https://github.com/boundlessgeo/qgis-testing-environment-docker" target="_blank">https://github.com/<wbr>boundlessgeo/qgis-testing-<wbr>environment-docker</a><br>
</div>
</div>
</div>
</div>
</blockquote>
<br></span>
Thanks for the pointer.<br>
So you are mostly using the dependencies directly from Ubuntu 16.04
except for one dependency that you build?<span class=""><br></span></div></blockquote><div><br></div><div><br>Yes, the purpose of that docker is provide a testing environment for plugins, in that context, QGIS is "the" dependency that is being built.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><span class="">
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> <br>
We were wondering if this could be done directly on
travis before the build starts (skipping any package
that is already up to date in the cache).<br>
</div>
</blockquote>
<div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> The nice benefit of this compared
to a separate build as you are doing on AWS would be
that if something needs a new dependency (thinking of
qca, qtkeychain, ...) it can directly integrated in the
same pull request.<br>
What do you think about this approach?<br>
<br>
Thanks<span class="m_7774211359815247136gmail-HOEnZb"><font color="#888888"><br>
Matthias</font></span><span class="m_7774211359815247136gmail-"><br>
<br>
</span></div>
</blockquote>
<div><br>
<br>
<div>I'm not sure I get it right, do you mean that you
want to build the dependencies on Travis within the same
job that builds QGIS?<br>
</div>
</div>
</div>
</div>
</div>
</blockquote></span>
Correct<span class=""><br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div><br>
</div>
<div>I think that this would add a considerable time,
that's why we are building on AWS and pull the pre-built
docker, but I guess that the goal is completely
different.<br>
</div>
</div>
</div>
</div>
</div>
</blockquote></span>
Normally they should be cached and not rebuilt.<span class=""><br></span></div></blockquote><div><br></div><div><br>Oh, I see, I would not know how to do that though. <br><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF"><span class="">
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>
<div>I was thinking that the dependencies do not vary
frequently, hence we should be able to build the
dependencies docker nightly and when the main QGIS
Travis job starts we can pull the docker with the
dependencies substantially lowering the time needed to
run the entire job.<br>
</div>
</div>
</div>
</div>
</div>
</blockquote></span>
I imagine version updates happen more often. For example a new gdal
version that brings in some geopackage functionality which is
covered by a unit test.<br></div></blockquote><div><br></div><div>More often than daily? I was thinking at building the dependencies in the docker daily with AWS. We could even have different set of dependencies in different docker tags and use a Travis matrix to test them all ... given that it does not add too much to the Travis allowed time.<br> <br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div text="#000000" bgcolor="#FFFFFF">
<br>
What's the workflow here?<br>
<br>
If this can be added directly inside a pull request this has some
advantages like<br>
<br>
a) responsibility (even without commit rights on the qgis repo you
can build the deps in the pull request)<br>
b) sandboxing (if the library is updated in a centralized repository
and the new gdal version kills some other unit tests we will have
that also failing on master).<span class=""><br>
<br></span></div></blockquote><div><br></div><div>That's a big advantage, I agree.<br><br></div><div>The bottomline is that if you know how to do that and it will work without timing out, your solutions is for sure the best one.<br><br></div><div> <br></div><div>Cheers<br clear="all"></div></div><br>-- <br><div class="gmail_signature" data-smartmail="gmail_signature">Alessandro Pasotti<br>w3: <a href="http://www.itopen.it" target="_blank">www.itopen.it</a></div>
</div></div>