<div dir="ltr">Hi All,<div><br></div><div>For me at least, maintaining working dev environments for GDAL can be frustrating... there are a lot of dependencies, platforms, and a huge number of config options. Let alone switching branches for backports/etc. I typically use OSX as a desktop and linux environments running under that via Docker & Virtualbox.</div><div><br></div><div>GDAL has really comprehensive CI, but I wonder if there's ways to improve local dev & testing workflows? There is a Vagrant image from a while back, but it runs Ubuntu Precise which is well EOL, so I assume it's not in regular use. </div><div><br></div><div>What are other people doing? Does anyone have an elegant dev environment they could share?</div><div><br></div><div>Suggestion:</div><div><br></div><div>1. docker images with all the build/test/library dependencies already installed — publish them so getting the right dependencies & environment is only a download.</div><div><br></div><div>2. docker-compose files with ccache and source trees mapped to the host checkout. This should make builds & rebuilds comprehensive, and as quick as possible (but won't be as fast as native builds). Could also be wrappers around docker run.</div><div><br></div><div>I made a very rough proof of concept out of the trusty_clang travis scripts...</div><div><br></div><div><a href="https://github.com/koordinates/gdal/tree/docker-dev-poc/gdal/ci/docker">https://github.com/koordinates/gdal/tree/docker-dev-poc/gdal/ci/docker</a><br></div><div><br></div><div><font face="monospace, monospace"># build the base image with the dependencies</font></div><div><font face="monospace, monospace">$ docker-compose -f gdal/ci/docker/docker-compose.yml build<br></font></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"># start the dev environment</font></span></div><div><font face="monospace, monospace"><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">$ docker-compose -f gdal/ci/docker/docker-compose.yml up -d</span></font></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"><br></font></span></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"># repeat this bit as you develop</font></span></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace"><div style="text-decoration-style:initial;text-decoration-color:initial"><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">  # use the CI steps</span></div><div style="text-decoration-style:initial;text-decoration-color:initial"><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">  $ docker-compose -f <span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">gdal/ci/docker/docker-compose.yml exec /ci.sh all  # build+test</span></span></div><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">  $ docker-compose -f </span><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">gdal/ci/docker/docker-compose.yml exec /ci.sh build</span><br class="gmail-Apple-interchange-newline"><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">  $ docker-compose -f </span><span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">gdal/ci/docker/docker-compose.yml exec /ci.sh test</span><br class="gmail-Apple-interchange-newline"><br class="gmail-Apple-interchange-newline">  # or do it manually<br></font></span></div><div><span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline"><font face="monospace, monospace">  $ docker-compose -f <span style="text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">gdal/ci/docker/docker-compose.yml exec bash</span></font></span></div><div><br></div><div><font face="monospace, monospace"># tear it all down</font></div><div><font face="monospace, monospace">$ docker-compose <span style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">-f </span><span style="font-size:small;text-decoration-style:initial;text-decoration-color:initial;float:none;display:inline">gdal/ci/docker/docker-compose.yml down</span></font></div><div><br></div><div>Thoughts? Comments?</div><div><br></div><div>Without other steering, I guess the next steps would be:</div><div><br></div><div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">- do it for the best/cleanest/modern/featureful environment & compiler rather than the venerable Trusty (which one?)</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">- don't run as root, make sure steps are fully repeatable, etc</div><div style="font-size:small;text-decoration-style:initial;text-decoration-color:initial">- ideally a bunch of the stuff in the existing ci .sh files would be in Makefiles or elsewhere I think?<br></div></div><div><div style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">- have a script that runs configure/etc with all the options + dependencies that will work in the environment. (ie. extract more of install.sh)</div></div><div style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">- keep the .o files in the container instance rather than the source tree? <a href="http://make.mad-scientist.net/papers/multi-architecture-builds/">http://make.mad-scientist.net/papers/multi-architecture-builds/</a> via some autotools voodoo?</div><div style="font-size:small;background-color:rgb(255,255,255);text-decoration-style:initial;text-decoration-color:initial">- make it easier to extend for supporting other environments</div><div><br></div><div>Cheers,</div><div><br></div><div>Rob :)</div><div><br></div><div>
</div></div>