[Board] Travis-CI & OSGeo

Tue May 1 22:00:18 PDT 2018

Even,

I'm going to say this to save people some time.  I don't think Jenkins is
right for GDAL project or at least not yet. It might be useful to augment
GDAL testing,

but I wouldn't expect GDAL to give up using travis.  So the only question is
if you are being seriously inconvenienced by the lack of workers right now
or is it something you are willing to wait a month or 2 to revisit.

If you feel it's holding you back, then by all means yes OSGeo should pay
for it , and maybe add it as a GDAL/Proj.4 budget item.

My concern with having brought it up is my fear that people will start
thinking they should get into the github/OSGeo.org group and then the
pricing becomes exorbitant and starts straining other funds.

I can say for me, I am not happy with having just travis, I want to use
other tools gitlab, Jenkins, etc. in addition to travis.  I enjoy tinkering
with configuring software just as much as I like programming. I think others
feel the same way that travis is insufficient given what people have said
and that has surprised me, as I thought I was alone feeling travis was
inadequate.

I also think as Mike Ternier says whether we should be in the business of
dictating what projects can and can't use.  I definitely don't want to be in
that business.

Each project should have a budget they can use for what they think they
need, and be allowed to pool their budgets with projects that have similar
needs.

> On mardi 1 mai 2018 12:22:26 CEST Howard Butler wrote:

>> On 5/1/18 2:39 AM, Regina Obe wrote:

>> > I'm sure Hobu will correct my ignorance and rave about all the goodness
of

>> > Travis :)

>> My point(s) about Travis are not about its technical non-superiority.

> I would not be able to articulate things more clearly than Howard did and
due to his experience on this very precise topic, the value of his testimony
should be considered with care.

Yes Howard's testimony was very moving and I am thinking about it earnestly.

> Due to lack of recent experience with Jenkins, I can't really comment if
it is appropriate for the task. I think it can trigger builds on pull
requests, can't it ? 

               Yes

> What about configuration of builds: is there the equivalent of a config
file hosted in the source code repo (ala .travis.yml) to which any
contributor can contribute changes to the CI scripts:

>  fix them, add new configurations, etc ? If it is a task that can only be
done by a select administrator of each project, then we add extra burden on
its shoulders.

Yes it can via a Jenkins file and that is what we would need to do if we
start growing Jenkins yes.  I admit to not having experience using
JenkinsFile pipeline approach. That's the other reason I was thinking one
instance might not be sufficient as people works with Jenkins a little
differently   That said the Jenkins file structure is a very different from
travis and gitlab, so travis and gitlab I feel are probably more comparable.
It can also use docker images

https://jenkins.io/doc/book/pipeline/syntax/

> But the above is just details compared to a more fundamental difference
when comparing Jenkins to Travis-CI/CircleCI/GitLab-CI/etc... Jenkins is
just a software. Which machines would run it ? 

> Has OSGeo a stock of 20 unused cores somewhere to support 10 concurrent
builds ?

               I'm not absolutely sure how many cores we have to spare.  We
just ordered a new server, and much of the stuff on our old servers will be
moved to this which would free up those machines.  Funtoo has also provided
us hardware – I forget the details of how many cores that is.  

I was also thinking that now companies like Amazon, Microsoft Azure, Google
Cloud are making serious money from selling things like PostGIS. With the
right recipe for coaxing  we can get hardware and bandwidth since our work
will make their product stronger. Since PostGIS relies on GDAL, well it's in
their best interest to make sure GDAL has massive amounts of testing.  We
need someone with good coaxing skills.  I'm looking at you Mike Ternier :).

>  I don't really believe either in solutions where people bring their
machines. I think that was how buildbot worked with the build slaves, right
?

I have my reservations on that too.

> My mention of having sufficiently hardware power is not just a theoretical
concern: ironically a lot of flakes I've seen with GDAL and Travis were due
to the GDAL autotest suite doing the mundane task of downloading (small)
files from download.osgeo.org

> 

 and the server being unresponsive due to probably being overloaded by other
tasks. I've finally modified those tests to pull from rawcontent.github.com
and those flakes have gone away.

That's a bit of a concern there and something we should investigate.  Sounds
more like a bandwidth issue than hardware.

> And even if you have the appropriate software + the appropriate machines,
the crucial human factor as raised by Howard remains. I feel my time is
better spent at taking care at the sofware I'm 

> knowleadgable of rather than at doing system administration/maintainance
work for which I've little expertise/interest, and which implies to be
available at the moment when things break.

> Even

I wouldn't have it any other way – keep GDAL going and focus on that.

Thanks,

Regina

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/board/attachments/20180502/9d7297d4/attachment.htm>