[Board] Travis-CI & OSGeo

[Even Rouault]

On lundi 23 avril 2018 11:01:34 CEST Sandro Santilli wrote:
> On Fri, Apr 20, 2018 at 05:46:12PM -0400, Michael Smith wrote:
> > I don’t see any reason why we wouldn’t want to provide this for OSGeo
> > projects.
> I do: we'd be an OpenSource Software Foundation paying a ClosedSource
> Software Company to keep being successful in their business model.

The aim here would be that *OSGeo projects* are successful in their developments
needs. The fact that a commercial company makes profit from that is a side effect,
not the primary goal.

Interestingly the Apache foundation has 1538 repositories listed under
https://github.com/apache
I've just found this interesting post:
https://blogs.apache.org/infra/entry/apache_gains_additional_travis_ci
where they mention they have subscribed for 30 concurrent jobs.

Given the current use of github.com/OSGeo, 10 would be enough for now IMHO.

> 
> Compare that to giving money to community members willing to provide
> build agents to the existing OSGeo build infrastructure (drone.osgeo.org).

According to the specs displayed at
https://docs.travis-ci.com/user/reference/overview/#Virtualization-environments
the Ubuntu Precise workers must probably run on a n1-standard-2 GCE machine
According to https://cloud.google.com/compute/pricing , the monthly price for such
a pre-emptive VM is $14.60
If you have '10 concurrent jobs' it means that you can have in theory a need for
10 such VMs. So 10 VMs for a year would be 14.60 * 12 * 10 = $1752 / year
which is not so far from the $1925/year of their quote.

Of course in practice the CI company must have some good statistics on the average
load actually used by a project, and thus it knows it does not need to rent as
many VMs, and still have some money to pay for their employees to monitor
this complex beast.

https://drone.io/pricing/ is $129 * 12 = $1548 / year for 2 concurrent builds, so
for 10, naively that would be $7740 

Owing your own server(s) is now a luxury, which is rather costly and unefficient,
given that they are either under-loaded or over-loaded, but rarely utilized at their
full capacity.

I also hear your argument that there are solutions like Drone or GitLab that use
open source software, which is great. But at some point you need to make that
software run on servers, that you must buy or rent, and that you must monitor, maintain, etc.

Regarding Regina's
"First of all if this is something GDAL needs and Proj needs, then it should
come out of GDAL's/ Proj's budget "
This is a good point, and a reasonable solution. OSGeo would need to know which
projects are hosted under github.com/OSGeo and use Travis-CI to be able to select
the appropriate number of workers (which is a bit tricky since the actual use of the
CI infrastructure depends very much on the project activity)

"If you each had your own project group on github you'd each have 5 workers
end of story."
1) I wasn't aware of the per-org 5 job limitation until very recently
2) https://github.com/gdal /proj /proj4 are already occupied.

"You wouldn't even need to waste funding on this."

That's the kind of workaround/cheating which causes others to pay for resources you
use. And thus increase their bills... and/or may decrease the quality of the free tier.

I'm not against considering other CI solutions that integrate will with github such as
https://docs.gitlab.com/ee/ci/ci_cd_for_external_repos/github_integration.html
that Sandro mentionned to me, although I can't find information about what the
limitations are regarding the number of concurrent jobs, and if there would be
per project or organization.
And there would be "some" labor cost to migrate the existing 12 Travis-CI
configurations currently used by GDAL to that
( https://travis-ci.org/OSGeo/gdal/builds/370031450 )

By the way, who has admin rights on
https://gitlab.com/osgeo
and grant me appropriate rights so I can have a try at setting up the
gitlab-ci-for-github thing for GDAL ?

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/board/attachments/20180423/9880f0d3/attachment.htm>