[OSGeo-Discuss] Status report for April - System Contract I 2020

[Regina Obe]

Below is list of things I've been involed with in April revolving around
core infrastructure

1) Rebuilding Wiki.  I built a new debian 10 container with latest
Wikimedia, PHP, apache installed  -- but still exploring the db and stuff to
create a script to chuck bad user accounts.
I also need to retest the data restore script I have setup and revisit some
plugins that did not come thru (no upgrade path for them).

https://wiki.new.osgeo.org  (this is running on OSGeo3)

I'm hoping to do the final migration sometime late May.

Once this is done we can setup a dev of this new version to do the LDAP
integration.  This we can contract out -- we have two proposals from outside
contractors for LDAP work.

The below services are new/updated ones that use OSGeo LDAP for
authentication

2) Server Monitoring -- To replace the old munin monitoring, I put in place
Prometheus / Grafana  (https://prometheus.io/docs/visualization/grafana/  )
(this is so I can address recent complaints about speed etc and know when a
container needs more resources)

https://prometheus.io/  - Prometheus is a monitoring tool with already a lot
of what they call "Exporters" available for it, most of the exporters are
written in Go. Some are specifically geared for specific applications and
seems like perhaps not much effort to write our own if needed.  Since Go is
statically compiled these work on old servers as well and can be installed
by just copying the binaries and setting up the service script.

Exporters run as services on each Container / VM.  I haven't installed any
of these on osgeo6 or osgeo5, but have many installed on the new LXD
containers
Exporter service scripts I have committed to this repo -
https://git.osgeo.org/gitea/sac/prometheus-config  (and have collected the
node exporter (for OS monitoring) and nginx exporter (for nginx monitoring)

The nginx monitoring is installed on all the nginx proxies (OSGeo7, OSGeo3,
OSGeo4) and download container

There are 3 prometheus servers  (I have 3 instead of 1 to ease network
management since the Prometheus servers are all on the same private network
of the containers they pull metrics from they don't need any additional
whitelisting)
OSGeo7 - nginx (collects all the metrics from the OSGeo7 container
exporters)
OSGeo3 -  monitor (also runs Grafana and collects all metrics from OSGeo3
containers)
OSGeo4 - osgeo4-nginx (collects metrics from OSGeo4 containers)

Grafana is running on OSGeo3 monitor container -> https://monitor.osgeo.org
(all OSGeo LDAP users can log in to see the metrics).
I whitelisted it on OSGeo7 and OSGeo4 so it can query the Prometheus servers
for monitor stats.


3) https://repo.osgeo.org -- Jody Garnett spear-headed this effort.  The
Repo service is running Nexus repository management.
Feel free to login and check it out and if you want to use it in some way
for your projects -- Please put in a trac ticket -
https://trac.osgeo.org/osgeo/ 
Nexus supports the following kinds of repos - Maven, Docker Registry, Apt,
Yum, Nuget, R CPAN, RubyGems, npm (and some other stuff I've never heard of)

At the moment the following projects are using it (GeoTools, GeoServer,
GeoNetwork -- for their maven repo), (PostGIS / GEOS to manage docker
containers we use for drone bot regression testing)

4) https://matrix.osgeo.org -- container running Matrix (synapse server
https://matrix.org/docs/guides/installing-synapse )  bridges with our
existing IRC channels and can be used for private chat rooms like GSoc ones
for example
Requires an OSGeo LDAP account to use.  Talk to Sandro Santilli if you have
questions about how to use it.

5) nextcloud.osgeo.org - upgraded to 18.03  (Hub) (from 15)
https://nextcloud.com/hub/ .  There was some discussion about coediting
documents being slow from I think QGIS group.  Our version is running
Collabora for (Libre Office / MS Office online document editing)
  I am debating if we should switch to the Community server version which
uses  OnlyOffice for document editing / collaboration (which I have
installed on experimental https://nextcloud.gallery.osgeo.org )
Waiting for feedback from QGIS PSC group to see if it's worthwhile to
switch. From a cursory play with both, the OnlyOffice seems faster (but I
have no one to collaborate with to test the group editing features). 
As far as editing goes
Collabora seems to be better for LibreOffice , I had issues trying to
upgrade it though in that the PDF export no longer worked so had to revert
back.
OnlyOffice seems to be better for Microsoft documents (Word, Excel,
Powerpoint)  (it screwed up one of my libre .odt docs, and also doesn't have
ability to view LibreOffice drawings from what I can tell)

Thanks,
Regina