[SAC] Server Monitoring (was: Status report for April - System Contract I 2020)

[Sandro Santilli]

On Sat, May 02, 2020 at 11:46:54PM -0400, Regina Obe wrote:

> 2) Server Monitoring -- To replace the old munin monitoring, I put in place

> The nginx monitoring is installed on all the nginx proxies (OSGeo7, OSGeo3,
> OSGeo4) and download container
> 
> There are 3 prometheus servers  (I have 3 instead of 1 to ease network
> management since the Prometheus servers are all on the same private network
> of the containers they pull metrics from they don't need any additional
> whitelisting)
> OSGeo7 - nginx (collects all the metrics from the OSGeo7 container
> exporters)
> OSGeo3 -  monitor (also runs Grafana and collects all metrics from OSGeo3
> containers)
> OSGeo4 - osgeo4-nginx (collects metrics from OSGeo4 containers)
> 
> Grafana is running on OSGeo3 monitor container -> https://monitor.osgeo.org
> (all OSGeo LDAP users can log in to see the metrics).
> I whitelisted it on OSGeo7 and OSGeo4 so it can query the Prometheus servers
> for monitor stats.

Where am I supposed to find the graphs of nginx metrix ?
When going to monitor.osgeo.org, and logging in via OSGeo UserID
credentials, and picking "Home" menu item from the "Dashboard"
menu entry (the 4 squares icon), I only see:

  - "Grafana metrics"
  - "Prometheus 2.0 Stats"

Clicking on each of them shows me a few graphs, but all empty (No
Data) except from one under the "Promoetheus 2.0 Stats" and being
called "Scrape Duration". That graph shows 3 lines:

  - osgeo4-prometheus
  - osgeo4_nginx_os_exporter
  - osgeo4_nginx_web_exported

Where are the actual stats ?
Why can't I see them, if they are present ?

--strk;