Accessing metrics on your own analytics platform
This page explains how to view SpatialOS metrics within your own in-house or third-party analytics platform, using the monitoring proxy that lets you access your deployment’s metrics.
In your analytics platform, you’ll need to:
- Add Prometheus and Elasticsearch as data sources.
- Add the monitoring proxy URL.
- Enter the HTTP basic authentication username and password described below.
Keeping your password secure
- Follow best practice for the solutions you’re using.
Sending the password
- Use client-side encryption software such as GPG, or share the password via LastPass.
- Make sure the recipient is who they claim to be: use a system with strong two-factor authentication or rigorous key-agreement protocols.
- Make sure recipients are aware of the sensitivity of the password.
Storing the password
- Do not store the password in plaintext on machines. Use a cryptographically secure secret management system like LastPass, Vault, or GPG to save a backup of the key.
- Require full-disk encryption on machines that handle the password.
- Securely wipe the password from machines when it is no longer needed.
Using the password
- Make sure the analytics platform daemon runs as its own user, and no other unprivileged users on the system have permission to access the password.
- Control physical access to the server.
- Restrict administrative access to the server to a small and well-documented set of people.
- Use reputable and securely configured servers for admin access (for example, using SSH key-based authentication).
- Make sure the machine that hosts the local analytics platform instance does not expose endpoints unnecessarily, especially to the Internet.
- Make sure the system does not run other services unless they are segregated into an isolated security context.
We’ve provided a worked example below using Grafana.
The monitoring proxy uses HTTP basic authentication. You will need to supply
longshot_user as the username, and a valid refresh token as the password.
For access from a dashboard or other automated service, we strongly recommend you use the refresh token of a service account. To set up a service account, please reach out to Improbable customer support via the helpdesk (for customers with a service agreement) or the forums.
Worked example: Grafana
Before you start
This service is in alpha, so is subject to change. In particular, all URLs (especially
https://monitoring.service.improbable.io/@proxyhost/) and metric names are likely to change in the future.
Set up SpatialOS metrics in your own Grafana instance
- Log in to your Grafana server as an admin.
Under Data Sources, add a new Prometheus Data Source:
Data source fields Details Data source name SpatialOS Data source type Prometheus URL https://monitoring.service.improbable.io/@proxyhost/metrics.monitoring.spatial.improbable.io HTTP access Proxy HTTP auth Basic auth Basic auth details The username and password you got from Improbable customer support
Now you can add graphs of these metrics to your dashboard.
For details of how to do this, see the Grafana docs on Creating a Prometheus graph.
You can see which metrics are available, and get example queries to run, on the Metrics reference page.
Restrictions on metric querying
Queries must specify the project being queried.
Each project has a limit of 200 queries per minute.
For example, if one user issues 200 of the above queries, no users will be able to query metrics for that project for up to a minute.
Each project has a limit of 60s of query processing time, reset every minute.
Queries will time out after 30s of processing.
2019-09-23 Page updated with editorial review: Removed Set up SpatialOS logs in your own Grafana instance