Get SpatialOS

Sites

Menu
These are the docs for 13.2, an old version of SpatialOS. 13.3 is the newest →

Accessing metrics and logs on your own analytics platform

This page explains how to view SpatialOS metrics and logs within your own in-house or third-party analytics platform, using the monitoring proxy that lets you access your deployment’s logs and metrics.

The monitoring proxy is currently only available to select users for testing. If you’d like access, raise a support request (for customers with a service agreement) or ask on our forums.

You can also use the monitoring proxy to access metrics through code.

In your analytics platform, you’ll need to:

  1. Add Prometheus and Elasticsearch as data sources.
  2. Add the monitoring proxy URL.
  3. Enter the HTTP basic authentication username and password that we’ve given you.

We’ve provided a worked example below using Grafana.

Worked example: Grafana

Before you start

  • If you don’t already have a Grafana server, you’ll need to set one up. You can install one yourself or use Grafana cloud.

  • This service is in alpha, so is subject to change. In particular, all URLs (especially https://monitoring.service.improbable.io/@proxyhost/), metric names, and log messages are likely to change in the future.

Set up SpatialOS metrics in your own Grafana instance

  1. Ask Improbable customer support to set up a metrics service account for you.

    They’ll send you a username and password.

  2. Log in to your Grafana server as an admin.

  3. Under Data Sources, add a new Prometheus Data Source for each cluster you use (eg eu1-prod, us2-prod).

    For example, for a deployment running on eu2-prod, you’d add the following details:

    Data source fields Details
    Data source name eu2-prod (this must be the exact name of the cluster)
    Data source type Prometheus
    URL https://monitoring.service.improbable.io/@proxyhost/eu2-prod.monitoring.spatial.improbable.io
    HTTP access Proxy
    HTTP auth Basic auth
    Basic auth details The username and password you got from Improbable customer support
  4. Click Save & Test. You’ll see the following error:

    HTTP Error: Forbidden

    This is fine. The connection is successful, but the metric used on Prometheus to test the connection by Grafana is “forbidden” to your account.

  5. Now you can add graphs of these metrics to your dashboard.

    For details of how to do this, see the Grafana docs on Creating a Prometheus graph.

    You can see which metrics are available, and get example queries to run, on the Metrics reference page.

  • Only create alerts using documented metrics.

  • Be careful with the queries you write: if they’re very expensive, they can cause problems with the metrics or logging servers. If you’re not sure, ask customer support to review the queries.

Set up SpatialOS logs in your own Grafana instance

  1. Ask Improbable customer support to set up a logs service account for you.

    They’ll send you a username and password.

  2. Log in to your Grafana server as an admin.

  3. Under Data Sources, add a new Elasticsearch Data Source.

    For example, for the project test_project on the eu2-prod cluster:

    Data source fields Details
    Data source name eu2-prod (this must be the exact name of the cluster)
    Data source type Elasticsearch
    URL https://monitoring.service.improbable.io/@proxyhost/es-eu2-prod.logs.spatial.improbable.io
    HTTP access Proxy
    HTTP auth Basic auth
    Basic auth details The username and password you got from Improbable customer support
    Elasticsearch-specific fields Details
    Index name [fablog-test_project-]YYYY.MM.DD
    Pattern Daily
    Time field name @timestamp
    Version 2.x
  4. Click Save & Test. You’ll see the following:

    Success: Index OK. Field name OK.

Don’t create alerts from the exact wording of logs. The wording can change without notice.

Things you should bear in mind

The gross query rate is limited to 50 requests per minute. To avoid hitting this limit, in the Grafana refresh time settings, set the auto-refresh interval to at least 1 minute.

Search results

Was this page helpful?

Thanks for letting us know!

Thanks for your feedback

Need more help? Ask on the forums