The Inspector is a web-based tool that you use to explore the internal state of a SpatialOS world. It gives you a real time view of what’s happening in a deployment, locally or in the cloud. Among other things, it displays:
- which SpatialOS entities are in the world, and what their components’ properties are
- which worker instances are connected to the deployment, and how much load each one is under
- the load balancing strategies of the worker instances
- the area of authority and area of interest for each worker instance
This makes the Inspector a crucial companion to monitoring and debugging SpatialOS deployments.
Accessing the Inspector
For a local deployment, access the Inspector at http://localhost:21000/inspector. When running locally, the Inspector automatically downloads and caches the latest Inspector Client from the internet. If it can’t download the latest version (for example, because there is no internet connection or because of the firewall configuration), it uses the latest cached version instead.
For a cloud deployment, access the Inspector from your deployment’s Overview page by clicking Inspect:
The Inspector has a connectivity message on the top left to indicate whether there are any delays in the ‘freshness’ of the deployment data being displayed.
More about the connectivity message
To bring back information about your deployment, the Inspector connects to a number of endpoints on your deployment. The connectivity message, on the top left, shows the status of these connections. You can use this to spot and diagnose any potential issues.
|All connections are healthy, and you can inspect your deployment. The Inspector’s view of your deployment is up to date.|
|One or more connections is down, and the Inspector is trying to reconnect. The Inspector’s view of your deployment is not up to date.|
The connection status is based on the status of four subconnections:
|World Bounds||Connection to the SpatialOS Runtime. If disconnected, the deployment is likely not up.|
|Chunks||Connection to the chunks endpoint, which provides information about which worker instances have interest in which sections of the world (collections of “chunks”).|
|Position||Connection to the entity positions stream. This connection is required to display entities in the Inspector.|
|Heatmap||Connection to the heatmap view of the deployment, showing an aggregate number of entities per area. This connection is only used in Heatmap mode (see Performance impact).|
Entities and their components
When you have selected an entity, either by finding it in the world or entering its ID, the entity sidebar populates:
From here, you can:
- Delete an entity: remove it from your deployment.
- Jump to an entity: center the view on it.
- Observe all of the entity’s component properties in real time. The components are presented either in tree structure or in JSON, and you can filter them. All values are sampled approximately every second.
- Observe which worker instances have write access authority over components on this entity.
Workers and load balancing
The top right-hand side of the Inspector shows a list of the worker instances currently connected to your deployment:
From this list, you can select a particular worker instance to center the view on that instance.
Much like selecting entities, clicking on a worker instance’s name brings up its information in the sidebar:
From here, you can:
- Stop the worker instance.
- Jump to a worker instance: center the view on its area of authority.
- Go to the Logs viewer or the raw worker logs page.
- Observe a number of useful worker metrics, such as FPS, Send/Receive Queue sizes, and counts of entities the worker instance is acting on broken down by entity type.
You can also view the load balancing strategy and areas of interest and authority for particular worker instances. Select one or more worker instances by clicking the checkbox next to the instance name, then use the “Show me” box to choose what you want to see:
Load balancing strategy
Authority area and interest area
From the Settings tab, you can:
- Change the grid units from world units to chunks
- Turn on the ability to jump to an entity
- Change the maximum zoom level to use until heatmap mode kicks in (see Performance impact)
- Import and export your entity coloring settings
Using the Inspector can impact the performance of the deployment you’re inspecting.
As a rule of thumb, the more information the Inspector is displaying, the greater the performance impact. For example, zooming out a lot causes a large performance impact, because it vastly increases the number of entities shown.
Changing the maximum zoom level to a very high number can have a similar effect.
The performance impact is decreased in heatmap mode, as this mode displays ‘density’ of entities per area rather than all individual entities:
Turning on heatmap mode doesn’t remove the performance impact completely, though. It still runs
operations which also impact performance. You can track these operations in the “Debug entities and commands” dashboard of the
Why is there a performance impact?
To display the state of your deployment, the Inspector accesses a real-time stream of the entities in the area being inspected, and their components’ properties/values, along with all worker instances, their interest areas, their metrics, and more.
This information is provided by a set of APIs on an Inspection Service, located on one of the nodes running in the deployment being inspected.
To provide these APIs, the Inspection Service queries other nodes within your deployment to stream the state of entities; this therefore puts additional stress on the deployment when you use the Inspector.
How to mitigate the performance impact
Minimize the number of concurrent Inspector windows. Each Inspector window separately connects to the Inspection Service - this means the more are open, the greater the load.