Sending data to SpatialOS
A worker can send data such as logging, metrics, and component updates to SpatialOS using the
improbable.worker.Connection
.
Component updates
Sending component updates
When the worker has authority over a component on an entity, it can send component updates to SpatialOS by using the improbable.worker.Connection
method sendComponentUpdate
, which takes an EntityId
instance and an instance of appropriate update class defined in the schema-generated code. The update classes are always called Update
and are defined as a static nested classes of each implementation of ComponentMetaclass
. For example, to update a component MyComponent
, the appropriate update class is nested in it and can by accessed as MyComponent.Update
.
A component update can modify the value of a property or trigger an event. You can modify multiple properties and trigger multiple events in the same component update.
Component updates sent by the worker will appear in the operation list returned by a subsequent call to improbable.worker.Connection.getOpList
. This means that Ops.ComponentUpdate
callbacks will also be invoked for component updates triggered by the worker itself, but not immediately.
There are a couple of (related) reasons that callbacks for sent component updates should not be invoked immediately:
- This can lead to extremely unintuitive control flow when components are recursively updated inside a callback.
- It violates the guarantee that callbacks are only invoked as a result of a call to the
process
method on theimprobable.worker.Dispatcher
.
Receiving component updates
To be notified when a worker receives a component update on an entity in the worker’s local
view of the simulation, use the improbable.worker.Dispatcher
method onComponentUpdate
with the same
ComponentMetaclass
as for sending updates.
Note that the component updates:
- can be partial, i.e. only update some properties of the component
- do not necessarily have to contain data that is different from the workers current view of the component
- could have been sent by SpatialOS rather than a worker for synchronization purposes.
Sending and receiving component events
Sending and receiving events works much in the same way as component updates. For a schema like the following,
package example;
type SwitchToggled {
int64 time = 1;
}
component Switch {
id = 1234;
bool is_enabled = 1;
event SwitchToggled toggled;
}
To trigger an event:
private static void triggerEvent(Connection connection, EntityId entityId) {
Switch.Update update = new Switch.Update();
update.addToggled(new SwitchToggled(1));
connection.sendComponentUpdate(Switch.COMPONENT, entityId, update);
}
If you are not authoritative on the component, your event will be silently ignored.
Receiving an event works just like receiving a component update, by registering a callback on the dispatcher:
callbackKey = dispatcher.onComponentUpdate(Switch.COMPONENT, op -> {
Switch.Update update = op.update;
for (SwitchToggled toggleEvent : update.getToggled()) {
System.out.println("Switch has been toggled at " + toggleEvent.getTime());
}
});
Updating component interests
For each entity in its view, a worker receives updates for a set of the entity’s components. This set is the union of the set of components the worker has authority over, and a set of components it is explicitly interested in.
The initial set of explicitly interested components can be configured in the
bridge settings.
This is the default set of explicit interest for every entity. At runtime, the worker can override this
set on a per-entity basis, by using the Connection.sendComponentInterest
method.
Whenever the set of interested components for an entity changes, the worker will receive the appropriate
onAddComponent
and onRemoveComponent
callbacks to reflect this change.
For example, you might be interested in one specific switch, but not every switch in the world:
Note that a worker is always considered to be interested in the components of the entities it is authoritative on, in addition to any others specificed in the bridge settings or using the method described above.
Component commands
Sending component commands
To send a command request, use the improbable.worker.Connection
method sendCommandRequest
. The command is executed by the worker that currently has authority over the component that contains the command on the entity specified in the command request.
The sendCommandRequest
function takes:
- a
Class<C>
instance - an entity ID
- a request object
- an optional timeout
- an optional
CommandParameters
object
where:
- The
C
type needs to be a subclass ofCommandMetaclass
defined in the schema-generated code. - The request object needs to be of the same type as
defined in schema for given command
C
. - The
CommandParameters
object contains a field calledAllowShortCircuit
, which if set to true will try to “short-circuit” the command and avoid a round trip to SpatialOS in some cases. For more information, see documentation on commands.
Before sending the command, a callback to handle the response should be registered with the
improbable.worker.Dispatcher
with onCommandResponse(Class<C>)
. The request ID (of type
improbable.worker.RequestId<OutgoingCommandRequest>
) returned by
sendCommandRequest
can be matched up with the one in the improbable.worker.Ops.CommandResponse
to
identify the request that is being responded to.
Receiving component commands
To handle commands that another worker issues, use the opposite flow:
- Register a callback with the
improbable.worker.Dispatcher
withonCommandRequest(Class<C>)
. - When the callback is executed:
- To respond to the command, call the
improbable.worker.Connection
methodsendCommandResponse
. Supply the request ID (of typeimprobable.worker.RequestId<IncomingCommandRequest>
) provided by theimprobable.worker.Ops.CommandRequest
and an appropriate response object. The request ID should match the response type defined in the schema. - To fail the command, the worker can call
sendCommandFailure
.
- To respond to the command, call the
Command failures
Commands can fail. When a command fails, check the improbable.worker.StatusCode
field in the improbable.worker.Ops.CommandResponse
, and then retry the command as necessary.
The caller always gets a response callback, which can be one of the following failure cases:
APPLICATION_ERROR
: The command is rejected by the target worker or by SpatialOS.AUTHORITY_LOST
: The target worker lost authority, or no worker had authority.NOT_FOUND
: The target entity, or target component on the entity, didn’t exist.PERMISSION_DENIED
: The sending worker didn’t have permission to send request.TIMEOUT
INTERNAL_ERROR
: A bug might exist in SpatialOS. Raise a support request or ask on our forums.
Entity queries
Note: In order to send an entity query, a worker must have permission to do so. For more information, see the Worker permissions page.
A worker can run remote entity queries against the simulation by using the improbable.worker.Connection
method sendEntityQueryRequest
. This takes an improbable.worker.Query.EntityQuery
object and an optional timeout.
The query object is made of an improbable.worker.Query.Constraint
and an improbable.worker.Query.ResultType
.
The constraint determines which entities are matched by the query, and the result type determines what data is
returned for matched entities. Available constraints and result types are described below.
Constraint | Description |
---|---|
EntityIdConstraint |
Matches a specific entity ID. |
ComponentConstraint |
Matches entities with a particular component. |
SphereConstraint |
Matches entities contained in the given sphere. |
AndConstraint |
Matches entities that match all of the given subconstraints. |
OrConstraint |
Matches entities that match any of the given subconstraints. |
NotConstraint |
Matches entities that do not match the given subconstraint. |
Result type | Description |
---|---|
CountResultType |
Returns the number of entities that matched the query. |
SnapshotResultType |
Returns a snapshot of component data for entities that matched the query. To select all components, use new SnapshotResultType() . To select every component whose ID is contained in the given set, use new SnapshotResultType(componentIdSet) (thus, pass an empty set to get no components but entity IDs only). |
Important: You should keep entity queries as limited as possible. All queries hit the network and cause a runtime lookup, which is expensive even in the best cases. This means you should:
- always limit queries to a specific sphere of the world
- only return the information you need from queries (eg the specific components you care about)
- if you’re looking for entities that are within your worker’s checkout radius, search internally on the worker instead of using a query
Like other request methods, this returns an improbable.worker.RequestId
request ID, which can be
used to match a request with its response. The response is received via a callback registered with the
improbable.worker.Dispatcher
using the onEntityQueryResponse
method.
The Ops.EntityQueryResponse
contains an int resultCount
field (for CountResultType
requests)
and a Map<EntityId, Entity>
(for SnapshotResultType
) requests. Again, success or failure of the
request is indicated by the statusCode
field of the response object, but in the failure case the
result may still contain some data: the count or snapshot map might still contain the data for
some entities that matched the query, but won’t necessarily contain all matching entities. This is
because the worker might still be able to do something useful with a partial result.
Sending and receiving metrics
You can optionally send metrics by calling Connection.sendMetrics
. Metrics can be viewed on deployment dashboards and in the Inspector.
Typical use cases for sending metrics are as follows:
- Reporting your own metrics. There are APIs for both time series and histogram metrics. However, only time series metrics are currently exposed via the Inspector. Unlike the metrics that are listed on Metrics reference, your own metrics are not persistent.
Updating a worker’s load. The load of a worker is a floating-point value. The reported values direct SpatialOS’s load balancing strategy:
- A value of 0 indicates an unloaded worker.
- Values above 1 correspond to an overloaded worker.
Example
The following example demonstrates both use cases:
// A queue of tasks the worker has to complete.
private final static java.util.Queue TaskQueue = new java.util.ArrayDeque<>();
private final static double MAXIMUM_QUEUE_SIZE = 200.0; // An arbitrary maximum value.
// Collect and send user-defined metrics to SpatialOS.
// Note that you should generally call sendMetrics on the same user thread as you call
// other Connection methods.
private static void sendUserMetrics(Connection connection) {
Metrics metrics = new Metrics();
// Update the current load of the worker.
double load = TaskQueue.size() / MAXIMUM_QUEUE_SIZE;
metrics.load = improbable.collections.Option.of(load);
// Add custom metrics.
metrics.gaugeMetrics.put("MyCustomMetric", 1.0);
connection.sendMetrics(metrics);
}
Workers automatically send several built-in, internal metrics at a period defined by the
builtInMetricsReportPeriodMillis
field of ConnectionParameters
. You can register a callback
to receive these metrics inside a Ops.MetricsOp
using Dispatcher.OnMetrics
:
private static void registerMetricsCallback(Dispatcher dispatcher) {
dispatcher.onMetrics(op -> {
double shortCircuitRate = op.metrics.gaugeMetrics.get("connection_command_request_short_circuit_rate");
// Do something with the metric, or store it...
System.out.println("Command requests short-circuited per second: " + shortCircuitRate);
});
}
The full list of built-in gauge metrics is as follows. All rate metrics are per-second.
Metric name (String ) |
Metric value (double ) |
---|---|
connection_send_queue_size |
The current size of the send queue (the messages waiting to be sent to SpatialOS). |
connection_send_queue_fill_rate |
The rate at which messages are being added to the send queue. |
connection_receive_queue_size |
The current size of the send queue. |
connection_receive_queue_fill_rate |
The rate at which messages are being received from SpatialOS and added to the receive queue. |
connection_oplist_queue_size |
The current size of the op list. |
connection_oplist_queue_fill_rate |
The rate at which ops are being added to the internal OpList (the queue of processed messages that workers operate on). |
connection_log_message_send_rate |
The rate at which log messages are being sent. |
connection_component_update_send_rate |
The rate at which component updates are being sent. |
connection_add_component_send_rate |
The rate at which add components are being added sent. |
connection_remove_component_send_rate |
The rate at which remove components are being sent. |
connection_command_request_send_rate |
The rate at which command requests are being sent. |
connection_command_response_send_rate |
The rate at which successful command responses are being sent. |
connection_command_failure_send_rate |
The rate at which command failure responses are being sent. |
connection_local_command_timeouts |
The total local commands that timed out when waiting for a response. |
connection_local_command_timeouts_rate |
The rate at which local commands time out when waiting for a response. |
connection_unexpected_command_response_receives |
The total unexpected command responses recieved. |
connection_unexpected_command_response_receives_rate |
The rate at which unexpected command responses are recieved. |
connection_reserve_entity_id_request_send_rate |
The rate at which requests to reserve an entity ID are being sent. |
connection_reserve_entity_ids_request_send_rate |
The rate at which requests to reserve multiple entity IDs are being sent. |
connection_create_entity_request_send_rate |
The rate at which entity creation requests are being sent. |
connection_delete_entity_request_send_rate |
The rate at which entity deletion requests are being sent. |
connection_entity_query_request_send_rate |
The rate at which entity query requests are being sent. |
connection_component_interest_send_rate |
The rate at which component interest updates are being sent. |
connection_authority_loss_imminent_acknowledgement_send_rate |
The rate at which imminent authority loss acknowledgements are being sent. |
connection_command_request_short_circuit_rate |
The rate at which command requests are being short-circuited. |
connection_command_response_short_circuit_rate |
The rate at which successful command responses are being short-circuited. |
connection_command_failure_short_circuit_rate |
The rate at which command failure responses are being sent. |
connection_flag_update_op_receive_rate |
The rate at which FlagUpdate Ops are being received. |
connection_critical_section_op_receive_rate |
The rate at which CriticalSection Ops are being received. |
connection_add_entity_op_receive_rate |
The rate at which AddEntity Ops are being received. |
connection_remove_entity_op_receive_rate |
The rate at which RemoveEntity Ops are being received. |
connection_reserve_entity_id_response_op_receive_rate |
The rate at which ReserveEntityIdResponse Ops are being received. |
connection_reserve_entity_ids_response_op_receive_rate |
The rate at which ReserveEntityIdsResponse Ops are being received. |
connection_create_entity_response_op_receive_rate |
The rate at which CreateEntityResponse Ops are being received. |
connection_delete_entity_response_op_receive_rate |
The rate at which DeleteEntityResponse Ops are being received. |
connection_entity_query_response_op_receive_rate |
The rate at which EntityQueryResponse Ops are being received. |
connection_add_component_op_receive_rate |
The rate at which AddComponent Ops are being received. |
connection_remove_component_op_receive_rate |
The rate at which RemoveComponent Ops are being received. |
connection_authority_change_op_receive_rate |
The rate at which AuthorityChange Ops are being received. |
connection_component_update_op_receive_rate |
The rate at which ComponentUpdate Ops are being received. |
connection_command_request_op_receive_rate |
The rate at which CommandRequest Ops are being received. |
connection_command_response_op_receive_rate |
The rate at which CommandResponse Ops are being received. |
connection_egress_bytes |
The number of bytes that have been sent. This refers to bytes encoded by the application layer - the actual number of bytes transmitted on the transport may be slightly higher. |
connection_egress_bytes_rate |
The rate at which data is being sent, in bytes per second. |
connection_ingress_bytes |
The number of bytes that have been received. This refers to bytes decoded by the application layer - the actual number of bytes received on the transport may be slightly higher. |
connection_ingress_bytes_rate |
The rate at which data is being received, in bytes per second. |
connection_delta_compression_egress_bandwidth_saved_bytes |
The number of network egress bytes saved through delta compressing component updates. |
connection_delta_compression_egress_bandwidth_saved_bytes_rate |
The rate at which network egress bandwidth is saved through delta compressing component updates, in bytes per second. |
connection_delta_compression_egress_total_diffs_sent |
The number of delta compressed component updates sent. |
connection_delta_compression_egress_total_diffs_sent_rate |
The rate at which delta compressed component updates are sent, in updates per second. |
connection_delta_compression_egress_diffs_abandoned |
The number of delta compressed component updates abandoned (due to taking too long to compute or being too large). |
connection_delta_compression_egress_diffs_abandoned_rate |
The rate at which delta compressed component updates are abandoned, in updates per second. |
connection_delta_compression_ingress_bandwidth_saved_bytes |
The number of network ingress bytes saved through delta compressing component updates. |
connection_delta_compression_ingress_bandwidth_saved_bytes_rate |
The rate at which network ingress bandwidth is saved through delta compressing component updates, in bytes per second. |
raknet_receive_buffer_size |
The current size of the RakNet receive buffer. |
raknet_send_buffer_size |
The current size of the RakNet read buffer. |
raknet_send_buffer_size_bytes |
The number of bytes in the RakNet send buffer. |
raknet_resend_buffer_size |
The number of messages waiting in the RakNet resend buffer. |
raknet_resend_buffer_size_bytes |
The number of bytes in the RakNet resend buffer. |
raknet_packet_loss_last_second |
The packet loss over the last second. This number will range from 0.0 to 1.0. |
raknet_packet_loss_lifetime |
The packet loss average over the lifetime of the connection. This number will range from 0.0 to 1.0. |
raknet_last_ping_seconds |
The response time of the last ping emitted by the RakNet client. |
kcp_send_queue_size_packets |
The number of KCP packets currently in the send queue, waiting to be sent. |
kcp_recv_queue_size_packets |
The number of KCP packets currently in the receive queue, waiting to be processed. |
kcp_smoothed_round_trip_time_seconds |
A time-weighted moving average of the round-trip time between the worker and the Runtime. |
kcp_round_trip_time_variation_seconds |
A time-weighted moving average of the difference between each round-trip time sample and the smoothed round-trip time. |
erasure_coding_completed_batches |
The number of erasure codec batches whose packets were all successfully delivered. |
erasure_coding_completed_batches_rate |
The number of erasure codec batches whose packets were all successfully delivered, per second. |
erasure_coding_recovered_batches |
The number of erasure codec batches for which the original data was recovered even though not all packets were delivered. |
erasure_coding_recovered_batches_rate |
The number of erasure codec batches for which the original data was recovered even though not all packets were delivered, per second. |
erasure_coding_unrecoverable_batches |
The number of erasure codec batches for which the original data could not be recovered because too many packets were missing. |
erasure_coding_unrecoverable_batches_rate |
The number of erasure codec batches for which the original data could not be recovered because too many packets were missing, per second. |
The full list of built-in histogram metrics is as follows.
Metric name (String ) |
Metric values (double ) |
---|---|
kcp_packet_send_count |
The number of times each KCP packet had to be sent before its delivery was acknowledged. |
You need to register an OnMetrics
callback with the dispatcher to receive histogram metrics
as they aren’t currently exposed in the Inspector.