Get SpatialOS

Sites

Menu
You are viewing the docs for 11.0, an old version of SpatialOS. 12.0 is the newest →

Load balancing

You can configure how each type of worker is scaled and load balanced. This configuration is set for each worker type in the launch configuration file, in the load_balancing field:

...
"workers": [
  {
    "worker_type": "my_worker",
    "load_balancing": {
       // configuration goes here
    }
  }
]
...

On this page:

Singleton workers

Note: Only use this configuration for external (not managed by SpatialOS) workers like clients. It won’t work for any other type of worker.

If there will only ever be one instance of a particular worker, tell SpatialOS that it doesn’t need to scale or load balance the worker type.

To configure a worker as a singleton, use the following:

"load_balancing": {
    "singleton_worker": {}
}

Statically allocated workers

To have a fixed number of workers whose authoritative regions don’t change in response to load, use the static hex-grid configuration. This configuration starts a fixed number of workers, laid out in an ‘n by m’ hexagonal grid. centred on the origin of the world. The example below shows a 4 by 4 grid of workers:

Hexagonal grid

NOTE: We recommend running at least two managed workers of a type. Only running one worker can cause problems which are hard to debug.

Automatic grid creation

Use auto_hex_grid to specify the number of workers you want to use. A configuration that covers the world will be generated automatically.

For example:

"load_balancing": {
  "auto_hex_grid": {
    "num_workers": 20
  }
}

Manual grid creation

Alternatively, you can manually specify the size of your grid, using static_hex_grid. You need to make sure your grid is slightly bigger than the world, so that all of the world is covered.

The fields in static_hex_grid are:

Field name Type Meaning
worker_radius_meters double The radius of a worker’s region of write access.
grid_width int The number of workers along the horizontal edge of the grid.
grid_height int The number of workers along the vertical edge of the grid.

All of these fields are required.

For example:


"load_balancing": {
  "static_hex_grid": {
    "worker_radius_meters": 1000.0,
    "grid_width": 5,
    "grid_height": 4
  }
}

Dynamically load-balanced workers

To tell SpatialOS to dynamically choose how many workers to run, and how to load balance between them, use the dynamic configuration.

For each dynamically load-balanced worker type, you must configure:

  • how workers are scaled
  • how new workers are placed
  • how workers are load balanced

The dynamic configuration is specified like this:

"load_balancing": {
  "dynamic_loadbalancer": {
    "worker_scaler_config": {...},
    "worker_placer_config": {...},
    "loadbalancer_config": {...}
  }
}

Dynamic worker scaling configuration

Worker scaling is the business of figuring out how many workers should be running in order to simulate the world. There are two worker scaling strategies: “constant” and “entity count”.

The “constant” scaler uses a fixed number of workers, no matter how much worker there is to be done. To configure this, use:

"worker_scaler_config": {
  "constant_config": {
    "num_workers": 12
  }
}

The “entity count” scaler counts the total number of entities in the world and divides by a fixed number to get the number of workers that should be started. To configure this, use:

"worker_scaler_config": {
  "entity_count_config": {
    "entities_per_worker": 1000
  }
}

Dynamic worker placement configuration

When a worker is started, SpatialOS needs to pick where to start it. There are two positioning strategies: “random” and “hex grid”. (Hex grids can be specified in two ways.)

Once a worker has been started in a particular spot, SpatialOS can move the position and size of its region of write access.

The “random” strategy picks a position uniformly at random somewhere in the world. To configure this, use:

"worker_placer_config": {
  "random_params": {}
}

The “hex grid” strategy places workers in the same way as the static hex-grid allocator above.

You can specify the maximum number of workers to make use of, and allow the spacing to be calculated automatically, like this:

"worker_placer_config": {
  "auto_hex_grid_params": {
    "num_workers": 20
  }
}

Note: The number of workers specified is a maximum. If a static allocation is calculated that can place workers with the same/lower load, or lower wastage, then this configuration will be used instead.

Alternatively, specify the details of the grid:

"worker_placer_config": {
  "hex_grid_params": {
    "worker_radius_meters": 1000.0,
    "grid_width": 5,
    "grid_height": 4
  }
}

Load balancing configuration

This configuration determines how large an area a worker can be authoritative for and how that area can be moved around or changed in size to balance load. To visualize how loadbalancing happens:

  • If a worker is underloaded, it can expand its radius (but not higher than the defined maximum) to take on more work.
  • If a worker is overloaded, it can contract its radius (but not lower than the defined minimum) to give up work to other workers.
  • The rate at which the worker radius contracts or expands can be configured.
  • The ‘centre’ of a worker, the point in space around which it will try to be authoritative, can be moved to rearrange workers into a more optimal configuration. The rate at which worker centres move is limited by a speed parameter.

To configure how these settings, use the following stanza:

"loadbalancer_config": {
  "min_range_meters": 500.0,
  "max_range_meters": 5000.0,
  "speed_meters_per_second": 100.0,
  "expansion_time_millis": 60000
}

All fields are mandatory.

The fields are:

Field name Type Meaning
min_range_meters double The radius of the minimum area a worker will be authoritative for.
max_range_meters double The radius of the maximum area a worker will be authoritative for.
speed_meters_per_second double The speed at which a worker’s centre can move.
expansion_time_millis int The minimum time a worker needs to expand its region of authority from minimum to maximum size.

Updating worker loads

Workers that make use of dynamic load balancing need to inform SpatialOS about their current load as this enables the load balancer to make informed decisions about the workers that it manages. The load of a worker is a special metric which can be sent to SpatialOS following the same method as all other metrics. Please refer to the documentation of your target language to learn more about sending metrics:

Was this page helpful?

Thanks for letting us know!

Thanks for your feedback

Need more help? Ask on the forums