Sites

Menu
These are the docs for 13.5, an old version of SpatialOS. The docs for this version are frozen: we do not correct, update or republish them. 14.2 is the newest →

Load balancing (deprecated)

The functionality described on this page is deprecated.

This page outlines the load balancing strategies available in SpatialOS, and explains how to configure them. Using these strategies you can adjust how many components on entities in the world each server-worker has write access to, so that your server-workers don’t get overloaded. You can also scale the number of server-workers in the world, adding or removing server-workers as needed.

You can set up load balancing to occur dynamically throughout the lifecycle of your deployment or to conform to a static model of allocation.

You set your load balancing configuration for each server-worker type in the launch configuration file, in the load_balancing field:

...
"workers": [
  {
    "worker_type": "my_worker",
    "load_balancing": {
       // configuration goes here
    }
  }
]
...

Singleton workers

Only use this configuration for client-workers. It won’t work for any other type of worker.

If your game will only ever have one instance of a type worker, use this setting to tell SpatialOS that it doesn’t need to scale or load balance that worker type.

To configure a worker as a singleton, use the following:

"load_balancing": {
    "singleton_worker": {}
}

Statically allocated workers

To have a fixed number of server-workers whose authoritative regions don’t change in response to load, use the static hex-grid configuration. This configuration starts a fixed number of server-workers, laid out in an ‘n by m’ hexagonal grid. centred on the origin of the world. The example below shows a 4 by 4 grid of server-workers:

Hexagonal grid

We recommend running at least two server-workers of a type. Only running one server-worker can cause problems which are hard to debug.

Automatic grid creation

Use auto_hex_grid to specify the number of server-workers you want to use. A configuration that covers the world will be generated automatically.

For example:

"load_balancing": {
  "auto_hex_grid": {
    "num_workers": 20
  }
}

Manual grid creation

Alternatively, you can manually specify the size of your grid, using static_hex_grid. You need to make sure your grid is slightly bigger than the world, so that all of the world is covered.

The fields in static_hex_grid are:

Field name Type Meaning
worker_radius_meters double The radius of a server-worker’s region of write access.
grid_width int The number of server-workers along the horizontal edge of the grid.
grid_height int The number of server-workers along the vertical edge of the grid.

All of these fields are required.

For example:


"load_balancing": {
  "static_hex_grid": {
    "worker_radius_meters": 1000.0,
    "grid_width": 5,
    "grid_height": 4
  }
}

Points of Interest

The Points of Interest strategy allows you to specify key locations in the world that should not be simulated by more than one server-worker.

As an input, the strategy takes a list of seeds or points of interest. From those seeds, the load balancer divides the world into regions. Each region consists of the area of the world that is closest to the point of interest (similar to a Voronoi diagram).

Once the world has been split into regions, the strategy distributes the regions amongst a specified number of server-workers.

Here is an example of a configuration for the Points of Interest load balancer:

"load_balancing": {
  "points_of_interest": {
    "num_workers": 2,
    "points": [
      {
        "x": 100,
        "z": 100
      },
      {
        "x": 100,
        "z": -100
      },
      {
        "x": -100,
        "z": 100
      },
      {
        "x": -100,
        "z": -100
      }
    ]
  }
}

This configuration specifies four points of interest and two server-workers. Since there are four points of interest, there are four regions: top left, top right, bottom left and bottom right.

The load balancer distributes the four regions amongst the two server-workers that are available. It will try to make sure that it picks regions which are adjacent to each other. This minimizes the total area that each server-worker will have to simulate. In this example, the server-workers will either split the regions such that both the regions on the left are on one server-worker and both regions on the right are on the other server-worker; or split the regions such that both regions on the top are on one server-worker and both regions on the bottom are on the other server-worker. The load balancer will not split the regions such that diagonal points are on the same server-worker.

Dynamically load-balanced workers

To tell SpatialOS to dynamically choose how many server-workers to run, and how to load balance between them, use the dynamic configuration.

For each dynamically load-balanced server-worker type, you must configure:

  • how server-workers are scaled
  • how new server-workers are placed
  • how server-workers are load balanced

The dynamic configuration is specified like this:

"load_balancing": {
  "dynamic_loadbalancer": {
    "worker_scaler_config": {...},
    "worker_placer_config": {...},
    "loadbalancer_config": {...}
  }
}

Dynamic worker scaling configuration

Worker scaling is the business of figuring out how many server-workers should be running at any one time in order to simulate the world. There are two server-worker scaling strategies: “constant” and “entity count”.

The “constant” scaler uses a fixed number of server-workers, no matter how much work there is to be done. To configure this, use:

"worker_scaler_config": {
  "constant_config": {
    "num_workers": 12
  }
}

The “entity count” scaler counts the total number of entities in the world and divides by a fixed number to get the number of server-workers that should be running. It repeats this calculation throughout the lifecycle of your deployment, starting and stopping server-workers dynamically. To configure this, use:

"worker_scaler_config": {
  "entity_count_config": {
    "entities_per_worker": 1000
  }
}

Dynamic worker placement configuration

When a server-worker is started, SpatialOS needs to pick where to start it. There are two positioning strategies: “random” and “hex grid”. (Hex grids can be specified in two ways.)

Once a server-worker has been started in a particular spot, SpatialOS can move the position and size of its region of write access.

The “random” strategy picks a position uniformly at random somewhere in the world. To configure this, use:

"worker_placer_config": {
  "random_params": {}
}

The “hex grid” strategy places server-workers in the same way as the static hex-grid allocator above.

You can specify the maximum number of server-workers to make use of, and allow the spacing to be calculated automatically, like this:

"worker_placer_config": {
  "auto_hex_grid_params": {
    "num_workers": 20
  }
}

The number of server-workers specified is a maximum. If a static allocation is calculated that can place server-workers with the same/lower load, or lower wastage, then this configuration will be used instead.

Alternatively, specify the details of the grid:

"worker_placer_config": {
  "hex_grid_params": {
    "worker_radius_meters": 1000.0,
    "grid_width": 5,
    "grid_height": 4
  }
}

Load balancing configuration

This configuration determines how large an area a server-worker can be authoritative for and how that area can be moved around or changed in size to balance load. To visualize how loadbalancing happens:

  • If a server-worker is underloaded, it can expand its radius (but not higher than the defined maximum) to take on more work.
  • If a server-worker is overloaded, it can contract its radius (but not lower than the defined minimum) to give up work to other server-workers.
  • The rate at which the server-worker radius contracts or expands can be configured.
  • The ‘centre’ of a server-worker, the point in space around which it will try to be authoritative, can be moved to rearrange server-workers into a more optimal configuration. The rate at which server-worker centres move is limited by a speed parameter.

To configure how these settings, use the following stanza:

"loadbalancer_config": {
  "min_range_meters": 500.0,
  "max_range_meters": 5000.0,
  "speed_meters_per_second": 100.0,
  "expansion_time_millis": 60000
}

All fields are mandatory.

The fields are:

Field name Type Meaning
min_range_meters double The radius of the minimum area a server-worker will be authoritative for.
max_range_meters double The radius of the maximum area a server-worker will be authoritative for.
speed_meters_per_second double The speed at which a server-worker’s centre can move.
expansion_time_millis int The minimum time a server-worker needs to expand its region of authority from minimum to maximum size.

Updating worker loads

Server-workers that make use of dynamic load balancing need to inform SpatialOS about their current load as this enables the load balancer to make informed decisions about the server-workers that it manages. The load of a server-worker is a special metric which can be sent to SpatialOS following the same method as all other metrics. Please refer to the documentation of your target language to learn more about sending metrics:

Search results

Was this page helpful?

Thanks for letting us know!

Thanks for your feedback

Need more help? Ask on the forums