Sites

Menu

Non-Unreal server-worker types

By default, the GDK for Unreal uses a single Unreal server-worker type to handle all server-side computation. However, you can set up additional server-worker types that do not use Unreal or the GDK.

You can use these non-Unreal server-worker types to modularize your game’s functionality so you can re-use the functionality across different games. For example, you could use a non-Unreal server-worker type written in C# that interacts with a database or other third-party service, such as Firebase or PlayFab.

How to integrate non-Unreal server-worker types into your game

In order to interact with each other, Unreal and non-Unreal server-worker instances need to send and receive updates to and commands for the same SpatialOS components. We recommend that you define the SpatialOS components used by non-Unreal worker instances manually in schema files, separate from those automatically generated by the GDK. This is because:

  • SpatialOS command data in GDK-generated schema files is encoded as byte strings, making deserialization of commands in non-Unreal server-worker types more difficult.
  • It aids portability; you can more easily re-use any non-Unreal server-worker types when you have an accompanying schema file.

Default single Unreal server-worker type development doesn’t accommodate schema files that haven’t been generated by the GDK, so you need to set your game up to handle this.

TIP: If you’re using schema from outside the GDK, you can customise snapshot generation from the GDK toolbar’s Snapshot button. This is to serialize additional entities with these external components which the default Unreal snapshot generation cannot currently do. See Add to the snapshot below for how to do this.

How to set up interaction

To set up your game to interact with SpatialOS components defined outside the GDK (external SpatialOS components), you have two options:

  • (recommended) Using the GDK ExternalSchemaCodegen.bat helper script to generate classes for external schema types and an interface for sending updates and registering callbacks for receiving updates.

  • Sending updates and registering callback for receiving updates directly through the GDK, this requires you to serialize and deserialize the low-level C API types.

Note: Your external SpatialOS components must have an ID between 1000 and 2000 to be registered by the pipeline.

Using the code generator

The external schema code generator is a tool in the GDK that generates C++ code in your game module given a specific directory containing schema files. The generated code includes classes representing each external schema component, enumeration and type, and an ExternalSchemaInterface class for sending updates and registering callbacks for receiving network operations. The ExternalSchemaInterface generates methods for each component and command within a component in your external schema:

  • For each component, the ExternalSchemaInterface will contain methods for SendComponentUpdate, OnAddComponent, OnComponentUpdate, OnAuthorityChange, and OnRemoveComponent.
  • For each command, the ExternalSchemaInterface will contain methods for SendCommandRequest, SendCommandResponse, OnCommandRequest and OnCommandResponse.
Setting up

Below is a summary of the steps required to use the external schema code generator. There is a more complete example in the examples section below.

  • Run the ExternalSchemaCodegen.bat file as described in the helper scripts documentation to generate code from a directory containing external schema files.
  • Regenerate the Visual Studio project files.
  • Instantiate the ExternalSchemaInterface class inside your game module. If you want to ensure that the SpatialOS worker connection registers your callbacks to receive the network operations that are sent as soon as your worker connects, you need to register the callbacks inside your game instance’s OnConnected event callback.
  • To send SpatialOS component updates and commands, call the overloaded SendComponentUpdate, SendCommandRequest, or SendCommandResponse method for the relevant schema type.
  • To register callbacks for receiving network operations, call the overloaded OnAddComponent, OnComponentUpdate, OnAuthorityChange, OnRemoveComponent, OnCommandRequest or OnCommandResponse method for the relevant schema type.
  • To deregister callbacks, call the RemoveCallback method with CallbackId returned by the corresponding call to one of the callback registration functions.

Manual (de)serialization

As an alternative to using the code generator, you can interact with external schema components through manually serializing and deserializing C API network operations. This is what the generated code above does internally.

To send SpatialOS component updates and commands, use methods defined in the SpatialWorkerConnection.h file:

  • void SendComponentUpdate(Worker_EntityId EntityId, const Worker_ComponentUpdate* ComponentUpdate);
  • Worker_RequestId SendCommandRequest(Worker_EntityId EntityId, const Worker_CommandRequest* Request, uint32_t CommandId);
  • void SendCommandResponse(Worker_RequestId RequestId, const Worker_CommandResponse* Response);

To receive network operations for external SpatialOS components, you must provide custom callbacks for specific component IDs and operation types via USpatialDispatcher functions. These functions are parameterized with a Worker_ComponentId and a callback function const reference that takes a different network operation types as an argument:

  • OnAddComponent parameterized with const TFunction<void(const Worker_AddComponentOp&)>&
  • OnRemoveComponent parameterized with const TFunction<void(const Worker_RemoveComponentOp&)>&
  • OnAuthorityChange parameterized with const TFunction<void(const Worker_AuthorityChangeOp&)>&
  • OnComponentUpdate parameterized with const TFunction<void(const Worker_ComponentUpdateOp&)>&
  • OnCommandRequest parameterized with const TFunction<void(const Worker_CommandRequestOp&)>&
  • OnCommandResponse parameterized with const TFunction<void(const Worker_CommandResponseOp&)>&

If you want to ensure that the SpatialOS worker connection registers your callbacks to receive the initial network operations, you need to register the callbacks inside your game instance’s OnConnected event callback.

Each callback registration function returns a CallbackId. You can deregister your callbacks using the USpatialDispatcher::RemoveOpCallback function and passing the CallbackId parameter that was returned by the corresponding call to one of the callback registration functions.

There are basic example in the Examples section below. For more examples of how to serialize and deserialize component updates, command requests, and more, see the SpatialOS documentation on serialization in the Worker SDK in C’s API. Worker_ComponentId and each network operation type are defined in the Worker SDK in C’s API.

Add to the snapshot

You can customize snapshot generation by creating a class derived from the GDK USnapshotGenerationTemplate base class, and implementing the method below. You have the responsibility of incrementing the NextEntityId reference. If you don’t, snapshot generation will fail by attempting to add multiple entities to the snapshot with the same ID.

  /** Write to the snapshot generation output stream.
    * @param OutputStream the output stream for the snapshot being created.
    * @param NextEntityId the next available entity ID in the snapshot, this reference should be incremented appropriately.
    * @return bool the success of writing to the snapshot output stream, this is returned to the overall snapshot generation.
    **/
bool WriteToSnapshotOutput(Worker_SnapshotOutputStream* OutputStream, Worker_EntityId& NextEntityId);

There is a basic example in the Examples section below. For more examples of how to deserialize see the SpatialOS documentation on serialization in the Worker SDK in C’s API.

Examples

Below is a simple example schema file which a non-Unreal server-worker type could use to track player statistics:

package improbable.testing;

type UnrealRequest {
    string some_request_string = 1;
}
type UnrealResponse {
    string some_response_string = 1;
}

component TestComponent {
    id = 1337;
    uint32 counter = 1;
    command UnrealResponse test_command(UnrealRequest);
}

component OtherTestComponent {
    id = 1338;
    uint32 other_counter = 1;
}

Using the code generator example

Using the above schema file placed in the project relative directory spatial\schema\my_external_schema\, the command below will output code to a new Game\Source\<your_project_name>\ExternalSchemaCodegen folder where <project_root> is the path to the root of your Unreal project which contains the spatial folder:

Game\Plugins\UnrealGDK\SpatialGDK\Build\Scripts\ExternalSchemaCodegen.batspatial\schema\my_external_schema Game\Source\<your_project_name>\ExternalSchemaCodegen

This will generate code in the Game\Source\<your_project_name>\ExternalSchemaCode folder. To see these changes in Visual Studio, the Visual Studio project files need to be regenerated.

To send or receive network operations, you need to instantiate the ExternalSchemaInterface class inside your game module.

To send component updates or command requests / responses, you can call any of the overrides for SendComponentUpdate, SendCommandRequest or SendCommandResponse, for example:

auto Update = new ::improbable::testing::TestComponent::Update(20 /* counter field */);
// further edit Update using constructor pattern methods, e.g. (void)Update->SetCounter(30);
ExternalSchema->SendComponentUpdate(30 /* entity_id */, Update);

where ExternalSchema has the type: ExternalSchemaInterface*.

To register callbacks to receive network operations, you can call any of the overrides for OnAddComponent, OnComponentUpdate, OnAuthorityChange, OnRemoveComponent, OnCommandRequest or OnCommandResponse, for example:

ExternalSchema->OnComponentUpdate([&](const ::improbable::testing::TestComponent::ComponentUpdateOp& Op) {
    // access via Op.Update.get_counter();
});

If you want to ensure that the SpatialOS worker connection registers your callbacks to receive the initial network operations, you need to register the callbacks inside your game instance’s OnConnected event callback.

For example, the following code instantiates the ExternalSchema member of type ExternalSchemaInterface on the UTPSGameInstance:

void UTPSGameInstance::Init()
{
    OnConnected.AddLambda([this]() {
        // On the client the world may not be completely set up, if so we can use the PendingNetGame
        USpatialNetDriver* NetDriver = Cast<USpatialNetDriver>(GetWorld()->GetNetDriver());
        if (NetDriver == nullptr)
        {
            NetDriver = Cast<USpatialNetDriver>(GetWorldContext()->PendingNetGame->GetNetDriver());
        }

        ExternalSchema = new ExternalSchemaInterface(NetDriver->Connection, NetDriver->Dispatcher);

        // register callbacks here
   });
}

Manual (de)serialization examples

You could serialize and send a component update in your Unreal project code in the following way:

void SendSomeUpdate(Worker_EntityId TargetEntityId, Worker_ComponentId ComponentId)
{
    Worker_ComponentUpdate Update = {};
    Update.component_id = ComponentId;
    Update.schema_type = Schema_CreateComponentUpdate(ComponentId);
    Schema_Object* FieldsObject = Schema_GetComponentUpdateFields(Update.schema_type);
    Schema_AddInt32(FieldsObject, 1, 123456);
    Cast<USpatialNetDriver>(World->GetNetDriver())->Connection->SendComponentUpdate(TargetEntityId, &Update);
}

You could serialize and send a command response in your Unreal project code in the following way:

Worker_RequestId SendSomeCommandResponse(Worker_EntityId TargetEntityId, Worker_ComponentId ComponentId, Schema_FieldId CommandId) {
    Worker_CommandResponse Response = {};
    Response.component_id = ComponentId;
    Response.schema_type = Schema_CreateCommandResponse(ComponentId, CommandId);
    Schema_Object* ResponseObject = Schema_GetCommandResponseObject(Response.schema_type);
    const char* Text = "Hello World.";
    Schema_AddBytes(ResponseObject, 1, (const uint8_t*)Text, sizeof(char) * strlen(Text));
    Cast<USpatialNetDriver>(World->GetNetDriver())->Connection->SendCommandResponse(TargetEntityId, &Response);
}

You could receive and deserialize a component update and command request in your Unreal project code in the following way:

void UTPSGameInstance::Init()
{
    // OnConnected is an event declared in USpatialGameInstance
    OnConnected.AddLambda([this]() {
        // On the client the world may not be completely set up, if s we can use the PendingNetGame
        USpatialNetDriver* NetDriver = Cast<USpatialNetDriver>(GetWorld()->GetNetDriver());
        if (NetDriver == nullptr)
        {
            NetDriver = Cast<USpatialNetDriver>(GetWorldContext()->PendingNetGame->GetNetDriver());
        }
        USpatialDispatcher* Dispatcher = NetDriver->Dispatcher;

        Dispatcher->OnComponentUpdate(1337, [this](const Worker_ComponentUpdateOp& Op) {
            // Example deserializing component update network operation
            uint32 CounterValue = Schema_GetUint32(Schema_GetComponentUpdateFields(Op.update.schema_type), 1);
            // do something with CounterValue

            // Example actor spawning from within a callback
            const FVector Location = FVector::ZeroVector;
            const FRotator Rotation = FRotator::ZeroRotator;
            GetWorld()->SpawnActor(CubeClass, &Location, &Rotation);
        });

        Dispatcher->OnCommandRequest(1338, [this](const Worker_CommandRequestOp& Op)
            // Example deserializing network operation
            auto RequestObject = Schema_GetCommandRequestObject(Op.request.schema_type);
            uint32 TextLength = Schema_GetBytesLength(RequestObject, FieldId);
            const uint8* Text = Schema_GetBytes(RequestObject, FieldId);
            auto MyString = FString(TextLength, ANSI_TO_TCHAR(reinterpret_cast<const char*>(Text)))
            // do something with MyString

            // Example serializing and sending command response
            Worker_CommandResponse Response = {};
            Response.component_id = 1338;
            Response.schema_type = Schema_CreateCommandResponse(1338, 1);
            Schema_Object* response_object = Schema_GetCommandResponseObject(Response.schema_type);
            FString text = "Here's my response.";
            Schema_AddBytes(response_object, 1, (const uint8_t*)TCHAR_TO_ANSI(*text), sizeof(char) * strlen(TCHAR_TO_ANSI(*text)));
            Cast<USpatialNetDriver>(GetWorld()->GetNetDriver())->Connection->SendCommandResponse(Op.request_id, &Response);
        });
    });
}

Adding to snapshot example

You could add a new entity with the given component in your Unreal project code in the following way:

UCLASS()
class UTestEntitySnapshotGeneration : public USnapshotGenerationTemplate
{
    GENERATED_BODY()

public:
    bool WriteToSnapshotOutput(Worker_SnapshotOutputStream* OutputStream, Worker_EntityId& NextEntityId) override {
        Worker_Entity TestEntity;
        TestEntity.entity_id = NextEntityId;

        TArray<Worker_ComponentData> Components;

        const WorkerAttributeSet TestWorkerAttributeSet{ TArray<FString>{TEXT("test_attribute")} };
        const WorkerRequirementSet TestWorkerPermission{ TestWorkerAttributeSet };
        const WorkerRequirementSet AnyWorkerPermission{ {SpatialConstants::UnrealClientAttributeSet, SpatialConstants::UnrealServerAttributeSet, TestWorkerAttributeSet } };

        WriteAclMap ComponentWriteAcl;
        ComponentWriteAcl.Add(SpatialConstants::POSITION_COMPONENT_ID, SpatialConstants::UnrealServerPermission);
        ComponentWriteAcl.Add(SpatialConstants::METADATA_COMPONENT_ID, SpatialConstants::UnrealServerPermission);
        ComponentWriteAcl.Add(SpatialConstants::PERSISTENCE_COMPONENT_ID, SpatialConstants::UnrealServerPermission);
        ComponentWriteAcl.Add(SpatialConstants::ENTITY_ACL_COMPONENT_ID, SpatialConstants::UnrealServerPermission);
        ComponentWriteAcl.Add(1337, TestWorkerPermission);
        ComponentWriteAcl.Add(1338, SpatialConstants::UnrealServerPermission);

        // Serialize NonUnrealAuthoritative component data
        Worker_ComponentData NonUnrealAuthoritativeComponentData{};
        NonUnrealAuthoritativeComponentData.component_id = 1337;
        NonUnrealAuthoritativeComponentData.schema_type = Schema_CreateComponentData(1337);
        Schema_Object* NonUnrealAuthoritativeComponentDataObject = Schema_GetComponentDataFields(NonUnrealAuthoritativeComponentData.schema_type);
        Schema_AddInt32(NonUnrealAuthoritativeComponentDataObject, 1, 1); // set counter field to 1 initially

        // Serialize FromUnreal component data
        Worker_ComponentData UnrealAuthoritativeComponentData{};
        UnrealAuthoritativeComponentData.component_id = 1338;
        UnrealAuthoritativeComponentData.schema_type = Schema_CreateComponentData(1338);
        Schema_Object* UnrealAuthoritativeComponentDataObject = Schema_GetComponentDataFields(UnrealAuthoritativeComponentData.schema_type);
        Schema_AddInt32(UnrealAuthoritativeComponentDataObject, 1, 1); // set other_counter field to 1 initially

        Components.Add(SpatialGDK::Position(SpatialGDK::Origin).CreatePositionData());
        Components.Add(SpatialGDK::Metadata(TEXT("TestEntity")).CreateMetadataData());
        Components.Add(SpatialGDK::Persistence().CreatePersistenceData());
        Components.Add(SpatialGDK::EntityAcl(AnyWorkerPermission, ComponentWriteAcl).CreateEntityAclData());
        Components.Add(NonUnrealAuthoritativeComponentData);
        Components.Add(UnrealAuthoritativeComponentData);

        TestEntity.component_count = Components.Num();
        TestEntity.components = Components.GetData();

        bool bSuccess = Worker_SnapshotOutputStream_WriteEntity(OutputStream, &TestEntity) != 0;
        if (bSuccess)
        {
            NextEntityId++;
        }

        return bSuccess;
    }
};


————

2019-06-28 Page updated with draft content: code generator, setting up, manual (de)serialzation added
2019-04-11 Page updated with limited editorial review
2019-03-15 Page added with editorial review

Search results

Was this page helpful?

Thanks for letting us know!

Thanks for your feedback

Need more help? Ask on the forums