Sites

Menu

The JSON language and this guide for building a custom code generator is currently experimental. We’re very open to feedback - don’t hesitate to get in touch on the forums if you have any thoughts.

Building a custom code generator

This page describes the schema bundle (--bundle_out and --bundle_json_out) and abstract syntax tree JSON (--ast_json_out) language targets of the schema compiler and how to use them to implement your own custom code generator.

Usually, when you run spatial worker codegen, it will process the schema with a target language as specified in the “Codegen” task inside your worker’s *.build.json. This will generate code, which allows you, as a developer, to interact with entities and their data. Unfortunately, the code generated this way can be hard to extend programmatically.

The schema bundle and abstract syntax tree are generated by parsing the schema files of your project, and generate an abstract representation of your schema. It is then possible to build a tool, which parses the schema bundle or abstract syntax tree data and uses it to generate code in any form required. Both the schema bundle and abstract syntax tree formats are deliberately verbose to provide as much information as possible to any custom code generator tool, and not all the fields need to be used.

You can use spatial package get schema test_schema_library <sdk-version> <output zip file> to get an exhaustive schema to test your code generator and ensure that it will work correctly with different schema language constructs.

When invoking the schema compiler ensure a schema path is set to the directory containing the ‘test_schema’ and ‘dependent_schema’ directories (these directories contain the exhaustive test schema). For example when the <output zip file> is unzipped to the directory path/to/test/schema/library, use the command schema_compiler --schema_path=path/to/test/schema/library --load_all_schema_on_schema_path --ast_json_out=path/to/output.json

Integrating with the standard build process

You can add another step to the “Codegen” task in your *.build.json, for example for the abstract syntax tree format, which looks something like the following:

{
  "name": "Json AST",
  "arguments": [
    "process_schema",
    "generate",
    "--cachePath=generated/schema_codegen_cache_json",
    "--output=generated/json",
    "--language=ast_json"
  ]
}

The “process_schema” argument has the same behaviour as running the spatial schema command. The task defined above is identical to running spatial schema generate --language=ast_json and placing the resulting JSON files inside build/assembly/generated/json.

Schema bundle format

A schema bundle is a serialized protobuf message that may be a binary file (conventionally with a .sb file extension) or a JSON text file (conventionally with a .sb.json file extension) with a structure specified below.

For a single invocation of the schema-compiler with --bundle_out (for a binary schema bundle file) or --bundle_json_out (for a JSON text schema bundle file), the schema compiler parses the inputted schema files and generates a single schema bundle file which contains complete information about all of the types defined in the input files.

This guide shows the protobuf definition of each element of a schema bundle, as well as an example of a schema bundle element serialized in JSON form.

You can get the protobuf schema_bundle.proto file by running spatial package get schema schema_bundle_proto <sdk-version> <output zip file>.

SchemaBundle

A schema bundle file contains a single serialized SchemaBundle message.

A SchemaBundle is comprised of:

Field name Field type Description
schema_files List of SchemaFiles The set of schema files contained in this schema bundle.

The protobuf message definition of SchemaBundle is:

message SchemaBundle {
  repeated SchemaFile schema_files = 1;
}

For example, in .sb.json form, a SchemaBundle would be an object with a single array titled schemaFiles:

{
    "schemaFiles": [...]
}

SchemaFile

A SchemaFile consists of:

Field name Field type Description
canonical_path string The path of the source .schema filename, relative to the schema_root provided to the schema compiler.
package Package The package this schema file belongs to.
imports List of Imports. The list of import statements in the file.
types List of TypeDefinitions. The list of type {} declarations in the file.
enums List of EnumDefinitions. The list of component {} declarations in the file.

The protobuf message definition of SchemaFile is:

message SchemaFile {
  string canonical_path = 1;
  Package package = 2;
  repeated Import imports = 3;
  repeated EnumDefinition enums = 4;
  repeated TypeDefinition types = 5;
  repeated ComponentDefinition components = 6;
}

For example, in .sb.json form, this would be:

{
    "canonicalPath": "bundle_example.schema",
    "package": [...], 
    "imports": [...],
    "enums": [...],
    "types": [...],
    "components": [...]
}

Package

A Package represents a package declaration in a schema file (for example package improbable.bundle_example;).

It consists of:

Field name Field type Description
source_reference SourceReference The location of the package definition in the source .schema file.
name string The name of the package (for example improbable.bundle_example).

The protobuf message definition of Package is:

message Package {
  SourceReference source_reference = 1;
  string name = 2;
}

For example, in .sb.json form, this would be:

"package": {
    "sourceReference": {
        "line": 1,
        "column": 1
    },
    "name": "improbable.bundle_example"
}

Import

An Import describes an import statement in a schema file (for example import "improbable/standard_library.schema").

It consists of:

Field name Field type Description
source_reference SourceReference The location of the import statement in the source .schema file.
path string The path of the imported .schema file, relative to the schema_root provided to the schema compiler.

The protobuf message definition of Import is:

message Import {
  SourceReference source_reference = 1;
  string path = 2;
}

For example, in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 3,
        "column": 1
    },
    "path": "improbable/standard_library.schema"
}

SourceReference

A SourceReference specifies the location of a token in the source .schema file. This information might useful for generating error messages in your custom code generator.

It consists of:

Field name Field type Description
line uint32 A line number in a source .schema file.
column uint32 A column number of a character on a line in a source .schema file.

The protobuf message definition of SourceReference is:

message SourceReference {
  uint32 line = 1;
  uint32 column = 2;
}

For example, in .sb.json form, this would be:

"sourceReference": {
    "line": 10,
    "column": 5
}

ComponentDefinition

A ComponentDefinition describes a component definition in a .schema file.

It consists of:

Field name Field type Description
source_reference SourceReference The location of the component definition in the source .schema file.
annotations List of Annotations. The annotations defined on this component.
qualified_name string The concatenation of the name of the package in which the component was defined and the component’s name.
name string The name of the component (for example for component Foo {}, this is Foo).
component_id uint32 The ID of this component.
data_definition string The fully qualified name of the data type, if the component has one; if the component has no data type, this is "".
fields List of FieldDefinitions. The list of fields in the component. This has values only if the component has no data_definition.
events List of EventDefinitions. The list of event definitions in the component.
commands List of CommandDefinitions. The list of command definitions in the component.

The protobuf message definition of ComponentDefinition is:

message ComponentDefinition {
  SourceReference source_reference = 1;
  repeated Annotation annotations = 2;

  string qualified_name = 3;
  string name = 4;
  uint32 component_id = 5;
  // Fully-qualified name of the data type (for example "improbable.example.Foo"). This is
  // set to the default value ("") if the component has no 'data' type.
  string data_definition = 6;
  // Should have values only if the component has fields instead of a 'data' type (and
  // 'data_definition' is set to the default value).
  repeated FieldDefinition fields = 7;
  repeated EventDefinition events = 8;
  repeated CommandDefinition commands = 9;
}

For example, in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 51,
        "column": 1
    },
    "annotations": [],
    "qualifiedName": "improbable.bundle_example.Foo",
    "name": "Foo",
    "componentId": 1234,
    "dataDefinition": "",
    "fields": [...],
    "events": [...],
    "commands": [...]
}

ComponentDefinition.EventDefinition

EventDefintion is defined within ComponentDefinition; it represents an event inside a component (for example event int32 foo_event;).

It consists of:

Field name Field type Description
source_reference SourceReference The location of the event definition in the source .schema file.
annotations List of Annotations. The annotations defined on this component.
name string The name of the event (for example foo_event).
type string The fully-qualified name of the event’s type (for example improbable.bundle_example.FooEvent)
event_index uint32 The 1-based position of the event in the order events appear in the component {} definition.

The protobuf message definition for EventDefintion is:

message EventDefinition {
  SourceReference source_reference = 1;
  repeated Annotation annotations = 2;

  string name = 3;
  // Fully-qualified name of the event type (for example "improbable.example.Foo").
  string type = 4;
  uint32 event_index = 5;
}

For example, in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 53,
        "column": 3
    },
    "annotations": [],
    "name": "foo_event",
    "type": "improbable.bundle_example.FooEvent",
    "eventIndex": 1
}

ComponentDefinition.CommandDefinition

CommandDefinition is defined within ComponentDefinition; it represents a command inside a component, such as command int32 example_command(improbable.example.ExampleType);

It consists of:

Field name Field type Description
source_reference SourceReference The location of the command definition in the source .schema file.
annotations List of Annotations. The annotations defined on this command.
name string The name of the command (for example example_command).
request_type string The fully-qualified name of the request type (for example improbable.bundle_example.FooRequest).
response_type string The fully-qualified name of the response type (for example improbable.bundle_example.FooResponse).
command_index uint32 The 1-based position of the command in the order commands appear in the component {} definition.

The protobuf message definition of CommandDefinition is:

message CommandDefinition {
  SourceReference source_reference = 1;
  repeated Annotation annotations = 2;

  string name = 3;
  // Fully-qualified name of the request type (for example "improbable.example.Foo").
  string request_type = 4;
  // Fully-qualified name of the response type (for example "improbable.example.Foo").
  string response_type = 5;
  uint32 command_index = 6;
}

For example, in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 51,
        "column": 1
    },
    "annotations": [],
    "name": "foo_command",
    "requestType": "improbable.bundle_example.FooResponse",
    "responseType": "improbable.bundle_example.FooResponse",
    "commandIndex": 1
}

TypeDefinition

TypeDefinition represents a type {} block in a .schema file.

It consists of:

Field name Field type Description
source_reference SourceReference The location of the type definition in the source .schema file.
annotations List of Annotations. The annotations defined on the type.
qualified_name string The concatenation of the name of the package in which the type is defined and the type’s name.
name string The name of the type (for example, for type Foo {}, this is Foo).
outer_type string The fully-qualified name of the type containing this type, if this is a nested type. If this is not a nested type, this field is "".
fields List of FieldDefinitions. The fields defined in this type.

The protobuf message definition of TypeDefinition is:

message TypeDefinition {
  SourceReference source_reference = 1;
  repeated Annotation annotations = 2;

  string qualified_name = 3;
  string name = 4;
  // Fully-qualified name of the outer type if applicable (for example "improbable.example.Foo"). This  
  // is set to the default value ("") if this type has no outer type.
  string outer_type = 5;
  repeated FieldDefinition fields = 6;
}

For example, in .sb.json form this would be:

{
    "sourceReference": {
        "line": 18,
        "column": 1
    },
    "annotations": [],
    "qualifiedName": "improbable.bundle_example.FooResponse",
    "name": "FooResponse",
    "outerType": "",
    "fields": [
        {
            "sourceReference": {
                "line": 19,
                "column": 5
            },
            "annotations": [],
            "name": "foo_message",
            "fieldId": 1,
            "transient": false,
            "singularType": {
                "type": {
                    "primitive": "String"
                }
            }
        }
    ]
}

FieldDefinition

A FieldDefinition represents a single field in a component or type, such as int32 some_field = 1;.

It consists of:

Field name Field type Description
source_reference SourceReference The location of the field definition in the source .schema file.
annotations List of Annotations. The annotations defined on the field.
name string The name of the field (for example some_field).
field_id uint32 The ID of this field (for example 1).
transient bool True if this field was defined with the transient keyword, false otherwise.
type oneof A reference to the type of this field.

type can contain exactly one of:

Field name Field type Description
singular_type SingularType If this field is a non-collection type.
option_type OptionType If this field is an option<>.
list_type ListType If this field is a list<>.
map_type MapType If this field is a map<>.

The protobuf message definition of FieldDefinition is:

message FieldDefinition {
  SourceReference source_reference = 1;
  repeated Annotation annotations = 2;

  string name = 3;
  uint32 field_id = 4;
  bool transient = 5;

  oneof type {
    SingularType singular_type = 6;
    OptionType option_type = 7;
    ListType list_type = 8;
    MapType map_type = 9;
  }
}

For example, in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 83,
        "column": 3
    },
    "annotations": [],
    "name": "coords",
    "fieldId": 1,
    "transient": false,
    "singularType": {
        "type": {
            "type": "improbable.Coordinates"
        }
    }
}

FieldDefinition.SingularType

SingularType is defined within FieldDefinition; it represents a type reference of a non-collection field. For example the FieldDefinition for int32 foo = 1; would contain a SinglularType with a TypeReference to int32.

Field name Field type Description
type TypeReference The type of this non-collection field.

The protobuf message definition of SingularType is:

message SingularType {
  TypeReference type = 1;
}

For example, in .sb.json form, this would be:

"optionType": {
    "type": {
        "primitive": "Int32"
    }
}

FieldDefinition.OptionType

OptionType is defined within FieldDefinition; it represents a type reference of an optional field. For example, the FieldDefinition for option<int32> foo_option = 1; would contain an OptionType with a TypeReference to int32.

Field name Field type Description
inner_type TypeReference The type reference of this optional type.

The protobuf message definition of OptionType is:

message OptionType {
  TypeReference inner_type = 1;
}

For example, in .sb.json form, this would be:

"optionType": {
    "innerType": {
        "primitive": "Int32"
    }
}

FieldDefinition.ListType

ListType is defined within FieldDefinition; it represents a type reference of a list field. For example, the FieldDefinition for list<int32> foo_list = 2; would contain a ListType with a TypeReference to int32.

Field name Field type Description
inner_type ListType The type reference of this list type.

The protobuf message definition of ListType is:

message ListType {
  TypeReference inner_type = 1;
}

For example, in .sb.json form, this would be:

"listType": {
    "innerType": {
        "primitive": "Int32"
    }
}

FieldDefinition.MapType

MapType is defined within FieldDefinition; it represents the type reference of a map field. For example, the FieldDefinition for map<int32> foo_map = 1; would contain a MapType with a TypeReference to int32.

Field name Field type Description
key_type ListType The type reference of this keys of this map type.
value_type ListType The type reference of this values of this map type.

The protobuf message definition of MapType is:

message MapType {
  TypeReference key_type = 1;
  TypeReference value_type = 2;
}

For example, in .sb.json form, this would be:

"mapType": {
    "keyType": {
        "primitive": "String"
    },
    "valueType": {
        "primitive": "Int32"
    }
}

EnumDefinition

An EnumDefinition represents an enum {} block in a .schema file. For example, enum FooEnum {BAR = 1, BAZ = 2};

It consists of:

Field name Field type Description
source_reference SourceReference The location of the enum {} definition in the source .schema file.
annotations List of Annotations. The list of annotations defined on this enum.
name string The name of the enum (for example FooEnum).
qualified_name string The fully-qualified name of this enum; in other words, the concatenation of the name of the package in which this enum is defined, any outer types in which this enum is defined, and the enum’s name.
outer_type string The fully-qualified name of the type in which the enum is defined, if it is a nested enum.
values List of EnumValueDefinitions. The set of values in this enum.

The protobuf message definition for EnumDefinition is:

message EnumDefinition {
  SourceReference source_reference = 1;
  repeated Annotation annotations = 2;

  string qualified_name = 3;
  string name = 4;
  // Fully-qualified name of the outer type if applicable (for example "improbable.example.Foo"). This
  // is set to the default value ("") if this enum has no outer type.
  string outer_type = 5;
  repeated EnumValueDefinition values = 6;
}

For example, in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 5,
        "column": 1
    },
    "annotations": [],
    "qualifiedName": "improbable.bundle_example.SomeEnum",
    "name": "SomeEnum",
    "outerType": "",
    "values": [
        {
            "sourceReference": {
                "line": 6,
                "column": 3
            },
            "annotations": [],
            "name": "FOO",
            "value": 1
        },
        {
            "sourceReference": {
                "line": 7,
                "column": 3
            },
            "annotations": [],
            "name": "BAR",
            "value": 2
        }
    ]
}

EnumValueDefinition

EnumValueDefinition represents a value in an enum; for example enum FooEnum {BAR = 1, BAZ = 2}; contains two enum values, BAR and BAZ.

It consists of:

Field name Field type Description
source_reference SourceReference The location where this enum value was defined in the source .schema file.
annotations List of Annotations. The list of annotations defined on this enum value.
name string The name of this enum value (for example BAR).
value int32 The value of this enum (for example 1).

The protobuf message definition for EnumValueDefinition is:

message EnumValueDefinition {
  SourceReference source_reference = 1;
  repeated Annotation annotations = 2;

  string name = 3;
  uint32 value = 4;
}

For example, in .sb.json form, this would be:

"values": [
    {
        "sourceReference": {
            "line": 6,
            "column": 3
        },
        "annotations": [],
        "name": "BAZ",
        "value": 1
    }
]

Annotation

An annotation represents a single type value (an instance of a user-defined type) annotated on a schema definition, such as [ExampleType(1)]. Any user-defined schema type can be instantiated. Instantiations of the individual fields correspond to their respective schema types.

Annotations are comprised of:

Field name Field type Description
source_reference SourceReference The location of this annotation in the source .schema file.
type_value Value.TypeValue The type value of this annotation.

The protobuf message definition of Annotation is:

message Annotation {
  SourceReference source_reference = 1;
  Value.TypeValue type_value = 2;
}

For example in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 44,
        "column": 1
    },
    "typeValue": {...}
}

Value

A Value corresponds to an instance of a type in an annotation on schema definition.

It consists of:

Field name Field type Description
source_reference SourceReference The location of this value in the source .schema file.
value oneof The value of this value.

The field value can contain exactly one of the following:

Field name Field type Description
bool_value bool An instance of a boolean value.
uint32_value uint32 An instance of a uint32 value.
uint64_value uint64 An instance of a uint64 value.
float_value float An instance of a float value.
double_value double An instance of a double value.
bytes_value bytes An instance of a bytes value.
string_value string An instance of a string value.
entity_id_value int64 An instance of an entity ID value.
enum_value EnumValue An instance of an enum value.
type_value TypeValue An instance of a user-defined type.
option_value OptionValue An instance of an option.
list_value ListValue An instance of a list.
map_value MapValue An instance of a map.

The protobuf message definition of Value is:

message Value {

  SourceReference source_reference = 1;

  oneof value {
    bool bool_value = 2;
    uint32 uint32_value = 3;
    uint64 uint64_value = 4;
    int32 int32_value = 5;
    int64 int64_value = 6;
    float float_value = 7;
    double double_value = 8;
    string string_value = 9;
    bytes bytes_value = 10;
    int64 entity_id_value = 11;

    EnumValue enum_value = 12;
    TypeValue type_value = 13;

    OptionValue option_value = 14;
    ListValue list_value = 15;
    MapValue map_value = 16;
  }
}

For example, in .sb.json form, this would be:

"value": {
    "sourceReference": {
        "line": 40,
        "column": 14
    },
    "boolValue": true
}

Value.EnumValue

EnumValue is defined within Value; it represents an instance of an enum value.

It consists of:

Field name Field type Description
enum string The fully-qualified name of the the enum which this value is an instance of.
value string The name of the value contained within the enum.

The protobuf message definition for EnumValue is:

message EnumValue {
  // Fully-qualified name of the enum (for example "improbable.example.Bar").
  string enum = 1;
  // The name of the value contained within the enum (for example "ALPHA").
  string value = 2;
}

For example, in .sb.json form, this would take the form:

"enumValue": {
    "enum": "improbable.bundle_example.SomeEnum",
    "value": "FOO"
}

Value.TypeValue

TypeValue is defined within Value; it represents an instance of a user-defined type.

It consists of:

Field name Field type Description
type string The fully-qualified name of the instantiated type (for example improbable.example.Foo).
fields List of FieldValues. The list of values in the fields of the instance of the type.

The protobuf message definition of TypeValue is:

message TypeValue {
  // Fully-qualified name of the instantiated type (for example "improbable.example.Foo").
  string type = 1;
  repeated FieldValue fields = 2;
}

For example, in .sb.json form, this would be:

"typeValue": {
    "type": "improbable.example.Foo",
    "fields": [...]
}
Value.TypeValue.FieldValue

FieldValue is defined within Value; it represents an instance of a field contained within a instance of a user-defined type.

It consists of:

Field name Field type Description
source_reference SourceReference The location of the field value in the source .schema file.
name string The name of the field contained within the type (for example some_field).
value Value The value of the instance of the field.

The protobuf message definition of FieldValue is:

message FieldValue {
  SourceReference source_reference = 1;

  // The name of the field contained within the type (for example "some_field").
  string name = 2;
  Value value = 3;
}

For example, in .sb.json form, this would be:

{
    "sourceReference": {
        "line": 40,
        "column": 14
    },
    "name": "some_field",
    "value": {...}
}

Value.OptionValue

OptionValue is defined within Value; it represents an instance of an option value.

It consists of:

Field name Field type Description
value Value The value contained within the option, if the option is empty this field is not set.

The protobuf message definition of OptionValue is:

message OptionValue {
  // Optional. Can be unset to indicate an empty option.
  Value value = 1;
}

For example, in .sb.json form, this would be:

"optionValue": {
    "value": {
        "sourceReference": {
            "line": 0,
            "column": 0
        },
        "int32Value": 1
    }
}

Value.ListValue

ListValue is defined within Value; it represents an instance of a list value.

It consists of:

Field name Field type Description
values List of values. The values contained within the list.

The protobuf definition of ListValue is:

message ListValue {
  repeated Value values = 1;
}

For example, in .sb.json form, this would be:

"listValue": {
    "values": [
        {
            "sourceReference": {
                "line": 47,
                "column": 46
            },
            "int32Value": 1
        },
        {
            "sourceReference": {
                "line": 47,
                "column": 49
            },
            "int32Value": 2
        }
    ]
}

Value.MapValue

MapValue is defined within Value; it represents an instance of a map value.

It consists of:

Field name Field type Description
values List of KeyValuePairs. The list of (key, value) pair entries in the map instance.

The protobuf message definition of MapValue is:

message MapValue {
  repeated KeyValuePair values = 1;
}

For example, in .sb.json form, this would be:

"mapValue": {
    "values": [...]
}
Value.MapValue.KeyValuePair

KeyValuePair is defined within MapValue.Value; it represents an instance of a key and value in a map.

It consists of:

Field name Field type Description
key Value The value of a key of an entry in a map instance.
value Value The value of the value of an entry in a map instance.

The protobuf message definition of KeyValuePair is:

message KeyValuePair {
  Value key = 1;
  Value value = 2;
}

In .sb.json form, this would be:

{
    "key": {
        "sourceReference": {
            "line": 47,
            "column": 66
        },
        "stringValue": "foo"
    },
    "value": {
        "sourceReference": {
            "line": 47,
            "column": 72
        },
        "int32Value": 1
    }
}

TypeReference

A TypeReference is a reference to a schema type.

Field name Field type Description
value_type oneof The value of the type reference.

value_type can contain exactly one of:

Field name Field type Description
primitive PrimitiveType A reference to one of the primitive schema types.
enum string The fully-qualified name of the enum which is being referenced.
type string The fully-qualified name of the user-defined type which is being referenced.

The protobuf message definition of TypeReference is:

message TypeReference {
  oneof value_type {
    PrimitiveType primitive = 1;
    // Fully-qualified name of the enum (for example "improbable.example.Bar").
    string enum = 2;
    // Fully-qualified name of the type (for example "improbable.example.Foo").
    string type = 3;
  }
}

For example, in .sb.json form this would be:

For a primitive type:

"type": {
    "primitive": "Int32"
}

For an enum:

"type": {
    "enum": "improbable.bundle_example.SomeEnum"
}

For a type:

"type": {
    "type": "improbable.example.Foo"
}

PrimitiveType

PrimitiveType is an enum used to encode a primitive schema type; it is an enum with 18 options as listed below:

Name Value Description
Invalid 0 Used to indicate an invalid primitive type.
Int32 1 A 32 bit integer primitive type.
Int64 2 A 64 bit integer primitive type.
Uint32 3 An unsigned 32 bit integer primitive type.
Uint64 4 An unsigned 64 bit integer primitive type.
Sint32 5 A zig-zag encoded signed 32 bit integer primitive type.
Sint64 6 A zig-zag encoded signed 64 bit integer primitive type.
Fixed32 7 A fixed-width encoded 32 bit integer primitive type.
Fixed64 8 A fixed-width encoded 64 bit integer primitive type.
SFixed32 9 A fixed-width encoded signed 32 bit integer primitive type.
SFixed64 10 A fixed-width encoded signed 64 bit integer primitive type.
Bool 11 A boolean (true or false) primitive type.
Float 12 A 32 bit floating point primitive type.
Double 13 A double-precision (64 bit) floating point primitive type.
String 14 An ASCII or UTF-8 string of characters.
EntityId 15 An int64 type used to encode entity IDs.
Bytes 16 A string of bytes.
Entity 17 A data structure that represents an arbitrary set of components.

The protobuf enum definition of PrimitiveType is:

enum PrimitiveType {
  Invalid = 0;
  Int32 = 1;
  Int64 = 2;
  Uint32 = 3;
  Uint64 = 4;
  Sint32 = 5;
  Sint64 = 6;
  Fixed32 = 7;
  Fixed64 = 8;
  Sfixed32 = 9;
  Sfixed64 = 10;
  Bool = 11;
  Float = 12;
  Double = 13;
  String = 14;
  EntityId = 15;
  Bytes = 16;
  Entity = 17;
}

Abstract syntax tree format

A generated abstract syntaxt tree (AST) is a *.json file with a structure specified below. For every *.schema file, exactly one *.json file will be generated. Each object in the tree (except a type reference) is tagged with a source reference, which gives information that may be useful for formatting error messages.

Schema file

The root of the AST specifies information about the schema file itself.

{
    "sourceReference": {...},
    "completePath": "./schema/improbable/example.schema",
    "canonicalName": "improbable/example.schema",
    "package": "improbable.example",
    "enumDefinitions": [...],
    "typeDefinitions": [...],
    "componentDefinitions": [...]
}
  • sourceReference is the source reference of this node.
  • completePath specifies the complete path to the schema file including the prefix specified by the --input flag. If the path specified in the --input flag is relative, then completePath will also be relative, otherwise, it will be an absolute path.
  • canonicalName is identical to completePath, minus the prefix specified in the --input flag.
  • package is the package name defined at the top of the schema file.
  • enumDefinitions is an array of enum definitions.
  • typeDefinitions is an array of type definitions.
  • componentDefinitions is an array of component defintions.

Source reference

A source reference specifies the location of the token that was parsed into the parent node.

"sourceReference": {
    "line": 2,
    "column": 1
}
  • line is a number specifying the line number of the node in the schema file.
  • column is a number specifying the column number of the character on the line, which corresponds with the beginning of the node.

Component definition

A component defintion represents a component {} block in the schema file.

{
    "sourceReference": {...},
    "name": "ExampleComponent",
    "qualifiedName": "improbable.example.ExampleComponent",
    "id": 1001,
    "dataDefinition": {...},
    "eventDefinitions": [...],
    "commandDefinitions": [...],
    "annotations": [...],
}
  • sourceReference is the source reference of this node.
  • name is a string specifying the name of the component.
  • qualifiedName is a string specifying the fully-qualified name of the component, which includes its package.
  • id is a number specifying the ID of the component.
  • dataDefinition is a type reference, which will refer to the type specified in the data field of the component. If the data structure of the component is instead specified inline with no data field, then a new type definition will be generated with the same name of the component, with Data appended to it, which will be specified here. See Reusable data types for more information.
  • eventDefinitions is an array of event definitions.
  • commandDefinitions is an array of command definitions.
  • annotations is an array of annotations.

Event definition

An event definition represents an event inside a component, such as event int32 example_event;.

{
    "sourceReference": {...},
    "name": "example_event",
    "type": {...},
    "eventIndex": 1,
    "annotations": [...]
}
  • sourceReference is the source reference of this node.
  • name is a string specifying the name of the event.
  • type is a type reference specifying the type of the data stored in the event.
  • eventIndex is a number specifying the 1-based position of the event in the order events appear in the schema
  • annotations is an array of annotations.

Command definition

A command definition represents a command inside a component, such as command int32 example_command(improbable.example.ExampleType);.

{
    "sourceReference": {...},
    "name": "example_command",
    "requestType": {...},
    "responseType": {...},
    "commandIndex": 1,
    "annotations": [...]
}
  • sourceReference is the source reference of this node.
  • name is a string specifying the name of the command.
  • requestType is a type reference specifying the type of the command request.
  • responseType is a type reference specifying the type of the command response.
  • commandIndex is a number specifying the 1-based position of the command in the order commands appear in the schema
  • annotations is an array of annotations.

Type reference

A type reference is a JSON object, which specifies a reference to either a built-in type (like int32) or a user type (like improbable.example.ExampleType) and includes a source reference.

{
    "sourceReference": {...},
    "builtInType": "int32"
}
{
    "sourceReference": {...},
    "userType": "improbable.example.ExampleType"
}

Type definition

A type definition represents a type {} block in the schema file.

{
    "sourceReference": {...},
    "name": "ExampleType",
    "qualifiedName": "improbable.example.ExampleType",
    "enumDefinitions": [...],
    "typeDefinitions": [...],
    "fieldDefinitions": [...],
    "annotations": [...]
}
  • sourceReference is the source reference of this node.
  • name is a string specifying the name of this type.
  • qualifiedName is a string specifying the fully qualified name of this type, which includes its package and any parent types.
  • enumDefinitions is an array of enum definitions.
  • typeDefinitions is an array of type definitions.
  • fieldDefinitions is an array of field definitions.
  • annotations is an array of annotations.

Enum definition

An enum definition represents an enum {} block in the schema file.

{
    "sourceReference": {...},
    "name": "TestEnum",
    "qualifiedName": "improbable.Test.TestEnum",
    "valueDefinitions": [
        {
            "sourceReference": {...},
            "name": "SOME_VALUE",
            "value": 0,
            "annotations": [...]
        },
        {
            "sourceReference": {...},
            "name": "SOME_OTHER_VALUE",
            "value": 1,
            "annotations": [...]
        }
    ],
    "annotations": [...]
}
  • sourceReference is the source reference of this node.
  • name is a string specifying the name of this enum.
  • qualifiedName is a string specifying the fully qualified name of this enum, which includes its package and any parent types.
  • valueDefinitions is an array of objects, which consist of three fields:
    • sourceReference is the source reference of this value.
    • name is a string specifying the name of this enum value.
    • value is a number specifying the value, which this enum value represents.
    • annotations is an array of annotations.
  • annotations is an array of annotations.

Field definition

A field definition represents a single field in a component or a type, such as int32 some_field = 1;. The field definition will have exactly one of the following fields: singularType, optionType, listType and mapType.

A field with a single type:

{
    "sourceReference": {...},
    "name": "some_field",
    "number": 1,
    "singularType": {...},
    "annotations": [...]
}

A field with an option type:

{
    "sourceReference": {...},
    "name": "some_option",
    "number": 2,
    "optionType": {
        "valueType": {...}
    },
    "annotations": [...]
}

A field with a list type:

{
    "sourceReference": {...},
    "name": "some_list",
    "number": 3,
    "listType": {
        "valueType": {...}
    },
    "annotations": [...]
}

A field with a map type:

{
    "sourceReference": {...},
    "name": "some_map",
    "number": 4,
    "mapType": {
        "keyType": {...},
        "valueType": {...}
    },
    "annotations": [...]
}
  • name is a string specifying the name of the field.
  • number is a number specifying the field number.
  • singularType is a type reference specifying the type of the field (if it’s a “simple” field).
  • optionType is a JSON object specifying the type of the field (if it’s an option type). It contains a valueType, which is a type reference specifying the type stored in the option.
  • listType is a JSON object specifying the type of the field (if it’s a list type). It contains a valueType, which is a type reference specifying the type of each item in the list.
  • mapType is a JSON object specifying the type of the field (if it’s a map type). It contains:
    • keyType, which is a type reference specifying the key type of the map.
    • valueType, which is a type reference specifying the value type of the map.
  • annotations is an array of annotations.

Annotations

An annotation represents a single type value (an instance of a user-defined type) annotated on a schema definition, such as [ExampleType(1)]. Any user-defined schema type can be instantiated. Instantiations of the individual fields correspond to their respective schema types. Each instantiation of a field contains a value.

Inside a value, one of the following may be set.

  • boolValue for bool fields.
  • uint32Value for uint32 and fixed32 fields.
  • uint64Value for uint64 and fixed64 fields.
  • int32Value for int32, sfixed32 and sint32 fields.
  • int64Value for int64, sfixed64 and sint64 fields.
  • floatValue for float fields.
  • doubleValue for double fields.
  • stringValue for string fields (note that these string values can make use of the fact that JSON files are UTF-8 encoded, as well as the esape sequences supported by JSON).
  • bytesValue for bytes fields (base64-encoded).
  • entityIdValue for EntityId fields.
  • enumValue for user-defined enum fields.
  • typeValue for user-defined schema type fields.
  • optionValue for option<...> fields.
  • listValue for list<...> fields.
  • mapValue for map<...> fields.

An annotation using a type with fields of all of the above:

{
    "sourceReference": {...},
    "typeValue": {
        "sourceReference": {...},
        "type": "improbable.example.ExampleType",
        "fields": [
            {
                "sourceReference": {...},
                "name": "bool_field",
                "number": 1,
                "value": {
                    "sourceReference": {...},
                    "boolValue": true
                }
            },
            {
                "sourceReference": {...},
                "name": "uint32_field",
                "number": 2,
                "value": {
                    "sourceReference": {...},
                    "uint32Value": 0
                }
            },
            {
                "sourceReference": {...},
                "name": "uint64_field",
                "number": 3,
                "value": {
                    "sourceReference": {...},
                    "uint64Value": "100"
                }
            },
            {
                "sourceReference": {...},
                "name": "int32_field",
                "number": 4,
                "value": {
                    "sourceReference": {...},
                    "int32Value": -1
                }
            },
            {
                "sourceReference": {...},
                "name": "int64_field",
                "number": 5,
                "value": {
                    "sourceReference": {...},
                    "int64Value": "-500"
                }
            },
            {
                "sourceReference": {...},
                "name": "float_field",
                "number": 6,
                "value": {
                    "sourceReference": {...},
                    "floatValue": 0.5
                }
            },
            {
                "sourceReference": {...},
                "name": "double_field",
                "number": 7,
                "value": {
                    "sourceReference": {...},
                    "doubleValue": -15.5
                }
            },
            {
                "sourceReference": {...},
                "name": "string_field",
                "number": 8,
                "value": {
                    "sourceReference": {...},
                    "stringValue": "foo"
                }
            },
            {
                "sourceReference": {...},
                "name": "bytes_field",
                "number": 9,
                "value": {
                    "sourceReference": {...},
                    "bytesValue": "YmFy"
                }
            },
            {
                "sourceReference": {...},
                "name": "entity_id_field",
                "number": 10,
                "value": {
                    "sourceReference": {...},
                    "entityIdValue": "100"
                }
            },
            {
                "sourceReference": {...},
                "name": "enum_field",
                "number": 11,
                "value": {
                    "sourceReference": {...},
                    "enumValue": {
                        "enum": "improbable.example.ExampleEnum",
                        "name": "FOO",
                        "value": 0
                    }
                }
            },
            {
                "sourceReference": {...},
                "name": "type_field",
                "number": 12,
                "value": {
                    "sourceReference": {...},
                    "typeValue": {
                        "sourceReference": {...},
                        "type": "improbable.example.SomeType",
                        "fields": [...]
                    }
                }
            },
            {
                "sourceReference": {...},
                "name": "option_field",
                "number": 13,
                "value": {
                    "sourceReference": {...},
                    "optionValue": {
                        "value": {
                            "sourceReference": {...},
                            "boolValue": true
                        }
                    }
                }
            },
            {
                "sourceReference": {...},
                "name": "list_field",
                "number": 14,
                "value": {
                    "sourceReference": {...},
                    "listValue": {
                        "values": [
                            {
                                "sourceReference": {...},
                                "boolValue": true
                            }
                        ]
                    }
                }
            },
            {
                "sourceReference": {...},
                "name": "map_field",
                "number": 15,
                "value": {
                    "sourceReference": {...},
                    "mapValue": {
                        "values": [
                            {
                                "key": {
                                    "sourceReference": {...},
                                    "boolValue": true
                                },
                                "value": {
                                    "sourceReference": {...},
                                    "boolValue": true
                                }
                            }
                        ]
                    }
                }
            }
        ]
    }
}

Search results

Was this page helpful?

Thanks for letting us know!

Thanks for your feedback

Need more help? Ask on the forums