Building a custom code generator

This page describes the JSON language target in spatial schema and how to use it to implement your own custom code generator.

Usually, when you run spatial worker codegen, it will process the schema with a target language as specified in the “Codegen” task inside your worker’s *.build.json. This will generate code, which allows you, as a developer, to interact with entities and their data. Unfortunately, the code generated this way can be hard to extend programmatically. There’s now a new language option ast_json, which will parse the schema files, and generate an abstract syntax tree (AST) for every schema file in JSON format.

With the provided ASTs, it is then possible to build a tool, which parses the JSON data and uses it to generate code in any form required. The JSON is deliberately verbose to provide as much information as possible to any custom code generator tool, and not all the fields in the AST need to be used.

Integrating with the standard build process

You can add another step to the “Codegen” task in your *.build.json, which looks something like the following:

{
  "name": "Json AST",
  "arguments": [
    "process_schema",
    "generate",
    "--cachePath=generated/schema_codegen_cache_json",
    "--output=generated/json",
    "--language=ast_json"
  ]
}

The “process_schema” argument has the same behaviour as running the spatial schema command. The task defined above is identical to running spatial schema generate --language=ast_json and placing the resulting JSON files inside build/assembly/generated/json.

Abstract Syntax Tree format

A generated abstract syntaxt tree (AST) is a *.json file with a structure specified below. For every *.schema file, exactly one *.json file will be generated. Each object in the tree (except a type reference) is tagged with a source reference, which gives information that may be useful for formatting error messages.

Schema file

The root of the AST specifies information about the schema file itself.

{
    "sourceReference": {...},
    "completePath": "./schema/improbable/example.schema",
    "canonicalName": "improbable/example.schema",
    "package": "improbable.example",
    "enumDefinitions": [...],
    "typeDefinitions": [...],
    "componentDefinitions": [...]
}

• sourceReference is the source reference of this node.
• completePath specifies the complete path to the schema file including the prefix specified by the --input flag. If the path specified in the --input flag is relative, then completePath will also be relative, otherwise, it will be an absolute path.
• canonicalName is identical to completePath, minus the prefix specified in the --input flag.
• package is the package name defined at the top of the schema file.
• enumDefinitions is an array of enum definitions.
• typeDefinitions is an array of type definitions.
• componentDefinitions is an array of component defintions.

Source reference

A source reference specifies the location of the token that was parsed into the parent node.

"sourceReference": {
    "line": 2,
    "column": 1
}

• line is a number specifying the line number of the node in the schema file.
• column is a number specifying the column number of the character on the line, which corresponds with the beginning of the node.

Component definition

A component defintion represents a component {} block in the schema file.

{
    "sourceReference": {...},
    "name": "ExampleComponent",
    "qualifiedName": "improbable.example.ExampleComponent",
    "id": 1001,
    "dataDefinition": {...},
    "eventDefinitions": [...],
    "commandDefinitions": [...]
}

• sourceReference is the source reference of this node.
• name is a string specifying the name of the component.
• qualifiedName is a string specifying the fully-qualified name of the component, which includes its package.
• id is a number specifying the ID of the component.
• dataDefinition is a type reference, which will refer to the type specified in the data field of the component. If the data structure of the component is instead specified inline with no data field, then a new type definition will be generated with the same name of the component, with Data appended to it, which will be specified here. See Reusable data types for more information.
• eventDefinitions is an array of event definitions.
• commandDefinitions is an array of command definitions.

Event definition

An event definition represents an event inside a component, such as event int32 example_event;.

{
    "sourceReference": {...},
    "name": "example_event",
    "type": {...},
    "eventIndex": 1
}

• sourceReference is the source reference of this node.
• name is a string specifying the name of the event.
• type is a type reference specifying the type of the data stored in the event.
• eventIndex is a number specifying the 1-based position of the event in the order events appear in the schema

Command definition

A command definition represents a command inside a component, such as command int32 example_command(improbable.example.ExampleType);.

{
    "sourceReference": {...},
    "name": "example_command",
    "requestType": {...},
    "responseType": {...},
    "commandIndex": 1
}

• sourceReference is the source reference of this node.
• name is a string specifying the name of the command.
• requestType is a type reference specifying the type of the command request.
• responseType is a type reference specifying the type of the command response.
• commandIndex is a number specifying the 1-based position of the command in the order commands appear in the schema

Type reference

A type reference is a JSON object, which specifies a reference to either a built-in type (like int32) or a user type (like improbable.example.ExampleType) and includes a source reference.

{
    "sourceReference": {...},
    "builtInType": "int32"
}

{
    "sourceReference": {...},
    "userType": "improbable.example.ExampleType"
}

Type definition

A type definition represents a type {} block in the schema file.

{
    "sourceReference": {...},
    "name": "ExampleType",
    "qualifiedName": "improbable.example.ExampleType",
    "enumDefinitions": [...],
    "typeDefinitions": [...],
    "fieldDefinitions": [...]
}

• sourceReference is the source reference of this node.
• name is a string specifying the name of this type.
• qualifiedName is a string specifying the fully qualified name of this type, which includes its package and any parent types.
• enumDefinitions is an array of enum definitions.
• typeDefinitions is an array of type definitions.
• fieldDefinitions is an array of field definitions.

Enum definition

An enum definition represents an enum {} block in the schema file.

{
    "sourceReference": {...},
    "name": "TestEnum",
    "qualifiedName": "improbable.Test.TestEnum",
    "valueDefinitions": [
        {
            "sourceReference": {...},
            "name": "SOME_VALUE",
            "value": 0
        },
        {
            "sourceReference": {...},
            "name": "SOME_OTHER_VALUE",
            "value": 1
        }
    ]
}

• sourceReference is the source reference of this node.
• name is a string specifying the name of this enum.
• qualifiedName is a string specifying the fully qualified name of this enum, which includes its package and any parent types.
• valueDefinitions is an array of objects, which consist of three fields:
  • sourceReference is the source reference of this value.
  • name is a string specifying the name of this enum value.
  • value is a number specifying the value, which this enum value represents.

Field definition

A field definition represents a single field in a component or a type, such as int32 some_field = 1;. The field definition will have exactly one of the following fields: singularType, optionType, listType and mapType.

A field with a single type:

{
    "sourceReference": {...},
    "name": "some_field",
    "number": 1,
    "singularType": {...}
}

A field with an option type:

{
    "sourceReference": {...},
    "name": "some_option",
    "number": 2,
    "optionType": {
        "valueType": {...}
    }
}

A field with a list type:

{
    "sourceReference": {...},
    "name": "some_list",
    "number": 3,
    "listType": {
        "valueType": {...}
    }
}

A field with a map type:

{
    "sourceReference": {...},
    "name": "some_map",
    "number": 4,
    "mapType": {
        "keyType": {...},
        "valueType": {...}
    }
}

• name is a string specifying the name of the field.
• number is a number specifying the field number.
• singularType is a type reference specifying the type of the field (if it’s a “simple” field).
• optionType is a JSON object specifying the type of the field (if it’s an option type). It contains a valueType, which is a type reference specifying the type stored in the option.
• listType is a JSON object specifying the type of the field (if it’s a list type). It contains a valueType, which is a type reference specifying the type of each item in the list.
• mapType is a JSON object specifying the type of the field (if it’s a map type). It contains:
  • keyType, which is a type reference specifying the key type of the map.
  • valueType, which is a type reference specifying the value type of the map.

Search results