Building a custom code generator
This page describes the JSON language target in spatial schema
and how to use it to
implement your own custom code generator.
Usually, when you run spatial worker codegen
, it will process the schema with a target
language as specified in the “Codegen” task inside your worker’s *.build.json
. This will generate
code, which allows you, as a developer, to interact with entities and their data. Unfortunately,
the code generated this way can be hard to extend programmatically. There’s now
a new language option ast_json
, which will parse the schema files, and generate an abstract
syntax tree (AST) for every schema file in JSON format.
With the provided ASTs, it is then possible to build a tool, which parses the JSON data and uses it to generate code in any form required. The JSON is deliberately verbose to provide as much information as possible to any custom code generator tool, and not all the fields in the AST need to be used.
Integrating with the standard build process
You can add another step to the “Codegen” task in your *.build.json
, which looks something like
the following:
{
"name": "Json AST",
"arguments": [
"process_schema",
"generate",
"--cachePath=generated/schema_codegen_cache_json",
"--output=generated/json",
"--language=ast_json"
]
}
The “process_schema” argument has the same behaviour as running the spatial schema
command.
The task defined above is identical to running spatial schema generate --language=ast_json
and placing the resulting JSON files inside build/assembly/generated/json
.
Abstract Syntax Tree format
A generated abstract syntaxt tree (AST) is a *.json
file with a structure specified below. For
every *.schema
file, exactly one *.json
file will be generated. Each object in the tree
(except a type reference) is tagged with a
source reference, which gives information that may be useful for formatting
error messages.
Schema file
The root of the AST specifies information about the schema file itself.
{
"sourceReference": {...},
"completePath": "./schema/improbable/example.schema",
"canonicalName": "improbable/example.schema",
"package": "improbable.example",
"enumDefinitions": [...],
"typeDefinitions": [...],
"componentDefinitions": [...]
}
sourceReference
is the source reference of this node.completePath
specifies the complete path to the schema file including the prefix specified by the--input
flag. If the path specified in the--input
flag is relative, thencompletePath
will also be relative, otherwise, it will be an absolute path.canonicalName
is identical tocompletePath
, minus the prefix specified in the--input
flag.package
is the package name defined at the top of the schema file.enumDefinitions
is an array of enum definitions.typeDefinitions
is an array of type definitions.componentDefinitions
is an array of component defintions.
Source reference
A source reference specifies the location of the token that was parsed into the parent node.
"sourceReference": {
"line": 2,
"column": 1
}
line
is anumber
specifying the line number of the node in the schema file.column
is anumber
specifying the column number of the character on the line, which corresponds with the beginning of the node.
Component definition
A component defintion represents a component {}
block in the schema file.
{
"sourceReference": {...},
"name": "ExampleComponent",
"qualifiedName": "improbable.example.ExampleComponent",
"id": 1001,
"dataDefinition": {...},
"eventDefinitions": [...],
"commandDefinitions": [...]
}
sourceReference
is the source reference of this node.name
is astring
specifying the name of the component.qualifiedName
is astring
specifying the fully-qualified name of the component, which includes its package.id
is anumber
specifying the ID of the component.dataDefinition
is a type reference, which will refer to the type specified in thedata
field of the component. If the data structure of the component is instead specified inline with nodata
field, then a new type definition will be generated with the same name of the component, withData
appended to it, which will be specified here. See Reusable data types for more information.eventDefinitions
is an array of event definitions.commandDefinitions
is an array of command definitions.
Event definition
An event definition represents an event inside a component, such as event int32 example_event;
.
{
"sourceReference": {...},
"name": "example_event",
"type": {...},
"eventIndex": 1
}
sourceReference
is the source reference of this node.name
is astring
specifying the name of the event.type
is a type reference specifying the type of the data stored in the event.eventIndex
is anumber
specifying the 1-based position of the event in the order events appear in the schema
Command definition
A command definition represents a command inside a component, such as
command int32 example_command(improbable.example.ExampleType);
.
{
"sourceReference": {...},
"name": "example_command",
"requestType": {...},
"responseType": {...},
"commandIndex": 1
}
sourceReference
is the source reference of this node.name
is astring
specifying the name of the command.requestType
is a type reference specifying the type of the command request.responseType
is a type reference specifying the type of the command response.commandIndex
is anumber
specifying the 1-based position of the command in the order commands appear in the schema
Type reference
A type reference is a JSON object, which specifies a reference to either a built-in type (like
int32
) or a user type (like improbable.example.ExampleType
) and includes a
source reference.
{
"sourceReference": {...},
"builtInType": "int32"
}
{
"sourceReference": {...},
"userType": "improbable.example.ExampleType"
}
Type definition
A type definition represents a type {}
block in the schema file.
{
"sourceReference": {...},
"name": "ExampleType",
"qualifiedName": "improbable.example.ExampleType",
"enumDefinitions": [...],
"typeDefinitions": [...],
"fieldDefinitions": [...]
}
sourceReference
is the source reference of this node.name
is astring
specifying the name of this type.qualifiedName
is astring
specifying the fully qualified name of this type, which includes its package and any parent types.enumDefinitions
is an array of enum definitions.typeDefinitions
is an array of type definitions.fieldDefinitions
is an array of field definitions.
Enum definition
An enum definition represents an enum {}
block in the schema file.
{
"sourceReference": {...},
"name": "TestEnum",
"qualifiedName": "improbable.Test.TestEnum",
"valueDefinitions": [
{
"sourceReference": {...},
"name": "SOME_VALUE",
"value": 0
},
{
"sourceReference": {...},
"name": "SOME_OTHER_VALUE",
"value": 1
}
]
}
sourceReference
is the source reference of this node.name
is astring
specifying the name of this enum.qualifiedName
is astring
specifying the fully qualified name of this enum, which includes its package and any parent types.valueDefinitions
is an array of objects, which consist of three fields:sourceReference
is the source reference of this value.name
is astring
specifying the name of this enum value.value
is anumber
specifying the value, which this enum value represents.
Field definition
A field definition represents a single field in a component
or a type
, such as
int32 some_field = 1;
. The field definition will have exactly one of the following fields:
singularType
, optionType
, listType
and mapType
.
A field with a single type:
{
"sourceReference": {...},
"name": "some_field",
"number": 1,
"singularType": {...}
}
A field with an option type:
{
"sourceReference": {...},
"name": "some_option",
"number": 2,
"optionType": {
"valueType": {...}
}
}
A field with a list type:
{
"sourceReference": {...},
"name": "some_list",
"number": 3,
"listType": {
"valueType": {...}
}
}
A field with a map type:
{
"sourceReference": {...},
"name": "some_map",
"number": 4,
"mapType": {
"keyType": {...},
"valueType": {...}
}
}
name
is astring
specifying the name of the field.number
is anumber
specifying the field number.singularType
is a type reference specifying the type of the field (if it’s a “simple” field).optionType
is a JSON object specifying the type of the field (if it’s an option type). It contains avalueType
, which is a type reference specifying the type stored in the option.listType
is a JSON object specifying the type of the field (if it’s a list type). It contains avalueType
, which is a type reference specifying the type of each item in the list.mapType
is a JSON object specifying the type of the field (if it’s a map type). It contains:keyType
, which is a type reference specifying the key type of the map.valueType
, which is a type reference specifying the value type of the map.