Current language reference

The current .yidl route is brace-blocked Lark concept syntax. The old indentation/transducer parser and %% fences belong to a different frontend. yidl-compile invokes that older frontend; use the Python API for this language.

Authoring rules

Surface	Syntax/meaning
Module	module example.generator labels the source module
Imports	import "base.yidl" as base; from "base.yidl" import concept Base
Export	export concept Classes makes a concept available to importers
Concept	concept Child extends base.Base { ... }
Property	property FieldName: str storage field_name; optional = value default
Record	record Field { FieldName FieldOrder } — no commas between entries
Family	family Fields { common Name, Order variant Plain {} }
Collection	collection Fields: Field identity FieldId many; tuple identities supported
Computed view	computed collection Selected: Field from Fields where Enabled == True
Resource	resource Store = template $[ ... ]$; code, literal/import forms also exist
Matcher	Named input aliases; defaults and named when rules; resource/contribution/operation result kinds
Contribution	Resource/production, as, numeric index, order, target and bindings
Data production	production P from Source to Target { set Name = source.Name }
Composable production	production P -> composable { root Root = Resource ... }
Assembly	assembly Module = P exports a named callable build
Operation	Declared collection inputs/outputs and resource or matcher implementation

Conditions are equality terms joined with and; this is not unrestricted Python expression syntax. Resource bodies are Python/Astichi, but contribution binding expressions have a smaller grammar (literal, visible semantic name or tuple). An accepted parse is not proof that every syntactic combination has runtime semantics for your chosen surface/backend.

Delimiters and small failures

Resources accept quoted strings, inline backticks, fenced blocks and $[ ... ]$ or $( ... )$ forms defined below. Use the tutorial's $[ form for multiline Python. YIDL record bodies use separate property entries: record R { Name Order }. record R { Name, Order } raises a comma parser diagnostic. A comma in common or an explicit tuple is a different grammar position.

A missing imported key fails resolution; a missing export fails symbol lookup; duplicate inherited productions/defaults fail composition. An assembly name is not inferred from a production's name.

Exact grammar of this candidate

This block is extracted from the compiler grammar at site build time. It provides all productions, including phase/operation/diagnostic syntax. Tested grammar acceptance/rejection is covered by tests/generation/test_yidl_lark_parser.py, test_yidl_lark_update_a_diagnostics.py and test_yidl_operation_matchers.py.

Source authority: src/yidl/concept_grammar.lark.

start: file

file: module_decl? import_decl* top_level_decl*

module_decl: "module" qname

import_decl: "import" STRING "as" CNAME                 -> import_alias
           | "from" STRING "import" import_item ("," import_item)* -> import_from

import_item: symbol_kind CNAME ("as" CNAME)?

symbol_kind: "concept" -> symbol_concept
           | "property" -> symbol_property
           | "record" -> symbol_record
           | "union" -> symbol_union
           | "collection" -> symbol_collection
           | "port" -> symbol_port
           | "resource" -> symbol_resource
           | "matcher" -> symbol_matcher
           | "contribution" -> symbol_contribution
           | "assembly" -> symbol_assembly
           | "operation" -> symbol_operation
           | "production" -> symbol_production

top_level_decl: export_decl
              | concept_decl
              | property_decl
              | resource_decl

export_decl: "export" symbol_kind CNAME

concept_decl: "concept" CNAME extends_clause? "{" concept_member* "}"

extends_clause: "extends" qname ("," qname)*

concept_member: use_decl
              | property_decl
              | record_decl
              | union_decl
              | family_decl
              | collection_decl
              | computed_collection_decl
              | computed_collection_filter_decl
              | port_decl
              | resource_decl
              | matcher_decl
              | contribution_decl
              | production_decl
              | composable_production_decl
              | composable_production_extension_decl
              | assemble_decl
              | assembly_decl
              | direct_operation_decl
              | matcher_operation_decl
              | diagnostics_decl

use_decl: "use" qname ("as" CNAME)?

property_decl: "property" CNAME ":" type_expr default_clause? storage_clause?
default_clause: "=" value_expr
storage_clause: "storage" CNAME

type_expr: qname
         | "object" -> type_object
         | "str" -> type_str
         | "int" -> type_int
         | "bool" -> type_bool
         | "float" -> type_float
         | "literal" "[" literal_list "]" -> type_literal

record_decl: "record" CNAME "{" property_ref* "}"
property_ref: qname

union_decl: "union" CNAME "{" variant_decl* "}"
variant_decl: "variant" CNAME "{" property_ref* "}"

family_decl: "family" qname "{" family_member* "}"
family_member: "common" property_ref ("," property_ref)* -> family_common
             | variant_decl

collection_decl: "collection" CNAME ":" type_ref identity_clause? cardinality_clause?
type_ref: qname
identity_clause: "identity" identity_expr
identity_expr: property_ref
             | "(" property_ref ("," property_ref)+ ")"
cardinality_clause: "single" -> cardinality_single
                  | "many" -> cardinality_many

computed_collection_decl: "computed" "collection" CNAME ":" type_ref "from" qname "where" condition_expr
computed_collection_filter_decl: "filter" qname "where" condition_expr

port_decl: "port" CNAME STRING cardinality_clause

resource_decl: "resource" CNAME "=" resource_expr
resource_expr: "literal" value_expr       -> resource_literal
             | "import" STRING "." CNAME  -> resource_import
             | "code" snippet resource_options? -> resource_code
             | "template" snippet resource_options? -> resource_template
             | "astichi_code" snippet resource_options? -> resource_astichi_code
             | "astichi_template" snippet resource_options? -> resource_astichi_template

snippet: STRING -> snippet_string
       | BACKTICK_INLINE -> snippet_backtick_inline
       | BACKTICK_BLOCK -> snippet_backtick_block
       | DOLLAR_PAREN_INLINE -> snippet_dollar_paren_inline
       | DOLLAR_PAREN_BLOCK -> snippet_dollar_paren_block
       | DOLLAR_SQUARE_INLINE -> snippet_dollar_square_inline
       | DOLLAR_SQUARE_BLOCK -> snippet_dollar_square_block

resource_options: "{" resource_option* "}"
resource_option: "keep" keep_name_list -> resource_keep
               | "edge" resource_edge_kind "=" resource_ref_expr -> resource_edge
keep_name_list: CNAME ("," CNAME)*
resource_edge_kind: "arg_names" -> edge_arg_names
                  | "bind" -> edge_bind
                  | "keep_names" -> edge_keep_names

resource_ref_expr: resource_ref -> resource_name_ref
                 | "match" "." "resource" "(" ")" -> resource_match_ref

matcher_decl: "matcher" CNAME "(" matcher_input_list? ")" matcher_kind? "{" matcher_default? matcher_rule* "}"
matcher_input_list: matcher_input ("," matcher_input)*
matcher_input: CNAME ":" qname
matcher_kind: "->" "resource" -> matcher_kind_resource
            | "->" "contribution" -> matcher_kind_contribution
            | "->" "operation" -> matcher_kind_operation
matcher_default: "default" "->" resource_ref_expr
matcher_rule: "rule" CNAME "when" condition_expr "->" resource_ref_expr weight_clause?
resource_ref: qname
weight_clause: "weight" NUMBER

contribution_decl: "contribution" CNAME contribution_source "{" contribution_member* "}"
contribution_source: "=" resource_ref_expr
contribution_member: contribution_as
                   | contribution_index
                   | contribution_order
                   | target_decl
                   | bind_decl
                   | contribution_diagnostic
contribution_as: "as" CNAME
contribution_index: "index" contribution_value_expr
contribution_order: "order" contribution_value_expr
contribution_diagnostic: "diagnostic"
target_decl: "target" CNAME "{" target_path_option* "}"
target_path_option: "build" path -> target_build
                  | "owner" path -> target_owner
path: ("/" path_segment)+
path_segment: CNAME path_index*
            | "." -> path_current
            | "?" -> path_optional
            | "*" -> path_any
            | "+" -> path_many
path_index: "[" contribution_value_expr ("," contribution_value_expr)* "]"
bind_decl: contribution_bind_kind CNAME "=" contribution_value_expr
contribution_bind_kind: "ident" -> bind_ident
                      | "external" -> bind_external
?contribution_value_expr: literal_expr
                        | qname
                        | contribution_tuple_expr
contribution_tuple_expr: "(" contribution_value_expr "," (contribution_value_expr ("," contribution_value_expr)*)? ")"

production_decl: "production" CNAME "from" source_expr "to" qname "{" production_member* "}"
source_expr: CNAME "." "results" "(" ")" order_clause? -> matcher_results_source
           | qname order_clause? -> collection_source
order_clause: "ordered" "(" property_ref ("," property_ref)* ")"
production_member: "identity" value_expr
                 | "policy" qname
                 | "set" property_ref "=" value_expr

composable_production_decl: "production" CNAME composable_production_inputs? "->" "composable" "{" composable_production_member* "}"
composable_production_inputs: "(" matcher_input_list? ")"
composable_production_member: root_decl
                            | apply_decl
                            | phase_decl
phase_decl: "phase" CNAME phase_position? phase_context? "{" apply_decl* "}"
phase_position: phase_anchor phase_order?
              | phase_order
phase_anchor: "after" CNAME
phase_order: "order" SIGNED_INT
composable_production_extension_decl: "extend" "production" qname "{" phase_extension_decl* "}"
phase_extension_decl: "phase" CNAME phase_position? phase_context? "{" apply_decl* "}"
phase_context: assemble_from where_clause?
             | where_clause
root_decl: "root" CNAME "=" resource_ref_expr root_options?
root_options: "{" bind_decl* "}"
apply_decl: "apply" CNAME apply_tail?
apply_tail: assemble_from? where_clause? "using" CNAME
assemble_decl: "assemble" CNAME assemble_context? assemble_from? where_clause? "using" CNAME
assemble_context: "(" matcher_input_list? ")"
assemble_from: "from" matcher_input_list
where_clause: "where" condition_expr
assembly_decl: "assembly" CNAME "=" qname

direct_operation_decl: "operation" CNAME operation_io "using" resource_ref_expr operation_options?
matcher_operation_decl: "operation" CNAME operation_from operation_io "using" "matcher" CNAME operation_options?
operation_from: "from" matcher_input
operation_io: "inputs" "(" operation_input_list? ")" "outputs" "(" operation_output_list? ")"
operation_input_list: qname_list
operation_output_list: qname_list
qname_list: qname ("," qname)*
operation_options: "{" operation_option* "}"
operation_option: "ordered" "(" property_ref ("," property_ref)* ")"
                | "diagnostics" qname

diagnostics_decl: "diagnostics" CNAME "{" "collection" CNAME "gate" CNAME "}"

?condition_expr: condition_term
               | condition_expr "and" condition_term -> condition_and
condition_term: value_expr "==" value_expr

?value_expr: literal_expr
           | qname
           | source_ref
           | match_ref
           | lookup_expr
           | port_address_expr
           | tuple_expr

source_ref: "source" "." property_ref
match_ref: "match" "." "resource" "(" ")" -> match_resource
         | "match" "." "record" "(" STRING ")" "." property_ref -> match_record
         | "match" "." "value" "(" INT ")" -> match_value
lookup_expr: "lookup" "(" qname "," "key" "=" value_expr "," "value" "=" property_ref default_arg? ")"
default_arg: "," "default" "=" value_expr
port_address_expr: qname "." "of" "(" value_expr ")"
tuple_expr: "(" ")" -> tuple_empty
          | "(" value_expr "," ")" -> tuple_single
          | "(" value_expr "," value_expr ("," value_expr)* ","? ")" -> tuple_multi

literal_expr: STRING
            | INT
            | NUMBER
            | "True" -> true
            | "False" -> false
            | "None" -> none

literal_list: literal_expr ("," literal_expr)*

qname: CNAME ("." CNAME)*

COMMENT: /#[^\n]*/
BACKTICK_BLOCK.20: /```(?:[A-Za-z_][A-Za-z0-9_ -]*)?\r?\n(?s:.*?)(?:\r?\n)[ \t]*```/
BACKTICK_INLINE.20: /`[^\r\n`]*`/
DOLLAR_PAREN_BLOCK.20: /\$\(\r?\n(?s:.*?)(?:\r?\n)[ \t]*\)\$/
DOLLAR_PAREN_INLINE.20: /\$\([^\r\n]*?\)\$/
DOLLAR_SQUARE_BLOCK.20: /\$\[\r?\n(?s:.*?)(?:\r?\n)[ \t]*\]\$/
DOLLAR_SQUARE_INLINE.20: /\$\[[^\r\n]*?\]\$/

%import common.CNAME
%import common.ESCAPED_STRING -> STRING
%import common.INT
%import common.SIGNED_INT
%import common.SIGNED_NUMBER -> NUMBER
%import common.WS
%ignore WS
%ignore COMMENT