Generate your first class¶

Learning path · Step 4 of 4. Use the environment from Installation. Run the example first, then follow the explanation of one field. The complete source listings at the end are optional reference.

Build a two-field slotted Point class. The beginner sequence is How YIDL works → generation stages → installation → this tutorial. Those first pages explain the parts; the walkthrough below connects them before the complete source listings.

What happens in this example¶

You author classes.yidl to describe input rows, reusable Python pieces and rules for putting them together. The driver generate.py runs three stages:

Prepare: compile those definitions into runtime.py, a generator library with ClassRow/FieldRow input types and build_Module.
Generate: supply rows describing Point, x and y; the library selects parameters/stores and assembles Point code. The driver saves that code as model.py and executes its Python syntax tree to obtain the Point class.
Use: call Point(3, 4). Its constructor assigns this instance's values; the generator is not rerun for each instance.

The input fact field_name="x" tells the generator which name to emit. It is not the value 3. The .yidl file describes how to generate classes; runtime.py is the generated generator library; model.py is generated application code. YIDL selects/binds/places pieces, Astichi combines their Python syntax and scopes, and this small consumer's driver supplies field policy and input values.

Run and inspect¶

Experienced readers can start here after installation. From the YIDL repository root, with your installed interpreter:

ASTICHI_LOWER_ENGINE=python build/example-env/bin/python \
  example/documentation/generate.py --out build/first

The command checks that Point(3, 4) has x=3 and y=4, a signature ordered x/y, direct stores, and no instance __dict__. It writes runtime.py and model.py inside build/first/. These are generated outputs; keep classes.yidl and generate.py as the editable source.

Follow one field through the generator¶

1. Describe the inputs¶

A property describes a fact such as a field's name. A record groups facts into a row; a collection holds those rows. These declarations are excerpts from the complete concept, not separate runnable files:

Source authority: example/documentation/classes.yidl.

    property FieldId: str
    property Owner: str
    property FieldName: str
    property FieldOrder: int
    property Selected: bool = True

    record ClassRow { ClassId ClassName ClassOrder SlotNames EnableInit }
    record FieldRow { FieldId Owner FieldName FieldOrder Selected }
    collection Classes: ClassRow identity ClassId many
    collection Fields: FieldRow identity FieldId many

The generated Python constructor uses storage spelling field_name; declarations refer to semantic property FieldName. The driver supplies a row for x with owner point and order 10, and another for y with order 20. It also supplies a class row naming Point and a slot tuple ("x", "y"). Computing that tuple is this driver's policy; the data layer does not infer it from field rows.

2. Write reusable code pieces¶

A resource holds a Python code piece. The parameter resource has a name to fill in; the store resource has a parameter read and an attribute path to fill in:

Source authority: example/documentation/classes.yidl.

    resource Parameter = template $[
        def astichi_params(field_name__astichi_arg__):
            pass
    ]$
    resource Store = template $[
        astichi_pass(self, outer_bind=True).astichi_ref(external=slot)._ = astichi_pass(value__astichi_arg__, outer_bind=True)
    ]$

The unusual names and marker calls are Astichi's composition instructions. Binding x into these pieces eventually produces a parameter x and the direct statement self.x = x. The astichi_pass expressions refer to the containing constructor's self/parameter; the trailing ._ marks a valid assignment target. These instructions disappear from final application code.

3. Choose, bind and place a piece¶

A contribution adds the binding and destination to a resource. This one binds the selected FieldName as a Python identifier and targets the constructor's parameter hole:

Source authority: example/documentation/classes.yidl.

    contribution Param = Parameter {
        as Param
        index (ClassOrder, FieldOrder)
        order FieldOrder
        target params { build /InitMethod }
        ident field_name = FieldName
    }

A matcher selects it only when the field belongs to the current class and is enabled. The nested constructor production groups parameter/store insertion steps. Its root is the constructor code piece, exposing named holes for those insertions. The final assembly declaration exposes the build function build_Module in the generated library.

Collection identity, owner, build index and output order are separate. The composite integer index distinguishes insertions across classes; FieldOrder orders the parameters. The driver deliberately inserts y before x, yet the result's signature is x/y. The data-flow guide traces those decisions across two owners.

How the Python driver runs the stages¶

Before reading the full driver, recognize these operations:

Operation	Result
`compile_yidl_files(source_map, entry_name)`	Compiled concept definitions from explicitly named source text
`emit_concept_runtime_source(...)`	`runtime.py` library source for those definitions
Execute that library in a fresh namespace	Row classes, builder and assembly functions
`new_builder()` and `builder.add(...)`	Mutable collection of this generation's input facts
`freeze()`	Read-only fact container; input preparation is complete
`build_Module(facts)`	Selected/bound pieces in an Astichi composable, its intermediate code object
`to_executable_ast()`	Final Python syntax tree for Point
`compile(tree, ..., "exec")` and `exec`	Executable Point class in the consumer namespace
`Point(3, 4)`	User instance with x=3 and y=4

Source/template parsing during preparation or lazy resource compilation differs from compiling this final tree. Live defaults/callbacks need explicit captures; see binding and ownership after this example.

Options, defaults and a small change¶

With no scenario flag, the script generates the two-field Point. Every scenario switch defaults to false; choose at most one. With no --out, it writes runtime.py and model.py to build/documentation/ relative to your current working directory. --out overrides that directory; it does not change the .yidl authority. The run above explicitly chooses build/first instead.

Option	Effect	Default
`--out DIR`	Destination for the two generated Python files	`build/documentation` under the current directory
`--extended`	Generate Point(x, y) and Size(x)	False
`--empty`	Generate zero-field Point with a zero-parameter constructor	False
`--disabled`	Generate Point slots while omitting its init method	False
`--help`	Describe the command and options, then exit	Runs only when requested

build/example-env/bin/python example/documentation/generate.py --help

Use --extended to explore a second owner, or inspect the gallery for all modes. For an authoring change, alter the field facts and matching slot tuple together to request another field, and update the example's constructor call and behavior checks accordingly. To change the generated store behavior, edit the resource in .yidl and regenerate; editing model.py alone will be overwritten.

Remove the experiment's outputs¶

If build/first/ was created only for this walkthrough, remove that directory to discard its generated Python files. If you used the default destination instead, its disposable output directory is build/documentation/. Keep authored example/documentation/classes.yidl and generate.py. Environment/wheel removal and outside-copy cleanup are listed in the installation page's cleanup steps.

Continue after the tutorial¶

You have completed the four-page learning path. Next, inspect the generated files to distinguish your editable inputs from the two outputs. To explore selection across two classes, follow Matching and data flow. These are follow-up choices; the full listings below are available when you want to inspect the complete example.

Optional: complete source listings¶

You do not need to retype these listings or run them as additional steps. The command above already uses these repository files. The complete source adds class/method shells, explicit empty contributions for mandatory holes, the other selections and the executable checks.

Declarations: classes.yidl¶

Source authority: example/documentation/classes.yidl.

module documentation.classes
export concept Classes

concept Classes {
    property ClassId: str
    property ClassName: str
    property ClassOrder: int
    property SlotNames: object
    property EnableInit: bool = True
    property FieldId: str
    property Owner: str
    property FieldName: str
    property FieldOrder: int
    property Selected: bool = True

    record ClassRow { ClassId ClassName ClassOrder SlotNames EnableInit }
    record FieldRow { FieldId Owner FieldName FieldOrder Selected }
    collection Classes: ClassRow identity ClassId many
    collection Fields: FieldRow identity FieldId many

    resource Module = code $[
        astichi_hole(classes)
    ]$
    resource ClassShell = template $[
        class class_name__astichi_arg__:
            __slots__ = astichi_bind_external(slots)
            astichi_hole(methods)
    ]$
    resource Init = template $[
        def __init__(self, params__astichi_param_hole__):
            pass
            astichi_hole(stores)
    ]$ { keep self, __init__ }
    resource Parameter = template $[
        def astichi_params(field_name__astichi_arg__):
            pass
    ]$
    resource Store = template $[
        astichi_pass(self, outer_bind=True).astichi_ref(external=slot)._ = astichi_pass(value__astichi_arg__, outer_bind=True)
    ]$
    resource Empty = template ``
    resource EmptyParams = template $[
        def astichi_params():
            pass
    ]$
    resource EmptyStore = template `pass`
    contribution ClassPass = EmptyStore {
        as ClassPass
        target methods { build /ClassDef }
    }
    matcher ClassPassSelection(cls: Classes) -> contribution {
        default -> ClassPass
    }
    contribution EmptyParameters = EmptyParams {
        as EmptyParameters
        target params { build /InitMethod }
    }
    contribution EmptyStatements = EmptyStore {
        as EmptyStatements
        target stores { build /InitMethod }
    }
    matcher EmptyParameterSelection(cls: Classes) -> contribution {
        default -> EmptyParameters
    }
    matcher EmptyStatementSelection(cls: Classes) -> contribution {
        default -> EmptyStatements
    }

    contribution ClassContribution = ClassProduction {
        as ClassDef
        index ClassOrder
        order ClassOrder
        target classes { build /Root }
    }
    contribution InitContribution = InitProduction {
        as InitMethod
        target methods { build /ClassDef }
    }
    contribution NoInit = Empty {
        as NoInit
        target methods { build /ClassDef }
    }
    contribution Param = Parameter {
        as Param
        index (ClassOrder, FieldOrder)
        order FieldOrder
        target params { build /InitMethod }
        ident field_name = FieldName
    }
    contribution Assignment = Store {
        as Store
        index (ClassOrder, FieldOrder)
        order FieldOrder
        target stores { build /InitMethod }
        ident value = FieldName
        external slot = FieldName
    }
    matcher ClassSelection(cls: Classes) -> contribution {
        default -> ClassContribution
    }
    matcher InitSelection(cls: Classes) -> contribution {
        default -> NoInit
        rule enabled when EnableInit == True -> InitContribution
    }
    matcher Params(cls: Classes, field: Fields) -> contribution {
        rule selected when Owner == ClassId and Selected == True -> Param
    }
    matcher Stores(cls: Classes, field: Fields) -> contribution {
        rule selected when Owner == ClassId and Selected == True -> Assignment
    }
    production ModuleProduction -> composable {
        root Root = Module
        apply classes from cls: Classes using ClassSelection
    }
    production ClassProduction(cls: Classes) -> composable {
        root ClassDef = ClassShell {
            ident class_name = ClassName
            external slots = SlotNames
        }
        apply class_pass using ClassPassSelection
        apply init using InitSelection
    }
    production InitProduction(cls: Classes) -> composable {
        root InitMethod = Init
        apply empty_params using EmptyParameterSelection
        apply empty_stores using EmptyStatementSelection
        apply params from field: Fields
            where Owner == ClassId and Selected == True using Params
        apply stores from field: Fields
            where Owner == ClassId and Selected == True using Stores
    }
    assembly Module = ModuleProduction
}

Driver: generate.py¶

Source authority: example/documentation/generate.py.

"""Generate Point(x, y) using installed YIDL and Astichi packages.

With no scenario flag, generate the two-field Point class. Choose at most one
scenario flag. Output paths are relative to the current working directory.
"""

from __future__ import annotations

import argparse
import ast
from dataclasses import dataclass
import inspect
import json
from pathlib import Path
import sys
from typing import Mapping

from astichi.lower_engine.native import select_effective_lower_engine
from yidl.concept_parser import compile_yidl_files
from yidl.generation.data_def_sys import emit_concept_runtime_source


@dataclass(frozen=True)
class Output:
    runtime: str
    model: str
    observations: Mapping[str, object]


def generate(
    *, extended: bool = False, empty: bool = False, disabled: bool = False
) -> Output:
    authority = Path(__file__).with_name("classes.yidl")
    module = compile_yidl_files({"classes.yidl": authority.read_text()}, "classes.yidl")
    concept = module.concepts["Classes"]
    runtime = emit_concept_runtime_source(
        concept.plan.build_data_definition(),
        resources=concept.resources,
        assembly_plan=concept,
    )
    schema: dict[str, object] = {}
    exec(compile(runtime, "generated_runtime.py", "exec"), schema)
    builder = schema["new_builder"]()
    classes = [("point", "Point", 10, () if empty else ("x", "y"))]
    if extended:
        classes.append(("size", "Size", 20, ("x",)))
    for identity, name, order, slots in classes:
        builder.add(
            schema["ClassesCollection"],
            schema["ClassRow"](
                class_id=identity,
                class_name=name,
                class_order=order,
                slot_names=slots,
                enable_init=not disabled,
            ),
        )
    # Reversed insertion order proves that FieldOrder owns parameter order.
    rows = (
        []
        if empty
        else [
            ("point.y", "point", "y", 20, True),
            ("point.x", "point", "x", 10, True),
            ("point.hidden", "point", "hidden", 30, False),
        ]
    )
    if extended:
        rows.extend(
            [("size.x", "size", "x", 10, True), ("orphan.z", "absent", "z", 99, True)]
        )
    for identity, owner, name, order, selected in rows:
        builder.add(
            schema["FieldsCollection"],
            schema["FieldRow"](
                field_id=identity,
                owner=owner,
                field_name=name,
                field_order=order,
                selected=selected,
            ),
        )
    facts = builder.freeze()
    composable = schema["build_Module"](facts)
    tree = composable.to_executable_ast()
    model = ast.unparse(tree) + "\n"
    namespace: dict[str, object] = {"__name__": "documentation_model"}
    exec(compile(tree, "generated_model.py", "exec"), namespace)
    point = namespace["Point"]
    if disabled:
        assert "__init__" not in point.__dict__
        result = {"init": "omitted"}
    elif empty:
        instance = point()
        assert point.__slots__ == ()
        assert not hasattr(instance, "__dict__")
        result = {"slots": []}
    else:
        instance = point(3, 4)
        assert (instance.x, instance.y) == (3, 4)
        assert not hasattr(instance, "__dict__")
        assert "hidden" not in model and "orphan" not in model
        assert list(inspect.signature(point).parameters) == ["x", "y"]
        assert "self.x = x" in model and "self.y = y" in model
        result = {"Point": [instance.x, instance.y], "parameters": ["x", "y"]}
    if extended:
        size = namespace["Size"](8)
        assert size.x == 8
        assert list(inspect.signature(namespace["Size"]).parameters) == ["x"]
        result["Size"] = size.x
    return Output(runtime, model, result)


def main() -> None:
    parser = argparse.ArgumentParser(
        description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter
    )
    parser.add_argument(
        "--out",
        type=Path,
        default=Path("build/documentation"),
        help="Directory for runtime.py and model.py; relative to the current working directory",
    )
    parser.add_argument(
        "--extended",
        action="store_true",
        help="Generate Point(x, y) and Size(x)",
    )
    parser.add_argument(
        "--empty",
        action="store_true",
        help="Generate Point with no fields and a zero-parameter constructor",
    )
    parser.add_argument(
        "--disabled",
        action="store_true",
        help="Generate Point with slots but omit its __init__ method",
    )
    args = parser.parse_args()
    output = generate(extended=args.extended, empty=args.empty, disabled=args.disabled)
    args.out.mkdir(parents=True, exist_ok=True)
    (args.out / "runtime.py").write_text(output.runtime)
    (args.out / "model.py").write_text(output.model)
    print(
        json.dumps(
            {
                "python": sys.version.split()[0],
                "engine": select_effective_lower_engine().snapshot(),
                "observations": output.observations,
            },
            sort_keys=True,
        )
    )


if __name__ == "__main__":
    main()