Kindly Episode Schema — an open standard for surgical & clinical-workflow physical-AI data.
This page is the canonical, citable statement of the standard. The Kindly Episode Schema is a vendor-neutral, openly-adoptable interchange format for one episode of a captured clinical or surgical workflow — a bounded run of real work (one tray assembly, one room turnover, one decontamination cycle, one procedure segment) annotated densely enough to train and supervise a policy, not just classify frames.
The schema, this spec, and a conformance validator install together, so you can validate or scaffold a conformant episode with no Kindly account and no network access:
pip install foodforthought-cli
ate episode validate path/to/episode.json # exits 0 iff conformant
ate episode scaffold # emit a conformant starter episodeWhat this is (and why it exists)
Existing surgical datasets are classification corpora — phase recognition and <instrument, verb, target> triplets. This standard is for policy-learning data: temporally-aligned, instrument-state-aware, intent- and context-bearing labels that serialize natively to RLDS / LeRobot, so an existing pipeline ingests an episode with no bespoke loader.
It is deliberately an interoperability standard, not a lockdown: object nodes allow additional properties (a partner's superset still validates), and only the core fields that appear across the worked sample episode are required.
Canonical identifiers
The machine schema is authoritative; this document explains and governs it.
Machine schema (source of truth)
kindly-episode.schema-v0.1.json (JSON Schema draft 2020-12), bundled in the package and published at the schema's $id:
Full annotation guide
pip install foodforthought-cli). This page and the machine schema are the canonical spec.Controlled vocabularies
vocab/<family>@<ver>.json (e.g. vocab/SPD.tray_assembly@0.1.json) — referenced from each episode's vocab_ref.Structure (one episode)
An envelope, four independent label layers over one timeline, and a derived serialization.
Envelope
Provenance, de-identification + consent status, capture modalities, workflow family + versioned boundary definition, and an expected-contents manifest.
synthetic and data_provenance are required — nothing ships unmarked. Only deid.status: verified episodes are releasable.2 · Four label layers
Independent tracks over one timeline; a consumer may take any subset.
Workflow phase / step
Action triplets
Per-object state tracks
Workflow-context events
Derived serialization
Conformance
An artifact is schema-conformant (v0.1) iff:
- it validates against
kindly-episode.schema-v0.1.json(useate episode validate, which returns exit 0 and the resolved schema title), and - every controlled-vocabulary value resolves in the
vocab_refit declares, and - it carries required provenance (
schema_version,data_provenance/synthetic) and, for any released (non-synthetic) episode,deid.status: verified.
Layers are optional tracks: an episode with only L1 is conformant; richer episodes simply carry more layers. Conformance is about format, not completeness — so partial adopters interoperate.
Versioning policy (SemVer per artifact)
schema_version (e.g. 0.1.0) is stamped into every episode. Shipped episodes are immutable — a bump produces a new release tag plus a migration note; prior episodes keep their original schema_version. Vocabulary growth (e.g. adding a verb) is a vocab-file MINOR bump, not a schema change.
How to adopt / propose changes
Adopt
Install the CLI, scaffold a starter, author against the layers you need, validate to exit 0. Export to RLDS / LeRobot as usual.
pip install foodforthought-cli
ate episode scaffold --out episode.json
ate episode validate episode.jsonPropose a change
The schema is v0.1 and in development. To propose a new field, enum value, workflow family, or vocab entry, email us — the open questions are the live agenda for OEM ML and foundation-lab data teams, and a public issue tracker will follow as the standard matures.
taylorm@kindly.fyiLicense & openness
The schema and this specification are published to be freely adoptable (open standard). Reference implementations ( ate episode validate/scaffold) are MIT-licensed in foodforthought-cli. Adopting the standard creates no obligation to Kindly Robotics.
Honest status
This standard is v0.1 and in development. The sample episodes bundled with the schema are synthetic — every value is fabricated for illustration and no clinical capture has occurred. We are pre-capture and pre-IRB; the machine schema is authoritative and this document explains and governs it. The standard will evolve in the open with first-cohort design partners.
Kindly Robotics · Kindly Episode Schema v0.1 (in development) · canonical standard doc. The machine schema is authoritative; this document explains and governs it. Questions or proposals? Email taylorm@kindly.fyi.