Enrichments

An Enrichment is a derived column for a Table: a blur flag, a label, or an embedding for every frame in frames. You define it once as an expression over each row, Workers compute it in the background, and every reader then gets the value next to the source columns. Nobody copies the Table or computes the value again, and the Table's own schema and files do not change.

An exampleCopy Anchor Link

The Enrichment frames/is_blurry adds a boolean to every row of frames. Its expression reads the row's image and returns true when the frame is too blurry to label, using an SPQL expression. After a refresh, a reader sees:

Its path has two parts: the source Table, frames, and the new value, is_blurry, which is named under the Table. A stored column of the Table cannot have the same name.

What the expression can beCopy Anchor Link

The expression produces exactly one value for each source row, from the columns of that row. It uses SPQL functions, and a function can read data from an external service, so the result does not have to be deterministic. Filters, joins, aggregates, functions that generate rows, and subqueries are not valid, because they do not map one row to one value.

LifecycleCopy Anchor Link

create        definition only                 every row reads NULL
refresh       Workers compute every row       values publish in one commit
source write  new rows have no value          new rows read NULL
refresh       Workers compute changed rows    values publish in one commit

Creating an Enrichment stores its expression and the type of its value, and evaluates nothing. Until the first refresh, every row reads a typed NULL.

A refresh runs as a Task that splits the work by fragment, the groups of rows that the source Table stores together, and hands it to Worker Tasks. It publishes the values in one commit only after every Worker Task succeeds, so readers see either the old values or the new ones. A Worker Task that fails is retried, and only the values of the attempt that succeeded are published. A refresh that is retried, or whose response is lost, publishes at most once. Keep at least two Workers available: one for the refresh and one for the Worker Tasks. Each Task needs two CPUs and 8 GiB of memory by default.

A refresh reads the commit it was started at. If the Enrichment changes in a later commit before the refresh publishes, for example because another refresh published first, the refresh fails without publishing.

FreshnessCopy Anchor Link

When frames gains new rows, those rows read NULL until the next refresh. A change to a column that the expression reads, such as image, makes only the affected rows stale, and the next refresh computes only those rows. Changes to other columns keep the value fresh and cost nothing to refresh. Deleted source rows disappear.

Each Enrichment version records the Table version that it covers, so you can check freshness before a job depends on it. See Inspect an Enrichment.

How it is storedCopy Anchor Link

An Enrichment stores its values in its own files in Project storage, aligned with the source Table's fragments: the same rows, at the same positions. That alignment is what lets a scan read the value alongside the source row without a join.

Table maintenance can regroup rows into new fragments. The next refresh then copies each moved row's value into the new layout without evaluating the expression again, so maintenance never reruns a model or an external call.

Dropping an Enrichment removes its name. It does not change the source Table's schema or rows.

Where nextCopy Anchor Link

• Create an Enrichment and refresh it from Python or Rust.
• Inspect an Enrichment to check which source version it covers.

AvailabilityCopy Anchor Link

Create and refresh Enrichments from the Python and Rust SDKs. Reading the values beside a scan of the source Table is planned. Enrichments created in Studio store an expression that refresh does not run, so create the Enrichments you refresh with the SDKs.

Asynchronous model inference functions that call remote inference providers are planned.

Where it runsCopy Anchor Link

Workers compute the values next to the data, and readers read them directly from Project storage. Spiral's control plane stores the definition, issues storage access, and publishes versions. See Architecture.

The Enrichments HTTP reference documents the resource operations.

HTTP APICopy Anchor Link

The HTTP reference documents the contract at /api/enrichments. It uses camelCase fields, tagged union payloads, and decimal strings for 64-bit counts and timestamps. Existing Rust and CLI clients keep using the compatible resource endpoints.