Resources
Name Tables and other resources in a Project, and select exact versions with path@ref addresses.
Everything your application stores in a Project is a resource: most often
a Table, but also Enrichments, Vector Indexes, Locations and Connections.
Each resource has a path, such as frames, and every change to it creates a
new version that never changes again. To read the current data you write
frames@main. To read exactly what last week's job saw, you write the
version address it recorded, such as
frames@v019a7c3e2f8d7b1e9f0a1b2c3d4e5f60.
Kinds of resource
| Kind | Holds | Use it when | Service |
|---|---|---|---|
| Table | Rows of structured and multimodal data | You store your application's data, such as camera frames | Tables |
| Enrichment | One derived value for each row of a Table | You compute something per row, such as is_blurry, without changing the Table | Enrichments |
| Vector Index | A similarity-search index over a vector column | You search for similar rows, for example by embedding | Vector Indexes |
| Location | A name for a folder in object storage you already have | You use existing files, such as s3://example-robotics/recordings/, without copying them | Locations |
| Connection | One credential for an external system | A Location or an import needs a private bucket or an account, such as Hugging Face | Connections |
Each kind is managed by its own service, which stores the contents of each version. The Catalog, the service that keeps names and commits, stores only which version each name selects.
Identity and versions
A resource has a resource ID that never changes, and a kind that is fixed when it is created. Renaming a resource changes its path, not its ID.
Each change to a resource makes a new version with its own version ID. A version is immutable, so anything that refers to it reads the same data forever. A new version is not visible under the resource's name until you commit it to a branch of the Project; see Commits and Branches.
Names
A resource's path is one or more segments separated by /, and it cannot
contain @. A path selects a version only at a particular commit: on the
main branch, frames can select one version today and a newer one after
the next commit.
Addresses
An address is a path with an optional @ref suffix that says which state of
the Project to read:
| Address | Selects |
|---|---|
frames | The version in the reader's default state, for example a pinned branch head |
frames@main | The version at the head of the branch main |
frames@c019a7c3e2f8d7b1e9f0a1b2c3d4e5f60 | The version at one exact Project commit |
frames@v019a7c3e2f8d7b1e9f0a1b2c3d4e5f60 | One exact version of the resource |
A ref that is c or v followed by 32 lowercase hexadecimal digits is a
commit or version ID. Any other ref is a branch name.
Resource addresses has the complete grammar.
Recording what a job read
A branch address can select a newer version tomorrow, but commit and version
addresses always select the same one. So when a job must be reproducible,
such as a training run or an evaluation, save the @v address of each Table
it read along with its results. In the Rust SDK, a Table opened by path
reports that address through resolved_address(); see
Sessions.