Connections

A Connection lets your whole team read a private bucket or a Hugging Face repository without anyone holding its keys. You let an AWS IAM role or a Google Cloud service account trust the Connection's own identity, or you store a key or a token once. When a teammate reads a Location that uses the Connection, Spiral gives them a read-only credential that reaches only that Location's root and expires within the hour. Nobody, including you, can read a stored secret back.

HTTP APICopy Anchor Link

The HTTP API is available under /api/connections. Requests use camelCase fields and tagged unions with matching discriminator and payload names. Counts and timestamps that use 64-bit integers are returned as decimal strings. Existing Rust and CLI clients continue to use the established endpoints.

An exampleCopy Anchor Link

Your robot recordings sit in a private bucket. You create a Connection robotics-bucket for s3://example-robotics/ with a read-only IAM key, and a Location recordings for the root s3://example-robotics/recordings/ inside it:

Connection robotics-bucket   s3://example-robotics/            holds the key
  └─ Location recordings     s3://example-robotics/recordings/ lists the files

A teammate with read access to recordings lists its files and reads them with their own S3 client. They need no AWS_* variables and no permission on the Connection itself. The credential that Spiral gives them can read recordings/ and nothing else in the bucket.

A Connection to Hugging Face works the same way. The Connection hugging-face holds a token for hf://datasets/example-robotics/robot-runs, and a Location over it, such as hf://datasets/example-robotics/robot-runs@main/data/, lists one revision of that repository.

Choose a providerCopy Anchor Link

A Connection's type is its provider. Each provider page covers the grants to make on your side, the Connection to create, and the test:

What readers receiveCopy Anchor Link

Spiral never hands out the stored credential. When a reader opens a Location, Spiral checks their permissions on the Location and exchanges the stored credential for a short-lived one:

• for a bucket, read and list access to the Location's root only, for one hour;
• for a Hugging Face repository, a short-lived read token for the whole repository, because the Hub cannot narrow a token further. The Location itself still lists and reads only its own root at its own revision.

The reader then reads the provider directly. Spiral signs no individual request and copies no bytes. It caches a credential for most of its life, so a burst of reads of one Location costs one exchange. Application user tokens cannot obtain cloud credentials through a Connection.

The URI sets the ceilingCopy Anchor Link

A Connection names one URI: an S3 or Cloud Storage bucket, a prefix in a bucket, or one Hugging Face repository. No credential that it issues reaches outside that URI, whatever the stored key could do. A Location's root must lie inside it: Spiral refuses s3://example-robotics-old/ for a Connection to s3://example-robotics/, and checks the root again every time it issues a credential.

Store a key, or grant a role, that can read only what the Connection names anyway. Your IAM grant is the outer limit; the Connection URI and the Location root narrow it further.

Trust one Connection, never all of SpiralCopy Anchor Link

Where a provider trusts the Connection's identity, as an AWS role or a Google Cloud service account does, Spiral's issuer https://auth.spiraldb.com signs a short-lived token whose subject names one Connection, such as conn_24041a03-26ef-45a2-88e2-3d89150ff6f7. Any Connection, in any Organization, can obtain a token from the same issuer, so the subject is what separates yours from everyone else's. Every trust policy and binding names the subject, and the policies that Spiral prints for you already do. A grant to the issuer as a whole, or to a whole workload identity pool, would let every Spiral customer in.

A Connection never asks you to trust a cloud identity of Spiral's, such as a Google service account or an AWS account, because those carry no subject of yours.

What a Connection does not doCopy Anchor Link

• It does not return the secret. Every read reports when the credential was last set. No operation returns the value.
• It does not have versions. The credential and the description change in place, and every read uses their current values.
• It does not branch. Only the branch main creates or drops a Connection, and a drop applies to every branch. A revert cannot restore a dropped Connection.
• It does not time-travel. Spiral uses the Connection as it is configured today, whatever Project commit the Location was read at.
• It does not write unless you allow it, and it never deletes. Every Connection starts read-only, and every Location reads only. An administrator can allow writes so that Spiral can export into the bucket. Spiral never deletes anything through a Connection, so Tables that Spiral manages are never stored in one.

Where nextCopy Anchor Link

• Connect Amazon S3, Google Cloud Storage, Cloudflare R2, Ceph, or Hugging Face.
• Manage a Connection: test it, rotate its credential, allow writes, and drop it.
• Create a Location that reads through the Connection.

ReferenceCopy Anchor Link

Providers and authenticationCopy Anchor Link

A Connection's type is its provider, the external system it reaches. How Spiral authenticates to the provider is a separate choice. AWS, Google Cloud Storage, and Ceph offer two; every other provider has one.

A hugging_face Connection without an endpoint reads the public Hub, https://huggingface.co. A minio, rustfs, or ceph endpoint, and a Hub endpoint other than the public one, must be https:// on a host other than loopback on a hosted deployment; local development also takes http://. An r2 Connection names no endpoint: Spiral derives it from the account ID and the jurisdiction, such as https://<account_id>.r2.cloudflarestorage.com. The URI and the provider are fixed for the life of a Connection; everything else about the credential can change with a credential update. An update also moves an aws or ceph Connection between access_key and oidc, and a gcs Connection between service_account_key and oidc.

An oidc credential holds no secret. The role or the service account trusts the Connection's own identity, which Spiral proves to the provider at each exchange, so there is no key to store and no key to rotate. Every other authentication method holds a secret.

How the credential is exchangedCopy Anchor Link

URI rulesCopy Anchor Link

• A prefix names a directory, so it ends with /.
• An hf:// URI names one whole repository, with its type: models, datasets, or spaces. It cannot name a directory or a revision, and a URI without a type, such as hf://example-robotics/robot-model, is refused. Hugging Face Storage Buckets (hf://buckets/...) are planned.
• Connection URIs and Location roots must not reach keys that contain * or ?, or a ${...} sequence, because IAM policies read these as wildcards and variables. A Connection URI is literal key text, so %2A in it stays the three characters %2A. A Location root is a URI, so %2A in it means * and is refused.
• A Location root inside a Hugging Face repository names one revision after the repository: hf://datasets/example-robotics/robot-runs@curated/mcaps/ reads mcaps/ on the branch curated. Write a branch name with /, such as team/curated, as team%2Fcurated, the way file URIs show it. A Location on a branch follows the branch; a Location at a commit lists the same files at every refresh.
• Two Connections can use the same bucket and path at different S3-compatible endpoints. A reader rejects such overlapping roots while their credentials are valid, so that it never reads an object from the wrong server. For overlapping roots at the same endpoint, it uses the most specific root.

PermissionsCopy Anchor Link

connection:use is limited to admin because it returns a credential with no Location in between, so nothing records which data it was meant for. Reading a Location that names a Connection needs no connection:* permission: the connection:bind check when the Location was created is the delegation.

The Connections HTTP reference documents every operation.