Store Tables on a Volume

Choose where a Project keeps its new Tables: Spiral storage, or your own bucket. An Organization owner registers the bucket once as a Volume; each Project administrator then selects it in the Project's settings. Spiral manages the Tables either way, so your application reads and writes them as before.

Before you startCopy Anchor Link

• To register a bucket: the Organization owner role, and one of these:
  • an Amazon S3 bucket and an IAM role that can read, write, list, and delete objects under the root you choose;
  • a Google Cloud Storage bucket, a service account with roles/storage.objectUser on it, and a workload identity pool provider whose issuer is https://auth.spiraldb.com;
  • a bucket on a Ceph Object Gateway and a role on the gateway that can read, write, list, and delete objects under the root. The Volume assumes the role with its own identity or with a user's access key;
  • an R2, MinIO, or RustFS bucket with an access key that can read, write, list, and delete objects under the root. Support for other S3-compatible stores, such as Tigris and Hetzner, is planned.
• To choose a Project's Volume: the Project admin role, or, when you create the Project, the Organization member role.
• Studio, the CLI signed in with spiral login, or an access token in $SPIRAL_TOKEN.

For a custom storage endpoint, follow the hosted endpoint requirements. They also apply when Spiral obtains a Volume's signing credential.

Register your bucketCopy Anchor Link

Give Spiral a bucket, or a prefix in one, that nothing else writes to: Spiral creates a folder for each Table under it and deletes files that Tables no longer need.

An Amazon S3 or Google Cloud Storage bucket trusts the Volume's own identity and needs no key: see Trust the Volume's identity. A Ceph bucket takes the Volume's identity or an access key: see Ceph. For an R2, MinIO, or RustFS bucket, open the Organization's settings in Studio, select Volumes, and fill in Register your bucket. From a client, send the bucket's root and its access key:

credential = StorageCredential(
    r2_access_key=StorageR2AccessKey(
        account_id="0123456789abcdef0123456789abcdef",
        access_key_id=os.environ["R2_ACCESS_KEY_ID"],
        secret_access_key=os.environ["R2_SECRET_ACCESS_KEY"],
    )
)
request = StorageRegisterVolumeRequest(name="archive", uri="s3://example-archive/spiral/", credential=credential)
volume = client.storage.register_volume(request).organization
if volume is not None:
    print(f"registered {volume.name} as {volume.id}")

The bucket must refuse a write with If-None-Match: * to a key that exists, because Spiral never overwrites a Table's files. Spiral does not try the bucket when you register it, so a wrong key or a missing permission shows up when a Table is first written. Spiral stores the secret access key in a secrets vault and returns the Volume without it.

A root cannot contain, or lie inside, another Volume's root on the same bucket.

CephCopy Anchor Link

A bucket on a Ceph Object Gateway registers as a ceph Volume. Like a ceph Connection, it names the gateway's endpoint and a role on the gateway. The Volume assumes the role with its own identity, or with the access key of a user that the role's trust policy names. The role needs the same permissions as an Amazon S3 role: list the bucket, and get, put, and delete objects under the root. The role also needs s3:PutObject on the bucket itself, because Ceph 19.2 checks that action on the bucket when a multipart upload starts. The gateway must serve STS, as Before you start on the Connection page describes. A hosted deployment accepts only an https:// endpoint on a host other than loopback. Registration does not contact the gateway: a wrong endpoint, role, or key fails at the first access.

An s3_like Volume on a gateway that serves STS can switch to ceph by replacing its credential. The Volume keeps its ID, root, and Tables, so the role must reach the whole root. Spiral can use the old key for up to an hour after the switch, so keep it valid for that hour.

Ceph does not tell a write from an overwrite. On Amazon S3 and Google Cloud Storage, a credential that Spiral hands out with write access to a folder can create objects there but cannot replace them. A Ceph gateway cannot enforce that limit, so a credential with write access to a folder on a ceph Volume can also replace the objects in that folder. Spiral's own writers never replace a file, but a client that holds such a credential can. To keep the earlier version of a replaced object, turn on bucket versioning on the gateway. No setting changes this: on a ceph Volume, every write uses a credential that Spiral hands out, never signed access through Spiral.

credential = StorageCredential(
    ceph_oidc=StorageCephOidc(endpoint="https://ceph.example.com", role_arn="arn:aws:iam:::role/spiral-archive")
)
request = StorageRegisterVolumeRequest(name="archive", uri="s3://example-archive/spiral/", credential=credential)
volume = client.storage.register_volume(request).organization
if volume is not None:
    print(f"registered {volume.name} as {volume.id}")

With "type": "cephOidc", register Spiral's issuer on the gateway and set the printed trust policy on the role, as Trust the Volume's identity describes. This needs Ceph 19.2.3 or later, for the same reason as a ceph Connection. With "type": "cephAccessKey", the cephAccessKey payload also carries accessKeyId and secretAccessKey. From the CLI, pass --auth access_key with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY set. The role's trust policy must then name the key's user, as Connect with an access key shows.

Trust the Volume's identityCopy Anchor Link

A Volume on Amazon S3, Google Cloud Storage, or Ceph with "type": "cephOidc" stores no key. For each access, Spiral signs a short-lived OpenID Connect token whose subject is the Volume's ID, and exchanges it with your cloud or gateway for a session narrowed to one Table's folder. Your role or service account trusts that one subject, so no other Volume or Connection can use it.

1. Set up the identity provider once, as for a Connection: an IAM OpenID Connect provider for Amazon S3, a workload identity pool provider for Google Cloud Storage, or an OpenID Connect provider on the gateway for Ceph. One provider serves Connections and Volumes alike. A Connection's subject is conn_… and a Volume's is vol_…, the Volume's ID. If your pool provider restricts subjects with an attribute condition, list the Volume's ID there too:

   gcloud iam workload-identity-pools providers update-oidc spiral \
     --location=global \
     --workload-identity-pool=spiral \
     --attribute-condition="assertion.sub in ['conn_24041a03-26ef-45a2-88e2-3d89150ff6f7', 'vol_01h455vb4pex5vsknk084sn02q']"

   Give the role or service account access to the root. On Amazon S3, the role needs these actions on the bucket and the root's objects; on Google Cloud Storage, the service account needs roles/storage.objectUser on the bucket:

   {
     "Version": "2012-10-17",
     "Statement": [
       {
         "Effect": "Allow",
         "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
         "Resource": "arn:aws:s3:::example-archive"
       },
       {
         "Effect": "Allow",
         "Action": [
           "s3:GetObject",
           "s3:PutObject",
           "s3:DeleteObject",
           "s3:AbortMultipartUpload",
           "s3:ListMultipartUploadParts"
         ],
         "Resource": "arn:aws:s3:::example-archive/spiral/*"
       }
     ]
   }

2. Register the bucket with the role or service account. In Studio, open the Organization's settings, select Volumes, choose Amazon S3, Google Cloud Storage, or Ceph with Volume identity (OpenID Connect) under Register your bucket, and fill in the role, with the gateway's endpoint for Ceph, or the workload identity provider and service account. From a client:

   PythonTypeScriptHTTPCLI

   credential = StorageCredential(
       aws_oidc=StorageAwsOidc(role_arn="arn:aws:iam::123456789012:role/spiral-archive", region="us-east-1")
   )
   request = StorageRegisterVolumeRequest(
       name="archive", uri="s3://example-archive/spiral/", region="us-east-1", credential=credential
   )
   volume = client.storage.register_volume(request).organization
   if volume is not None:
       print(f"registered {volume.name} as {volume.id}")

   For Google Cloud Storage, send a gs:// root and a credential with "type": "gcsOidc" and a gcsOidc payload containing workloadIdentityProvider and serviceAccount. For Ceph, send the credential that Ceph shows.

3. Set the trustPolicy of the response's organization.details.identity: as the role's trust policy, or as the service account's IAM policy. Studio shows it on the Volume under Trust this Volume in AWS, Google Cloud, or Ceph, with the issuer, audience, and subject to paste into your provider. Its condition names the Volume's ID, for example vol_01h455vb4pex5vsknk084sn02q. For a role or service account that already trusts something else, such as a Connection, add its statement or binding rather than replacing what is there. Until you set it, writing a Table on the Volume fails with an access error. AWS can take a minute to apply a changed trust policy.

Each access Spiral obtains reaches one Table's folder and lasts at most an hour. A write can create objects but cannot replace or delete them, because a Table's files never change once written. On a ceph Volume a write can also replace them: see Ceph.

Test a VolumeCopy Anchor Link

After you register a bucket, set a trust policy, or replace a credential, test the Volume as a Table write would use it. In Studio, open the Organization's settings, select Volumes, and choose Test Volume on the Volume. Over HTTP, this needs the Organization owner role, like every other change to a Volume:

curl -X POST "https://api.spiraldb.dev/api/storage/volumes/vol_01h455vb4pex5vsknk084sn02q/test" \
  -H "Authorization: Bearer $SPIRAL_TOKEN"

The test reports one result for each step:

1. vend: Spiral obtains a short-lived credential for the root from the stored key, or from the Volume's identity. For r2, Spiral first lists with the stored key, because it signs the short-lived credential itself.
2. access: Spiral lists at most one page of files under the root with the short-lived credential. This step is absent when step 1 fails.
3. write: Spiral starts and aborts one upload under the root with the credential that writes Tables, which leaves nothing behind. On Amazon S3, Google Cloud Storage, and Ceph that is the short-lived credential; on R2, MinIO, and RustFS it is the signing session that the stored key issues. This step is absent when step 2 fails.

A key or a role that the provider refuses fails at step 1, and a trust policy that does not name the Volume fails there too. A missing bucket or a role without list access fails at step 2. A role that can read but not write fails at step 3. Steps 2 and 3 run on the Spiral server, so they also fail when the server cannot reach your bucket, for example when it accepts requests only from your own network. The detail of each step tells which cause applies.

A Volume registered over HTTP as an unnamed S3-compatible store, with "type": "s3LikeAccessKey", cannot be tested: no token service checks its key, so the test refuses it.

Choose where a Project stores new TablesCopy Anchor Link

In Studio, open the Project, select Settings, choose Spiral storage or one of your Organization's buckets under Storage, and save. From a client, list the Volumes your Organization may use, then choose one. The response contains items; pass nextPageToken as pageToken until it is absent:

for volume in client.storage.list_volumes_items():
    if volume.organization is not None:
        print(f"{volume.organization.id}: {volume.organization.name}")
    elif volume.platform is not None:
        print(f"{volume.platform.id}: Spiral storage on {volume.platform.provider}")

choice = StorageProjectStorage(volume=StorageProjectStorageVolume(volume_id=volume_id))
storage = client.storage.select_project_storage("demo", choice)
if storage.volume is not None:
    print(f"demo stores new Tables on {storage.volume.volume_id}")

New Tables and Enrichments, including the Tables that Locations keep their file lists in, go to the chosen Volume. Existing ones stay where they are.

To choose the Volume when you create a Project, pass it to spiral projects create --volume vol_01h455vb4pex5vsknk084sn02q, or send volume_id to POST /v1/projects.

Selecting a Volume checks that it is active, belongs to your Organization or Spiral, and that the deployment is configured to grant access to it. A Volume that authenticates with its own identity requires an OIDC issuer configured on the deployment; otherwise creation or selection fails before the Project or its storage setting changes.

Selection does not contact the bucket or validate its IAM policy. The provider's trust policy and bucket permissions are exercised when you obtain storage access and read or write data.

A Project created before Volumes shows its file system as the current choice. It keeps storing new Tables there until you choose a Volume.

Verify the resultCopy Anchor Link

Read the Project's storage. volume.volumeId names the Volume that new Tables use, and volume.selected is false while the Project uses its default:

storage = client.storage.read_project_storage("demo")
if storage.volume is None:
    print("demo stores new Tables on its file system")
else:
    print(f"demo stores new Tables on {storage.volume.volume_id} (selected: {storage.volume.selected})")

Retire or revoke a VolumeCopy Anchor Link

Retire a Volume to stop new Tables on it; its Tables stay readable and writable. Revoke it to also stop new and renewed access to its Tables; access that Spiral already granted lasts until it expires, at most an hour. Projects that chose the Volume fail to create Tables until an administrator chooses another. In Studio, use Retire or Revoke on the Volume in the Organization's Volumes list, or send PUT /api/storage/volumes/{volume_id}/status.

To rotate the access key, or to name another role or service account, choose Update credential on the Volume in Studio, or send the new credential to PUT /api/storage/volumes/{volume_id}/credential. The Tables on the Volume keep their folders. The update does not check the new credential, so run the test after it.

Use spiral volumes status VOLUME_ID retired to retire a Volume from the CLI, or select revoked or active to change its status. To replace its credential, use spiral volumes credential VOLUME_ID --provider PROVIDER with the same credential flags as registration.

AWS uses --role-arn. GCS uses --workload-identity-provider and --service-account. R2 uses --account-id and optionally --jurisdiction. Ceph uses --auth access_key or --auth oidc with --endpoint and --role-arn. MinIO, RustFS, and other S3-compatible stores use --endpoint with providers minio, rustfs, and s3_like. Access-key providers read AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY; --access-key-id and --secret-file can override them.

The legacy fs show and fs list-providers commands are deprecated. Use projects storage and volumes ls instead. Mount commands remain available under fs mounts.