Amazon S3

Connect a private S3 bucket once, and everyone in the Project can read it through Locations without holding AWS credentials. Spiral exchanges the Connection's credential for a short-lived session that reaches only the Location's root, so the bucket's own permissions stay the outer limit and no reader ever sees a long-lived key. See Connections for what readers receive.

An aws Connection authenticates in one of two ways:

Before you startCopy Anchor Link

• connection:create in the Project. The editor and admin roles have it.
• The CLI, signed in with spiral login, or Spiral Studio.
• Permission to create IAM roles or users, and their policies, in the AWS account that owns the bucket.

Create Connections on the branch main. All branches share one credential for each Connection, so Spiral creates and drops Connections only on main.

Grant read access to the bucketCopy Anchor Link

Whichever way the Connection authenticates, its IAM role or user needs read access to what the Connection names, and nothing more. Spiral narrows every session further, to the root of one Location, but your grant is the ceiling. For a Connection to s3://example-robotics/, attach this policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListBucket",
      "Effect": "Allow",
      "Action": ["s3:ListBucket", "s3:GetBucketLocation"],
      "Resource": "arn:aws:s3:::example-robotics"
    },
    {
      "Sid": "ReadObjects",
      "Effect": "Allow",
      "Action": ["s3:GetObject"],
      "Resource": "arn:aws:s3:::example-robotics/*"
    }
  ]
}

For a Connection to a prefix, such as s3://example-robotics/recordings/, narrow ReadObjects to arn:aws:s3:::example-robotics/recordings/* and add the condition "StringLike": {"s3:prefix": ["recordings/", "recordings/*"]} to s3:ListBucket.

Grant no other action. A Connection that is to allow writes also needs s3:PutObject, s3:AbortMultipartUpload, and s3:ListMultipartUploadParts on the objects. Add those only when you raise the Connection's allowed access, and never grant s3:DeleteObject: Spiral never deletes through a Connection. Spiral's short-lived credentials create objects only: their session policy requires the If-None-Match header, with which S3 refuses to replace an object that exists. See conditional writes.

If objects use AWS KMS encryption (SSE-KMS), also grant kms:Decrypt on their keys, and let the key policies permit it. The keys must be in the bucket's region. Spiral's session can decrypt only through S3, and only for objects under the Location's root. See the AWS requirements for SSE-KMS.

Connect with an IAM roleCopy Anchor Link

A role Connection stores no key. At each exchange, Spiral signs a short-lived OpenID Connect token whose subject names this Connection, and AWS STS exchanges it for a session of the role. The role trusts that one subject, so no other Connection, in your Organization or any other, can assume it.

1. Once per AWS account, add an IAM OpenID Connect identity provider with the provider URL https://auth.spiraldb.com and the audience sts.amazonaws.com.

2. Create the role with the read policy from Grant read access to the bucket. Give it any temporary trust policy; the next steps replace it.

3. Create the Connection with the role's ARN:

   PythonTypeScriptCLI

   commit = client.project("demo").branch("main").commit()
   credential = ConnectionsCredential(
       aws_oidc=ConnectionsAwsOidc(
           role_arn="arn:aws:iam::123456789012:role/spiral-robotics-reader", region="us-east-1"
       )
   )
   created = commit.create_connection(
       "robotics-bucket", "s3://example-robotics/", credential, description="Robot recordings, read-only"
   )
   head = commit.publish()
   print(f"Connection {created.connection_id} at commit {head.commit_id}")

4. Print what the role must trust:

   PythonTypeScriptCLI

   head = client.project("demo").branch("main")
   found = client.catalog.resolve_path("demo", head.commit_id, CatalogResolvePathRequest(path=[name]))
   if found.resource is None:
       raise LookupError(f"no Connection named {name}")
   connection = client.connections.read_connection(found.resource.descriptor.resource_id).connection
   if connection.identity is None:
       print("this Connection stores a key and has no identity")
   else:
       print(f"issuer {connection.identity.issuer}, audience {connection.identity.audience}")
       print(connection.identity.trust_policy)

   The output includes a trust policy like this one, with the Connection's own subject:

   {
     "Version": "2012-10-17",
     "Statement": [{
       "Effect": "Allow",
       "Principal": {
         "Federated": "arn:aws:iam::123456789012:oidc-provider/auth.spiraldb.com"
       },
       "Action": "sts:AssumeRoleWithWebIdentity",
       "Condition": {
         "StringEquals": {
           "auth.spiraldb.com:sub": "conn_24041a03-26ef-45a2-88e2-3d89150ff6f7",
           "auth.spiraldb.com:aud": "sts.amazonaws.com"
         }
       }
     }]
   }

5. Set it as the role's trust policy, unchanged.

Keep the sub condition. Every Spiral Connection, in any Organization, can obtain a token from https://auth.spiraldb.com with the audience sts.amazonaws.com. Only the subject tells your Connection apart, so a role that trusts the identity provider without it trusts every Connection. To let two Connections assume one role, list both subjects in the condition. A Volume uses the same identity provider with its own ID, vol_…, as the subject; to let a Connection and a Volume share a role, list both.

A Connection that you drop and create again has a new subject, and the role must trust the new one. The role must be in the aws partition, because Spiral reads buckets through the endpoints of commercial AWS.

Connect with an access keyCopy Anchor Link

An access key Connection stores the secret of an IAM user, encrypted, and returns it to nobody. Spiral narrows it with GetFederationToken, so the user needs that permission as well as the read policy:

{
  "Sid": "FederateConnectionSessions",
  "Effect": "Allow",
  "Action": "sts:GetFederationToken",
  "Resource": "arn:aws:sts::123456789012:federated-user/spiraldb-connection"
}

Use a dedicated IAM user for the Connection, with only these policies. The CLI reads the secret from a file or from stdin, never from an argument:

commit = client.project("demo").branch("main").commit()
credential = ConnectionsCredential(
    aws_access_key=ConnectionsAwsAccessKey(
        access_key_id="AKIAIOSFODNN7EXAMPLE",
        secret_access_key=Path("aws-secret-access-key.txt").read_text().strip(),
        region="us-east-1",
    )
)
created = commit.create_connection(
    "robotics-bucket", "s3://example-robotics/", credential, description="Robot recordings, read-only"
)
head = commit.publish()
print(f"Connection {created.connection_id} at commit {head.commit_id}")

To move an existing Connection from a key to a role later, see Update a credential.

Create the Connection in StudioCopy Anchor Link

1. Select Create resource → Connection.
2. Enter a name, select the provider AWS, and select the authentication: IAM role (OpenID Connect) or Access key.
3. Enter the URI: a bucket, or a prefix that ends with /, such as s3://example-robotics/.
4. For a role, enter the AWS account ID and the role name, which is the part of the role ARN after role/, with its path if it has one. For an access key, enter the access key ID and the secret access key.
5. Create the Connection. For a role, the Connection page shows the trust policy to set on the role.

Verify the resultCopy Anchor Link

Test the Connection. Spiral obtains a session with the role or the key, and lists one page of objects under the Connection URI:

head = client.project("demo").branch("main")
found = client.catalog.resolve_path("demo", head.commit_id, CatalogResolvePathRequest(path=[name]))
if found.resource is None:
    raise LookupError(f"no Connection named {name}")
result = client.connections.test_connection(found.resource.descriptor.resource_id)
print(f"credential accepted: {result.vend.succeeded}")
if result.access is not None:
    print(f"storage reachable: {result.access.succeeded}")

See Check that a Connection works for each step. The test does not read encrypted objects, so read one to check the KMS grants.

TroubleshootingCopy Anchor Link

Next stepCopy Anchor Link

Create a Location with a root inside the Connection URI, such as s3://example-robotics/recordings/.