ServicesConnections

Hugging Face

Connect a private Hugging Face repository with a fine-grained token, and read revisions of it through Locations.

Available

Connect a private Hugging Face repository once, and everyone in the Project can list and read it through Locations without holding your token. Spiral exchanges the stored token for a short-lived repository token from the Hub, so readers never see the token you stored. See Connections for what readers receive.

A hugging_face Connection names one repository with its type, such as hf://datasets/example-robotics/robot-runs, and stores a token.

Before you start

  • connection:create in the Project. The editor and admin roles have it.
  • The CLI, signed in with spiral login, or Spiral Studio.
  • A Hugging Face account that can read the repository.

Create Connections on the branch main. All branches share one credential for each Connection, so Spiral creates and drops Connections only on main.

Create a fine-grained token

In your Hugging Face settings, create an access token of the type Fine-grained. Under Repositories permissions, select the one repository, and grant Read access to contents of selected repos. Grant nothing else.

Do not store a Read or Write token. A Read token reads every repository that its owner can read, and the Connection would delegate all of them. The Hub counts rate limits for each token owner, so all reads through the Connection count against the owner of this token.

Create the Connection

The CLI reads the token from a file or from stdin, never from an argument:

commit = client.project("demo").branch("main").commit()
credential = ConnectionsCredential(
    hugging_face_token=ConnectionsHuggingFaceCredential(token=Path("hf-token.txt").read_text().strip())
)
created = commit.create_connection(
    "hugging-face",
    "hf://datasets/example-robotics/robot-runs",
    credential,
    description="Robot runs, read-only",
)
try:
    head = commit.publish()
except (spiral.BranchMoved, spiral.PublishConflict) as error:
    raise RuntimeError("Project changed; reload before retrying") from error
print(f"Connection {created.connection_id} at commit {head.commit_id}")

The URI names the whole repository: hf://models/…, hf://datasets/…, or hf://spaces/…. It names no directory and no revision; a Location does.

In Studio, select Create resource → Connection, select the provider Hugging Face, and enter the URI and the token.

Verify the result

Test the Connection. Spiral first asks the Hub to accept the stored token, because the Hub issues a repository token for a public repository with any token. It then obtains a repository token and lists the main branch:

head = client.project("demo").branch("main")
found = client.catalog.resolve_path("demo", head.commit_id, CatalogResolvePathRequest(path=[name]))
if found.resource is None:
    raise LookupError(f"no Connection named {name}")
result = client.connections.test_connection(found.resource.descriptor.resource_id)
print(f"credential accepted: {result.vend.succeeded}")
if result.access is not None:
    print(f"storage reachable: {result.access.succeeded}")

See Check that a Connection works for each step.

Read a revision

A Location over the Connection names the revision in its root, after the repository: hf://datasets/example-robotics/robot-runs@main/data/ lists data/ on the branch main. Name a tag or a commit to pin the file list for every refresh. Write a branch whose name contains /, such as team/curated, as team%2Fcurated, the way file URIs show it. See Create a Location.

What the repository token reaches

The Hub narrows a token to one whole repository, not to a directory or a revision. The short-lived token that a reader receives can therefore read every file of the repository at every revision, even though the Location lists only its own root. Put data that different teams must not share in different repositories, each with its own Connection.

A Connection reads only. The Hub has no object storage to write to, so a hugging_face Connection allows only read.

Found a problem? Edit this page or report an issue.

On this page