Hugging Face
This command writes the token where huggingface_hub reads it (HF_TOKEN_PATH,
$HF_HOME/token, $XDG_CACHE_HOME/huggingface/token or ~/.cache/huggingface/token) and
prints the HF_ENDPOINT to export:
relic package setup huggingface hf --write
Export the endpoint:
export HF_ENDPOINT=https://relic.example.com/huggingface/hf
Supported clients
huggingface_hub 1.x and 2.x, with or without hf_xet, and the tools built on it (hf,
transformers, datasets).
create_repo creates a model or dataset repository with a first commit on main that holds
.gitattributes, as the Hub does, so repo_exists and push_to_hub find it. Creating one
that exists answers 409, which exist_ok=True accepts.
hf auth login and hf auth whoami check the token against the endpoint. list_repo_refs,
list_repo_commits and get_paths_info work; Relic lists every revision as a branch, and its
commits carry no title or authors. Through a remote, refs and commits come from the upstream.
Creating or deleting branches and tags, deleting or moving repositories and listing
repositories answer 501.
Xet storage
With hf_xet, huggingface_hub 1.20.0 and later upload Git LFS files over Hugging Face's
Xet protocol, which Relic serves:
- Relic issues a Xet token for one repository, valid for an hour, to a caller who may deploy to it (or read it, for a read token).
- The client uploads chunks packed in xorbs. Relic checks each chunk and xorb hash and stores
the xorb under
.hf-xet/xorbs/in the deployment repository. - The client registers its files. Relic checks their chunk ranges and hashes and rebuilds each into an ordinary item, SHA-256 checked, before the commit adds it.
The result is an ordinary, scanned file that any client downloads over HTTP. Chunks already in the repository are not uploaded again; Xet's global deduplication is not served.
Without huggingface_hub, upload a directory as a new commit of main:
relic package huggingface upload ./bert-tiny hf-local acme/bert-tiny
Download google-bert/bert-base-uncased at revision main:
relic package huggingface download hf-remote google-bert/bert-base-uncased ./bert --revision main