SeaweedFS

Relic is built and tested against SeaweedFS. The Compose stack, the Helm chart's bundled store and the end-to-end tests use it (image chrislusf/seaweedfs:4.48).

Why SeaweedFS

• Uploads, range reads, the readiness check, garbage collection, and the backup and restore drill are tested against its S3 gateway.
• Per-path TTLs expire abandoned uploads in temporary without a lifecycle job.
• Replication is configurable: placement 010 keeps a copy on another rack.
• It is Apache 2.0 licensed, one binary, with no external dependencies.

Architecture

Setup

1. S3 identity. Pass this file to the gateway with -config, or with -s3.config for weed server:

{
  "identities": [
    {
      "name": "relic",
      "credentials": [{ "accessKey": "<access-key>", "secretKey": "<secret-key>" }],
      "actions": ["Read", "Write", "List", "Tagging"]
    }
  ]
}

Relic creates no buckets, so its identity needs no Admin. Once identities are configured, the gateway refuses anonymous requests.

2. Buckets and the temporary TTL. This weed shell session creates both buckets and gives temporary a one-day TTL:

weed shell -master=<master>:9333 -filer=<filer>:8888 <<'EOF'
s3.bucket.create -name artifacts
s3.bucket.create -name temporary
fs.configure -locationPrefix=/buckets/temporary/ -ttl=1d -apply
EOF

To replicate artifacts, run fs.configure for /buckets/artifacts/ with -replication=010 (or another placement), or start the master with -defaultReplication.

3. Connection:

RELIC_OBJECT_STORAGE_ENDPOINT=http://seaweedfs-s3.storage:8333
RELIC_OBJECT_STORAGE_ACCESS_KEY=<access-key>
RELIC_OBJECT_STORAGE_SECRET_KEY=<secret-key>

The default region and bucket names work.

Helm:

seaweedfs:
  enabled: false
existingCredentialsSecret: relic          # holds the S3 key pair, adminPassword and encryptionKey
objectStorage:
  endpoint: http://seaweedfs-s3.storage:8333

The Helm chart's bundled SeaweedFS

seaweedfs.enabled=true installs SeaweedFS from its official chart (seaweedfs 4.48.0), one replica of each component:

Relic's values.yaml sets:

• PersistentVolumeClaims of the default StorageClass: 1 GiB (seaweedfs.master.data.size), 50 GiB (seaweedfs.volume.dataDirs[0].size) and 10 GiB (seaweedfs.filer.data.size). Set storageClass next to each size to change the class.
• seaweedfs.s3.enableAuth. The SeaweedFS chart generates the key pair in the <release>-seaweedfs-s3-secret Secret, and Relic reads admin_access_key_id and admin_secret_access_key from it.
• seaweedfs.s3.createBuckets. A Job creates the missing buckets after every install and upgrade. Keep the list in step with objectStorage.artifactsBucket and objectStorage.temporaryBucket.

Nothing is replicated (000). temporary has no TTL, so incomplete multipart uploads stay. These values give it a one-day TTL:

seaweedfs:
  s3:
    createBuckets:
      - name: artifacts
      - name: temporary
        ttl: 1d

For production, run a replicated SeaweedFS, through the SeaweedFS chart's values or the SeaweedFS operator. For one outside the release, point objectStorage.endpoint at its S3 gateway.

Gotchas

• TLS. The S3 gateway speaks plain HTTP unless configured. Keep it on a private network, or give it a certificate from a public CA (no private CA setting).
• Volume size. -volumeSizeLimitMB caps each volume; the bundled store keeps the default, 1000 (seaweedfs.master.volumeSizeLimitMB). -max=0 (maxVolumes: 0 in seaweedfs.volume.dataDirs, the default) sizes the volume count from free disk space.
• Filer metadata is state. Back it up with the volumes, or copy the artifacts bucket elsewhere with an S3 sync tool. See Backup.