Kubernetes deployment

Both Helm delivery forms use the same chart and the same procedure. They differ only in how usage is accounted, which is expressed in your values file.

Prerequisites

A cluster that meets the requirements, preferably a clean cluster dedicated to this deployment.
Shared storage mounted at the same path on every node.
Registry access configured, and a chart version chosen on the Self Host page — see Releases.
Prometheus installed, or approved for installation.
Reference voice archives prepared, if your traffic uses reference_id.

Set the values you will reuse:

CHART_REF='<chart-reference>'
CHART_VERSION='<chart-version>'
NAMESPACE=fish-audio
RELEASE=fish-audio

fish-audio is the supported default namespace. If you must use a different one, change both the Helm --namespace argument and MODEL_DISCOVERY_NAMESPACE in your values file. Otherwise the edge API looks for the model services in the wrong namespace.

Choose a delivery form

	Offline	Online
`global.offlineMode`	`true`	`false`
Model assets	Served in-cluster by the bundled model warehouse	The same — in-cluster, from the same warehouse
Usage accounting	Local signed ledger on shared storage	Validated and billed against the Fish Audio service
Object storage credentials	Yours to choose — a password shared inside the cluster	The same — the value is yours, but both sides read one secret
Runtime egress	None	The billing endpoint on 443. Enabling `qwen3-aligner` adds huggingface.co in either mode

“Offline” here means runtime-offline. Installation still pulls images and the chart from a registry. For an install with no network at all, see Air-gapped deployments.

Preflight

kubectl version
helm version
kubectl get nodes -o wide
kubectl get storageclass
kubectl describe nodes | grep -E "nvidia.com/gpu|Taints|Labels" || true

Verify on every node that the shared storage path is a real network mount, not a local directory, and that it is writable. Both matter and they fail differently: a node missing the mount quietly gets a local directory and looks healthy while data splits across nodes, and a read-only export stops every Pod before it starts, because Kubernetes creates directories on the volume for each workload. With node access, findmnt -T /mnt/share should report an nfs or nfs4 filesystem. Without node access:

for node in $(kubectl get nodes -o name); do
  echo "== ${node}"
  kubectl debug "${node}" -it --image=busybox:1.36 -- \
    sh -c 'grep " /mnt/share " /proc/1/mounts || echo "MISSING: /mnt/share is not a mount";
           probe=/host/mnt/share/.fish-audio-write-probe.$$;
           if touch "$probe" 2>/dev/null; then echo "writable";
           else echo "NOT WRITABLE: Kubernetes cannot create directories here"; fi;
           rm -f "$probe" 2>/dev/null'
done

Create the namespace and secrets

kubectl create namespace "$NAMESPACE" --dry-run=client -o yaml | kubectl apply -f -

The chart expects three existing secrets. The registry pull secret is created in Registry access. Create the other two:

# Runtime secret for the edge API.
kubectl -n "$NAMESPACE" create secret generic fish-audio-edge-api \
  --from-literal=jwt-secret="$(openssl rand -hex 32)" \
  --dry-run=client -o yaml | kubectl apply -f -

# Object storage credentials. Models are served from inside the cluster in
# either mode, so these authenticate nothing outside it and a placeholder is
# fine — but the in-cluster store serves with them and the workers present
# them back, so change them and restart both sides together.
kubectl -n "$NAMESPACE" create secret generic fish-audio-model-assets \
  --from-literal=aws-access-key-id='<access-key-id>' \
  --from-literal=aws-secret-access-key='<secret-access-key>' \
  --dry-run=client -o yaml | kubectl apply -f -

Secret	Keys	Used by
`fish-audio-registry`	Docker registry credentials	Every workload that pulls a private image.
`fish-audio-edge-api`	`jwt-secret`	The edge API. Generate once and keep it stable; changing it invalidates issued tokens.
`fish-audio-model-assets`	`aws-access-key-id`, `aws-secret-access-key`	Inference worker and vocoder model asset loading. Required in both forms. Models are served from inside the cluster, so these authenticate nothing outside it — the in-cluster store serves with them and the workers present them back. A placeholder is fine; change it and restart both sides together.

For production, prefer External Secrets, Sealed Secrets, or your cloud secret manager over plain Kubernetes Secrets.

Prepare a values file

The chart carries a values file for each delivery form under profiles/, so pulling the chart gets you a starting point:

helm pull "$CHART_REF" --version "$CHART_VERSION" --untar --untardir ./chart
cp ./chart/*/profiles/offline.yaml values.yaml   # or online.yaml

Edit the copy. A later helm pull --untar fails rather than overwriting the directory it created, so upgrading means deleting it first. A profile sets the deployment mode and the shared-storage path. What a cluster usually has to change:

Setting	When
`global.sharedStorageHostPath`	Shared storage is not mounted at `/mnt/share`.
The `MODEL_DISCOVERY_NAMESPACE` entry under `edge-api-production.env`	You install into a namespace other than `fish-audio`.
`sglang.worker.tolerations`, `vocoder.tolerations`, and the matching `nodeSelector`	Your GPU nodes are tainted or you run more than one GPU model.
`offlineS3.enabled: false`, then `sglang.worker.env.S3_ENDPOINT_URL` and `vocoder.env.S3_ENDPOINT_URL`	You serve model assets from a bucket of your own instead of the in-cluster warehouse. Also the one case where `fish-audio-model-assets` must hold real credentials — they become what the workers authenticate to your store with — and you will need the weights themselves from your account team, since they ship inside the warehouse image. All three: leaving the warehouse enabled while pointing the workers elsewhere is refused, and the vocoder is easy to miss.

Image references are pinned in the chart. Change them only when mirroring into your own registry, as Air-gapped deployments describes — a version is otherwise selected with --version, not by editing tags. Talk to your account team before changing replica counts. The profiles ship the single-worker shape the delivery was sized against, and the API, GPU, aligner, normalizer and router deployments read a replica change in a values file on install but not on helm upgrade, where the running count wins. oxidengine, the model warehouse and Redis are the other way round: they take the value every time, so a kubectl scale on those is undone by the next upgrade. Ingress and TLS are disabled by default. Enable them only if you want Kubernetes to manage your access layer; otherwise put your own ingress controller or load balancer in front of the edge-api-production service.

Install Prometheus

The chart exposes metrics through prometheus.io/* pod annotations and creates no ServiceMonitors. An annotation-scraping Prometheus works out of the box:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm upgrade --install prometheus prometheus-community/prometheus \
  --namespace monitoring --create-namespace \
  --set alertmanager.enabled=false

If you already run kube-prometheus-stack, add an annotation-based scrape config instead.

Stage reference voices

A self-hosted deployment never downloads reference audio from the network. Requests that pass a reference_id resolve from Redis, then from a local archive on shared storage. Stage one zip per voice before enabling that traffic:

<sharedStorageHostPath>/<namespace>/<release>/reference-archives/<reference_id>.zip

Each archive holds one or more audio files at the zip root, each paired with a UTF-8 .txt transcript of the same basename:

voice-alice-001.zip
|-- sample-01.wav
|-- sample-01.txt
|-- sample-02.mp3
`-- sample-02.txt

Supported audio extensions are .wav, .mp3, and .opus. Do not nest directories inside the archive, and do not add prefixes to the filename — the file name must match the reference id exactly. If you have no node shell, stage archives through a helper pod that mounts the share, then confirm the edge API can see the file:

kubectl -n "$NAMESPACE" exec deploy/edge-api-production -- \
  ls -lh /mnt/share/reference-archives/voice-alice-001.zip

These archives are durable data, not cache. Back them up and preserve the path across upgrades.

Validate the release

Render the manifests before touching the cluster:

helm show values "$CHART_REF" --version "$CHART_VERSION" > defaults.yaml

helm template "$RELEASE" "$CHART_REF" \
  --version "$CHART_VERSION" \
  --namespace "$NAMESPACE" \
  --values values.yaml

Install

helm upgrade --install "$RELEASE" "$CHART_REF" \
  --version "$CHART_VERSION" \
  --namespace "$NAMESPACE" \
  --create-namespace \
  --values values.yaml \
  --timeout 30m

Wait for readiness

On the first install the inference worker compiles its GPU graphs and the vocoder builds its inference engine, which takes far longer than later rollouts. Wait on every workload in the release, Deployments and StatefulSets alike:

(
  set -e
  resources="$(kubectl -n "$NAMESPACE" get deployment,statefulset \
    -l app.kubernetes.io/instance="$RELEASE" -o name)"
  test -n "$resources" || { echo "No Fish Audio workloads found" >&2; exit 1; }
  printf '%s\n' "$resources" | while read -r r; do
    kubectl -n "$NAMESPACE" rollout status "$r" --timeout=90m
  done
)

The timeout applies per workload, not to the loop. Watch progress with:

kubectl -n "$NAMESPACE" get pods,svc
kubectl -n "$NAMESPACE" get events --sort-by=.lastTimestamp

Smoke test

The acceptance signal is a successful generation, not a health probe. With no ingress configured, port-forward the edge API:

kubectl -n "$NAMESPACE" port-forward svc/edge-api-production 8080:80

curl -fsS --noproxy '*' -X POST http://127.0.0.1:8080/v1/tts \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -H 'model: <model-name>' \
  -d '{"text": "Hello from Fish Audio Enterprise.", "format": "mp3"}' \
  -o smoke-test.mp3

file smoke-test.mp3

The test passes when the request returns HTTP 200 and the file contains playable audio.

Detail	Notes
`model` header	Must match the model name registered by your values file.
`Authorization`, online form	Your Fish Audio API key. Every request is authorized and billed against the Fish Audio service.
`Authorization`, offline form	Any non-empty bearer token is accepted and recorded verbatim as the billing identity. An empty or missing token still returns 401. Use a stable, distinct token per tenant.
Proxies	A local HTTP or SOCKS proxy can break a port-forwarded request with an empty reply. `--noproxy '*'` and `127.0.0.1` avoid it.
Latency	Latency measured through `kubectl port-forward` includes the round trip to the API server and your own network. Measure from inside the cluster for representative numbers.

Also confirm that the GPU workloads see the shared storage mount, that Redis is ready, that reference archives are visible if you use them, and that no pod is in a crash or image-pull loop.

Upgrade

helm diff upgrade "$RELEASE" "$CHART_REF" \
  --version "$CHART_VERSION" \
  --namespace "$NAMESPACE" \
  --values values.yaml

helm upgrade "$RELEASE" "$CHART_REF" \
  --version "$CHART_VERSION" \
  --namespace "$NAMESPACE" \
  --values values.yaml \
  --timeout 30m

If the helm-diff plugin is not installed, review helm template output before applying. Re-run the smoke test after every upgrade and keep the previous revision available until the deployment is accepted.

Roll back

helm -n "$NAMESPACE" history "$RELEASE"
helm -n "$NAMESPACE" rollback "$RELEASE" '<revision>' --timeout 30m
kubectl -n "$NAMESPACE" get pods

Uninstall

helm -n "$NAMESPACE" uninstall "$RELEASE"

Uninstall does not remove data on shared storage. Reference archives, the usage ledger, and compile caches survive and should be deleted deliberately.

Next steps

Air-gapped deployments to install with no network access.
Operations for monitoring signals, scaling, and troubleshooting.

Set Up the SDK

Cookbook

Best Practices

Integrations

Compatible APIs

Self-Hosting

More

Kubernetes deployment

Prerequisites

Choose a delivery form

Preflight

Create the namespace and secrets

Prepare a values file

Install Prometheus

Stage reference voices

Validate the release

Install

Wait for readiness

Smoke test

Upgrade

Roll back

Uninstall

Next steps

​Prerequisites

​Choose a delivery form

​Preflight

​Create the namespace and secrets

​Prepare a values file

​Install Prometheus

​Stage reference voices

​Validate the release

​Install

​Wait for readiness

​Smoke test

​Upgrade

​Roll back

​Uninstall

​Next steps

Prerequisites

Choose a delivery form

Preflight

Create the namespace and secrets

Prepare a values file

Install Prometheus

Stage reference voices

Validate the release

Install

Wait for readiness

Smoke test

Upgrade

Roll back

Uninstall

Next steps