Fish Audio Drama 3 is a speech model with a 35B mixture-of-experts backbone and about 3B parameters active per token. That active count does not describe its resident weight memory. Fish Speech’s documented FP8 deployment requires at least 48 GB of NVIDIA GPU memory; quantized alternatives have different requirements and evidence.
The useful question is which checkpoint, runtime and speech workflow your machine can support. A model that can clone a voice is not automatically a server that exposes every editing feature described in the research paper.
Why 3B active does not mean a small download
The Drama 3 model card describes FP8 experts with the remaining weights in bf16, a hybrid-attention backbone and a ten-codebook audio decoder. Experts reduce the work performed for an individual token; they do not make the other experts disappear from a normally resident deployment.
The technical report explains how the audio path differs from ordinary text generation. One audio frame occupies one backbone position, while a separate decoder predicts its acoustic codes. The tokenizer produces roughly 21.5 frames per second of speech. References and earlier takes therefore occupy meaningful context even when the target script is short.
For capacity planning, distinguish downloaded checkpoint files, engine weight allocations and the full serving process. The last includes caches, preprocessing, audio decoding and runtime reservations. Adding these categories together without checking whether a figure already includes them can overcount memory; ignoring them can make a weight-only estimate look like a hardware recommendation.
The supported runtime determines the requirement
| Deployment path | Published memory evidence | What to check |
|---|---|---|
| Fish Speech, FP8 | At least 48 GB GPU memory | NVIDIA CUDA; Linux or WSL documented |
| vLLM-Omni, original checkpoint | 48 GB+ CUDA GPU; reference run on H100 80 GB | Its speech API omits some editing and chunking features |
| SGLang-Omni, int4 conversion | 20.8 GB engine weights; not total server memory | GPTQ-Marlin requires compute capability 8.0+ |
Fish Audio’s installation page explicitly excludes CPU, Apple MPS and AMD ROCm for its Drama 3 implementation. Its S2-Pro instructions on those same pages have broader device support; those instructions do not establish Drama 3 compatibility. Community ports may implement different execution paths, but their requirements must be assessed separately.
The vLLM-Omni recipe specifies vLLM 0.31 or newer with a Drama-capable vLLM-Omni build. It reports a warm H100 run, not a general promise for all 48 GB cards. Its API currently omits continuation editing and long-text chunking. It also notes that some takes continue emitting silence and that leading event tags can be less reliable. A minimal TTS request and a directed long scene exercise different capabilities.
What int4 actually saves
The int4 conversion’s model card labels it an unofficial quantization, credits AK and reports 20.8 GB of engine weights versus 37.0 GB for the original path. That is a calculated reduction of about 43.8%, using (37.0 − 20.8) ÷ 37.0.
The conversion keeps parts of the system, including the audio decoder, at higher precision. Its card reports testing on A100-80GB and H200, while full serving on 48 GB and 32 GB cards remains unverified there. A 20.8 GB weight footprint is therefore insufficient evidence for a complete 24 GB deployment.
The original model’s displayed Hugging Face parameter estimate and a packed quantized tensor count can also differ from the architecture’s conceptual parameter count. Use the checkpoint identity and measured serving allocations rather than a sidebar parameter badge to plan the machine.
Start with one runtime and plain speech
For SGLang-Omni, follow its Drama 3 cookbook and installation prerequisites. With an appropriate CUDA environment and uv already installed, the documented source path is:
git clone https://github.com/sgl-project/sglang-omni.git
cd sglang-omni
uv venv -p 3.12 .venv
. .venv/bin/activate
uv pip install --prerelease=allow -e '.[fishaudio]'
hf download fishaudio/drama-3
sgl-omni serve --model-path fishaudio/drama-3 --port 8000After startup completes, a plain request avoids reference-audio handling during the first check:
curl -X POST http://localhost:8000/v1/audio/speech \
-H 'Content-Type: application/json' \
-d '{"input":"The recording session starts after lunch."}' \
--output first-take.wavFor the int4 path, the cookbook names fishaudio/drama-3-int4; substitute that checkpoint in both download and serve commands. Its loader reads the quantization configuration. Keep startup logs showing allocations and selected kernels, then check actual process memory with a small request before increasing prompt length or concurrency.
The cookbook’s defaults reserve memory for multiple pipeline stages. It warns that increasing the engine’s memory fraction while sharing a GPU with the vocoder can exhaust memory for concurrent non-streaming jobs. Reducing weight precision does not remove that allocation problem. Start with the default placement rather than copying high-concurrency H200 tuning flags onto a smaller card.
Keep the API examples matched to the server
SGLang-Omni uses /v1/audio/speech with an input field. The native Fish Speech inference guide documents /v1/tts with a msgpack body and supplies its own API client. Identical model weights do not make these request formats interchangeable.
On the native server, the model is selected at startup. The client has explicit reference audio, transcript and speaker arguments, and Drama 3 uses speaker tags to assign roles. --compile is not supported for Drama 3 in that implementation, even though nearby S2-Pro examples use it.
After plain speech works, add a permissioned reference recording and its exact transcript. Then test multiple speakers, delivery instructions or editing individually. Save the source audio, script, checkpoint, runtime revision and settings with each take. That record makes a failed edit reproducible and prevents a successful short sample from standing in for a long-form evaluation.
Check the license file, not just the metadata badge
At the time of this check, the original card’s metadata still said research license, while its actual LICENSE file contained Fish Audio Community License v1.0, dated October 13, 2026. That discrepancy matters when selecting a production checkpoint.
The file permits research and non-commercial use and sets conditions for commercial use below a USD 500,000 annual-revenue threshold. Hosted speech services have separate rules and stated exceptions; the threshold alone does not authorize every service. Voice consent, attribution and other restrictions also apply. Read the file shipped with the exact revision you deploy rather than treating “open weights” as a complete permission statement.
For local evaluation, the original CUDA path has the clearest published hardware floor. Int4 offers substantial weight savings, but a smaller verified complete-server configuration would be needed before recommending it for a particular consumer GPU. The next useful measurement is peak memory and successful audio on the intended workload, not the active-parameter count.
Publisher information: Prepared with AI assistance from primary model and runtime documentation, checked October 18, 2026. Hardware figures are source-reported; the 43.8% weight reduction is an EyesTech calculation. No independent inference benchmark was performed. The cover is an original AI-generated conceptual illustration.
