Audio8's CPU-only runtime fits voice cloning in about 1 GiB of RAM

Here is the number that matters in the Audio8 TTS ONNX Runtime release: on a 16 GB Apple M2 MacBook Air with five ONNX Runtime threads, the inference service settles at about 1004 MiB of memory after loading and peaks between 1.1 and 1.2 GiB during synthesis. That is voice cloning and speech generation on a laptop CPU, with no GPU in the picture. The model files come to about 572 MiB online, or roughly 968 MiB if you add the optional voice-registration encoder.

The runtime is the CPU-only companion to Audio8-TTS-Preview-0.6B-ONNX-INT4, the open-weight model Audio8 released under Apache 2.0, which we covered in the model's release announcement. The pitch: 11 languages, zero-shot voice cloning from a short reference sample, a bundled 44.1 kHz codec, aimed at edge speech applications that need speed and privacy. This ONNX port is what makes that pitch practical outside a GPU box: after the weights are downloaded, inference needs neither PyTorch nor Transformers nor the Hugging Face Hub.

What the runtime actually costs

Precision is split by role. The slow and fast autoregressive weights, the part of the model that generates codec codes, are weight-only INT4. Activations, hidden states, and the KV cache stay in FP16, as do the codec encoder and decoder. Only the final waveform output is FP32.

Normal synthesis loads only three sessions: Slow AR, Fast AR, and the codec decoder. The bundled codec is part of why the pipeline stays this light: there is no separate vocoder to bolt on. Synthesis runs in two stages: the autoregressive models emit codec codes, and the codec decoder turns them into audio. The CLI writes those codes to a .npy file beside the .wav output.

Voice registration follows the same logic. A voice profile is a set of codec codes extracted from a reference recording: 0.5 to 30 seconds long, no larger than 50 MiB, readable by libsndfile, and converted to mono 44.1 kHz by the service. The recording must be paired with its exact transcript. The encoder loads only for this step, pushing memory to about 1.55 GiB, then unloads so the synthesis sessions come back. The README is blunt about the weak point: noisy, long, or mistranscribed references reduce stability and speaker similarity.

Built for deployment, not demos

The dependency budget is the quiet news here. Requirements are Python 3.11 or newer, the Hugging Face Hub CLI for a one-time download, and a fixed directory layout. Audio8 has tested the current release on macOS arm64 with the CPUExecutionProvider. From there it runs as an HTTP service on 127.0.0.1:8024, reachable three ways: a web page, a JSON API at /api/tts, and an OpenAI-compatible endpoint at /v1/audio/speech that answers with WAV. Code already written against that API shape can point at a local model instead. Streaming runs over /api/tts/stream, which returns newline-delimited JSON with base64-encoded 16-bit PCM at 44.1 kHz, and /api/tts/cancel stops an active stream. A CLI script covers scripted use, and logs are written to service.log. For context on where this fits in the wider ecosystem, see this roundup of the open-source AI stack, from agents to local inference.

The service serializes synthesis and registration requests, a deliberate trade that bounds memory use at the cost of concurrency. The README positions it for local use and low-concurrency CPU deployments. Thread count, model directory, voices directory, host, and port are environment variables with sensible defaults, and a stop script shuts the managed background service down.

The small-model moment

Audio8 is releasing into a period where small speech models keep closing the gap with big ones, part of the same pattern as the recent small-model wins over bigger siblings. Voxtral Realtime, an open-source streaming speech recognition model from researchers, claims transcription on par with OpenAI's Whisper at a delay of just 480 ms. The 0.6B TTS preview sits under the 1B mark where most open TTS models cluster, and it clones a voice from a short sample without fine-tuning. The open-weight movement is pressing the same argument in public: 41 signatories recently defended open models in a letter. The signature list tells its own story, one about nerves more than consensus. Audio8's contribution differs in kind: it is generative, creating speech rather than transcribing it, and the local runtime ships as a first-class artifact, not an afterthought.

The caveats are in the data. About 1 GiB after load is a working set that suits a 16 GB laptop, not a phone. The serialized request queue caps throughput by design, and measurements vary by operating system and ONNX Runtime allocator behavior, as the README notes. But the direction is hard to argue with: an autoregressive-plus-codec pipeline that would have demanded a GPU box not long ago now renders on CPU threads with room to spare. Local voice cloning is now an engineering choice, not an infrastructure decision, and that puts it squarely inside the fight over who controls AI infrastructure.