Block releases Dessa voice model for local GPU use

TechCrunch reports that Jack Dorsey’s fintech giant Block has released Dessa, a new voice model developed by the company’s internal research lab. The model is designed to run locally on consumer-grade hardware, specifically targeting machines like the MacBook M4 Max, and is trained entirely on human speech data rather than a hybrid text-and-speech pipeline. Block describes Dessa as a full-duplex model, meaning it can listen and speak simultaneously, a capability that mirrors the GPT-Live model OpenAI released earlier this year. The company asserts that this architecture allows Dessa to capture emotional nuances, such as laughs, pauses, and cries, that are typically lost when speech is transcribed to text before being processed by a large language model.

The details

Block’s researchers emphasize that Dessa is a speech-to-speech model, distinguishing it from competitors like ElevenLabs v4 or Fish Audio Drama 3, which often rely on voice tags to drive emotion in speech generation. According to the company’s blog post, "Interacting with current voice AI can just plain suck. It turns your voice into text and throws away an incredible amount of information text alone can’t carry." Block claims that Dessa’s knowledge stems directly from the conversational data it was trained on, rather than relying on a separate LLM for knowledge retrieval. The model was trained on a single NVIDIA GPU rack, a detail that underscores the efficiency of the training process. Block has shared multiple examples of conversations where the agent displayed different voices, cadences, and emotions, including a scenario where the bot sounded tired after discussing work and another where it appeared to break down emotionally while discussing comfort.

Context

The release of Dessa positions Block in direct competition with major AI labs and specialized voice AI companies. While OpenAI’s GPT-Live established the benchmark for full-duplex interaction, Block’s approach is distinct in its emphasis on local execution and the removal of the text intermediary. This aligns with a broader trend in the AI industry toward more natural, human-like interactions, but Block’s focus on consumer hardware accessibility sets it apart. The company’s background in finance and small business solutions, particularly through Square, suggests a clear commercial pathway for this technology. Square already uses voice AI for ordering, and Block’s in-house model could power a wider range of business and consumer experiences at a lower cost than cloud-based alternatives. The move also reflects a strategic shift for Block, moving from a pure fintech provider to a developer of foundational AI infrastructure.

What it means for developers

For developers, the primary appeal of Dessa is its ability to run locally on consumer-grade hardware, such as a MacBook M4 Max. This eliminates the need for expensive cloud infrastructure for voice interactions, reducing latency and costs. However, there is currently no public access to the model. Block has instead issued a call for safety researchers to test the model, focusing on potential risks such as voice agents tricking people on the other end of a conversation. This suggests that the model is not yet ready for general developer use. Developers should be cautious about the lack of public documentation and API access. The emphasis on local execution means that developers will need to ensure their hardware meets the minimum requirements, and they should be prepared to handle the model’s unique speech-to-speech architecture, which may differ significantly from traditional text-to-speech pipelines. The absence of a public release also means that developers cannot yet integrate Dessa into their products, making this a watch item rather than an immediate tool.

What to watch

• Public Release: Block has not announced a timeline for public access to Dessa. Developers should monitor the company’s blog and social media channels for updates on API availability and documentation.
• Safety Research: The call for safety researchers indicates that Block is aware of potential risks associated with highly realistic voice models. The results of this research could influence the final product’s features and safety guardrails.
• Hardware Requirements: While Block claims the model runs on a MacBook M4 Max, developers should verify the specific hardware requirements and performance benchmarks on different platforms. The model’s efficiency on a single NVIDIA GPU rack during training is a positive sign, but inference performance on consumer hardware will be critical.
• Competitive Response: The release of Dessa is likely to prompt responses from competitors like OpenAI, ElevenLabs, and Fish Audio. Developers should watch for new features or model releases from these companies that address the limitations of current voice AI, particularly in terms of emotional nuance and local execution.