llmovoice

Research paper

Voice needs a context layer.

Our paper, Scalable Context Orchestration for Serving LLMs Over Voice, argues that long-running voice systems need explicit context management between streaming audio and model inference.

The problem

Current voice systems commonly treat a conversation as one continuously growing stream. Every new turn can force the model to process a larger history, while conversational content, speaking style, and network conditions remain entangled in raw audio. That makes long sessions increasingly expensive, limits adaptation, and allows network artifacts to contaminate future context.

llmovoice treats this as an architectural problem rather than a prompt-engineering problem. It introduces a dedicated context-management layer that converts continuous, potentially unreliable signals into structured representations the model can selectively retrieve, project, and control.

The architecture

VoicePages

Bounded units representing completed conversational interactions, with audio, transcript, summary, style, and environment state.

VoiceThreads

Logical groups of related pages that preserve task and topic continuity beyond isolated semantic chunks.

Context Projector

A budget-aware mechanism that selects both relevant history and the right representation fidelity.

Context Orchestrator

A unified decision loop that coordinates content, speaking style, and environment-aware runtime control.

Reported results

40.1×
up to lower per-turn cost in long-form conversations
98.7%
up to retained baseline answer quality
52.35%
reduction in speaking-rate alignment error
2.29×
up to improvement in false-trigger rate under volatile networks

Results reported in the April 2026 llmovoice research manuscript across microbenchmarks, end-to-end workloads, simulated network volatility, and real-world case studies.