How targeted speech data moved Inkling

Voice agents do not fail gracefully when a transcript changes one key entity. A missing character in an email address, an extra underscore in an environment variable, or a split command-line flag can turn an otherwise readable transcript into the wrong action.

We built VoiceCodeBench to measure those failures. Then we used the same problem to test a practical question: how much can targeted human speech data move an already capable speech model?

We post-trained [Thinking Machines' Inkling](https://thinkingmachines.ai/inkling/) on selections of 1, 25, and 100 hours of VoiceCodeBench-style speech. At 100 hours, entity recovery reached 94.80%, task success reached 79.00%, and non-entity VoiceCodeBench WER fell to 1.6107%.

This is how we built the training data, adapted the model, and evaluated the result.

Start with the failure mode

Word Error Rate is useful, but it can hide the errors that matter most to a voice workflow. Most words in a command may be easy to transcribe while the one value that controls the next agent action is wrong.

VoiceCodeBench separates that problem into three measurements:

• CTEM measures whether each target entity is recoverable from the transcript.
• TSR measures whether every target entity in a recording is correct.
• Non-entity WER measures ordinary transcription quality outside those marked entities.

The evaluation set contains 300 workplace-style recordings and 1,482 target entities, including email addresses, phone numbers, file paths, commands, CLI flags, environment variables, IP addresses, account identifiers, dates, and measurements.

Our training goal was not to teach Inkling a lookup table of canonical values. It was to make the model better at hearing the acoustic structure that makes those values recoverable.

Two data sources, two jobs

Every run mixed two kinds of training speech.

VoiceCodeBench-style scripted recordings supplied the domain signal. These recordings contain the spoken forms of structured values and the surrounding workplace language in which they appear.

[FLEURS English speech](https://huggingface.co/datasets/google/fleurs) supplied replay data. Its job was to keep a wide range of ordinary English present throughout post-training instead of allowing every optimizer batch to become narrowly specialized.

Contributors in training were kept separate from the evaluation data.

Mix full recordings with aligned clips

Our scripted recordings had a relatively consistent duration. Training only on the complete recordings risked teaching the model to expect that length, rather than preparing it to handle speech of different lengths. We therefore represented approximately half of the domain duration as full recordings and half as shorter clips.

We created complete, non-overlapping clips of approximately 5–20 seconds by aligning each recording with its approved transcript. A source recording appeared in only one form: either as the full recording or as all of its clips. This kept the same speech from being counted twice.

If we could not align a recording safely and completely, we kept the full recording. That fallback added variety in duration without silently dropping or duplicating speech.

Train on what the speaker said

The target was the complete approved spoken transcript.

If a speaker said “D one two dash P L zero eight,” that spoken sequence remained the training target. We did not replace it with the canonical written entity D12-PL08 during training.

That distinction matters. Post-training taught acoustic transcription; canonical entity recovery remained an evaluation problem. Keeping those responsibilities separate made CTEM an honest measurement rather than a training shortcut.

Balance the mixture by supervised tokens

The two datasets have different transcript lengths, so equal recording counts would not produce an equal training signal. We balanced the mixture by supervised target tokens instead.

Across a run, approximately half of the supervised tokens came from VoiceCodeBench-style speech and half from FLEURS. Every optimizer batch contained examples from both domains, while the number of recordings from each source could vary.

The order was deterministic. When the pinned FLEURS split was exhausted, replay continued in deterministic cycles with cycle-specific example identities. That kept the experiment reproducible without pretending every replayed item was a new recording.

Adapt a small part of the model

We did not fully retrain Inkling. We attached a rank-8 LoRA adapter and trained the adapter while leaving the base model and output layer frozen.

The recipe used:

• Microbatch size: 32
• Gradient accumulation: 1
• Effective batch size: 32
• Starting learning rate: 1.75e-4
• Learning-rate schedule: linear decay toward zero
• Run length: one complete epoch

Evaluate the workflow, not one score

Each completed adapter was evaluated on the same standard 300-item VoiceCodeBench set. We measured CTEM, TSR, non-entity WER, and severe repetition. We also evaluated FLEURS validation and test recordings separately to keep general English visible throughout the experiment.

The results below show the untrained model and the three scaling checkpoints used in our report.

The first hour produced a modest improvement in entity recovery and task success, but not in VoiceCodeBench WER. The larger data additions changed the curve more substantially. From the untrained model to 100 hours:

• CTEM increased 7.96 percentage points.
• TSR increased 22.67 percentage points.
• VoiceCodeBench WER decreased 32.2% relative.
• Correct target entities increased from 1,287 to 1,405.

Across individual entities, 139 previous misses became correct and 21 previous hits regressed, for a net gain of 118 exact entities recovered.

What those gains look like

The aggregate scores reflect small transcription differences with large downstream consequences.

The largest entity-type gains were email addresses at +30.8 points, postal addresses at +30.0, file paths at +23.5, environment variables at +22.9, and IP addresses at +20.0.

These are not cosmetic edits. They determine whether mail is delivered, a command runs with the intended flag, software opens the right file, a configuration lookup succeeds, or a product is matched to the correct record.

Targeted human data still moves strong models

Inkling started from a strong baseline. One hundred hours of targeted speech did not replace that capability; it adapted the model toward the exact values that voice agents need to recover.

That is the central result: when the failure mode is specific, well-designed human data can move the production metric by much more than a broad aggregate score suggests.