We mixed full recordings with aligned 5–20 second clips to avoid overfitting to the scripted recording length.
Post-training research
Inkling post-training
We ran a controlled scaling study to measure how targeted human speech data improves the transcription of exact structured values. We combined VoiceCodeBench-style recordings with English replay data and fine-tuned Inkling using a lightweight LoRA adapter.
A transcript can read fluently and still fail the workflow when one exact value changes.
Product codeSN-7KX-9042one missing digit identifies the wrong product
CLI flag--revert-lastone extra space changes a single flag into two
File pathtests/auth/one missing character points at a different location
Scaling result
Targeted data moved task success by 22.67 points.
The same VoiceCodeBench evaluation was run against the untrained model and the 1-, 25-, and 100-hour adapters. Larger selections produced the clearest gains in both recoverable entities and complete-task success.
Untrained
CTEM86.84%
TSR56.33%
1 hour
CTEM88.06%
TSR59.33%
25 hours
CTEM92.85%
TSR72.67%
100 hours
CTEM94.80%
TSR79.00%
| Checkpoint | VCB WER | VCB CTEM | VCB TSR |
|---|---|---|---|
| Untrained | 2.3748% | 86.8421% | 56.3333% |
| 1 hour | 2.8188% | 88.0567% | 59.3333% |
| 25 hours | 1.8517% | 92.8475% | 72.6667% |
| 100 hours | 1.6107% | 94.8043% | 79.0000% |
Training design
Domain signal and replay in every batch.
VoiceCodeBench-style recordings supplied structured-value speech. We added FLEURS English replay data to keep ordinary English in training and prevent the adapter from specializing too narrowly. We balanced the two sources by supervised transcript tokens, so every optimizer batch contained signal from both domains.
The model learned what the speaker said; canonical entity recovery stayed in evaluation.
Recipe
Adapt the model without retraining it.
A rank-8 LoRA adapter carried the update while the base model and output layer stayed frozen.
Base model and output layer frozen
Targeted speech and FLEURS replay
Gradient accumulation of one
Linear decay over one epoch
100-hour receipts
Small transcript changes, different outcomes.
These examples compare the untrained transcript with the 100-hour result. Each corrected character, separator, or digit changes whether the target value can be recovered.
| Entity | Untrained | 100-hour adapter |
|---|---|---|
| CLI flag | --revert --last | --revert-last |
| File path | test/auth/login.spec.ts | tests/auth/login.spec.ts |
| Environment variable | ALLOW_LIST_CIDR | ALLOWLIST_CIDR |
| Product code | SN-7KX-904 | SN-7KX-9042 |
Where it moved
The largest gains came from structured strings.
Email and postal addresses improved by roughly 30 CTEM points. File paths, environment variables, and IP addresses also moved substantially—categories where one symbol or character can determine whether an action succeeds.
Evaluation
Measure the workflow, not one aggregate score.
CTEM measures whether individual target entities are recoverable. TSR requires every target entity in a recording to be correct. Non-entity WER tracks transcription quality outside the marked values, and severe repetition is checked separately.
- 01
139 previous entity misses became recoverable.
- 02
21 previous entity hits regressed.
- 03
Correct entities increased from 1,287 to 1,405.