22% of Our Training Set Was Invented by Another Model

The fabricated examples carried the highest sample weight in the run — and they were teaching the model to say a catchphrase instead of answering the question.

Summary

Our persona fine-tune produced answers that felt hollow — asked a substantive question, it would return a catchphrase and no content. We traced it to the dataset. About 22% of the training rows were generated by another language model rather than drawn from real transcripts, and the most fabricated category was oversampled 4×, giving it the highest per-example weight in the run. Then a correction from the person who knew the subject inverted our fix, and the real defect turned out to be subtler than "synthetic data bad."

Part 3 of a series on a consented voice clone. No names, no audio.

The audit

The dataset builder composes rows as mono + conv×3 + catch×4 + multi×3 — 10,346 rows total:

About 22% fabricated.

The builders said so in their own docstrings. One "generates training examples that teach the model to deploy catchphrases." Another notes that the generator "plays BOTH sides." A third admits its predecessor over-used two specific phrases. Only the real-Q&A extractor was constrained to authenticity — the answer is his actual words, with the model used solely to reconstruct the question and clean up transcription noise, explicitly instructed not to invent content.

Nothing was hidden. Every one of those scripts was written deliberately, by us, for good reasons at the time. The fabrication was documented and then forgotten, and the weights were never audited at all.

The oversampling is the part that turns a data-quality issue into a behavioral one. A 3.4% share sounds negligible. At 4× oversampling it was the most heavily weighted category in the run — the model saw those examples more often than anything else per unique example.

The behavior it produced

Asked about a golf swing, the model replied with a variant of "have no fear, I'm here to fix it."

Filler. Not an answer.

And it was double-reinforced: fabricated Q&A at 4× weight in the fine-tune, plus a bulleted catchphrase list in the system prompt. Fixing either one alone would have accomplished nothing, because the other would have kept the behavior alive. That's worth flagging generally — when a behavior is reinforced in both training and prompting, a single-sided fix reads as "the fix didn't work" and sends you looking for a deeper problem that isn't there.

The correction that inverted the plan

Our plan was obvious: authentic-only rebuild. Keep the monologues and the real Q&A, drop everything invented.

Then the person who actually knows the subject corrected a premise we'd never questioned:

He won't say the catchphrases on camera — these are personal catchphrases for a limited audience.

The phrases are authentic. They're simply absent from every recorded transcript.

That kills the plan entirely. Transcript-only training produces a model that never says them, and they're the entire point of the thing for the room it was built for.

This is the most transferable moment in the project. We had a clean audit, a clear metric (percentage fabricated), and a defensible plan — and the plan was wrong because it optimized a proxy. "Authentic" meant "appears in the transcripts," and the transcripts are a biased sample of the person. A recorded corpus captures the register that gets recorded.

Domain knowledge beat corpus statistics. No amount of dataset analysis would have surfaced that, because the evidence was defined out of the corpus.

The real defect: substitution, not presence

With the premise corrected, the actual problem became visible.

The fabricated pairs teach the phrase to stand in place of an answer:

Q: "What did you think of the new drug's Phase II results?"
A: "Those numbers are off the charts—[catchphrase]!"

No content at all. Same shape as the golf-swing reply.

The correct shape is a substantive answer in his real voice with the phrase as garnish — not as the payload.

So the fix became:

• Keep the phrase list. They're real.
• Drop the 4× oversampling. That's what makes it a reflex rather than a flourish.
• Rebuild the catchphrase examples from real transcript answers with a phrase appended where it naturally fits. Content 100% authentic, only the garnish added.
• Drop the generated Q&A and multi-turn dialogues outright. Both halves invented, and both replaceable.

"Reduce the weight and change the shape" instead of "delete the category." A more careful fix that only became visible after the premise was corrected.

Replacing the invented data with real dialogue

The one thing the fabricated multi-turn data provided was conversational context carry-over. Dropping it costs that.

The replacement was sitting in the pipeline, being discarded. Our speaker-isolation step classifies every 4-second window by speaker, then throws away the non-target audio. Keep the labels instead, and the multi-speaker recordings yield genuine exchanges — someone asks a question verbatim, he answers verbatim.

The best authentic multi-turn data available, and it requires no invention at all.

Yield from the first processed file: 55 minutes in, 33 minutes of target speech, 369 speaker turns, and 40 question-to-answer handoffs. Extrapolated across the corpus that's roughly 12,000 real pairs against the 840 authentic pairs in the existing dataset — about 14×, comfortably enough to drop all 2,261 fabricated rows without losing volume.

The signal was already being computed and thrown away. Before generating synthetic data, check whether the real version is a discarded intermediate in a pipeline you already run.

The register-balance warning

One trap we caught before it cost anything. The bulk of the available recordings are non-conversational narration. The conversational material is a small fraction of the total.

Concatenating naively would swamp the register the system actually needs with a 20:1 ratio of the register it doesn't.

Weight by register, not by hours. More data is not better data when the additional data is a different distribution. A corpus that's 95% one speaking style will produce a model that speaks that way regardless of what you ask it.

Key takeaways

1. Audit sample weights, not just row counts. Our most-fabricated category was 3.4% of rows and the highest-weighted thing in the run.
2. Synthetic data can teach a shape you didn't intend. Ours taught "say the phrase instead of answering," which is a different defect from "says the phrase too often."
3. A behavior reinforced in both training and prompting needs both fixed. Either alone reads as "the fix didn't work."
4. "Authentic" defined as "in the corpus" is a proxy. A recording captures the register that gets recorded, and someone with domain knowledge can tell you what's missing.
5. Check whether the real data is already a discarded intermediate. Ours was: the isolation step computed the speaker labels and threw them away.
6. Weight by register, not by hours. A 20:1 imbalance toward the wrong speaking style makes more data actively harmful.
7. The fabrication was documented in our own docstrings and forgotten. An audit is not a substitute for anyone reading them, but it's what finally did.