We Built a Voice Clone and Never Marked It as Synthetic

Every technical choice in the stack held up against 2026 practice. The gap was governance: no provenance marking, on a system whose entire output is synthetic speech of a real person.

Summary

We reviewed a consented, locally-hosted voice clone against current practice. The component choices held up — the architecture, the isolation pipeline, the quality gates were all defensible. The real gaps weren't technical. The system had no provenance marking of any kind on output that is, definitionally, synthetic speech of an identifiable real person. EU AI Act Article 50 transparency obligations are in force. This post is about what we should have built in from the start, and it applies to anything that generates a real person's voice or face, private audience or not.

Final part of a series on a consented voice clone. Nobody is named, no audio is published.

What consent actually covered

The clone was built with the subject's agreement, for a small private audience of people who know him. That's a real and meaningful permission, and it's the reason this project was reasonable to build at all.

It's also narrower than it feels, in ways worth being precise about:

Consent to be cloned is not consent to any output. He agreed to the existence of a system that speaks in his voice. He did not review, and could not review, the specific sentences it would generate. A generative system's output space is not something anyone can consent to in advance.

Consent for one audience is not consent for distribution. A private room of people who know it's a gag is a completely different context from a clip escaping into a group chat where nobody has that context. The audio doesn't carry the room with it.

Consent has no expiry unless you give it one. We never discussed how long, or how he'd withdraw it, or what happens to the model weights and the harvested audio if he did.

None of that makes the project wrong. It makes the artifacts — the weights, the corpus, the generated clips — things that need a lifecycle, and we hadn't given them one.

The gap: nothing was marked

Every clip the system produced was unmarked audio. Nothing in the file, nothing in the metadata, nothing audible.

If a clip left the room, it was indistinguishable from a recording.

That's the whole problem, and it's independent of intent. A private system with good intentions produces bits that are exactly as portable as a malicious system's bits. The only thing that travels with the audio is what you put inside it.

Three layers, cheapest first — and we had none of them:

Content credentials in the file. C2PA is the interoperable standard for signed provenance manifests. For synthetic audio it records what generated it, when, and with what inputs, cryptographically signed. It survives copying. It does not survive re-encoding by a platform that strips metadata, which is why it can't be the only layer.

An audio watermark. Perceptually inaudible, survives re-encoding and transcoding, detectable by tooling. This is the layer that still works after the file has been through a messaging app. Open implementations exist and are not hard to integrate.

A spoken disclosure. For a system that talks, the simplest and most robust marking is the system saying what it is. Ugly for a comedy bit, and it's the one that works on a phone speaker with no tooling at all.

The first two cost an afternoon each. We didn't do them because nobody asked, and there was no point in the build where the question naturally arose. That's the actual failure — not a decision to skip provenance, but never having reached a decision at all.

Article 50 is in force

EU AI Act Article 50 sets transparency obligations for systems that generate synthetic audio, image, video, or text, and for deep fakes specifically. Providers must ensure outputs are marked in a machine-readable format and detectable as artificially generated; deployers of deep fakes must disclose that the content is artificially generated.

The obligations reference technical solutions that are "effective, interoperable, robust and reliable as far as this is technically feasible" — which is what C2PA and watermarking are for.

Whether a private, non-commercial gag among consenting friends falls inside scope is a lawyer's question and I'm not going to pretend to answer it. The engineering point stands regardless of scope: the marking is cheap, it's the thing that makes the artifact safe when it leaves the room, and the regulatory direction is unambiguous. Building it in is straightforwardly correct, and arguing about applicability is more expensive than just doing it.

The retention question nobody asked

Beyond marking, the artifacts themselves:

• The harvested audio, plus a much larger raw corpus, sitting on disk indefinitely.
• Model weights encoding his voice, which is biometric data under most frameworks.
• Generated clips, cached, with no expiry.

The questions that should have been settled on day one and weren't: How long is any of this kept? What's the deletion path if he changes his mind? Who else can reach these machines? Is the voiceprint — built from a handful of real recordings for our similarity gate — treated differently from the rest?

For a private project on personal hardware, the answers can be simple. They just have to exist. "We'll delete it if he asks" isn't a retention policy if nobody has written down where "it" is.

The corpus is the part most easily overlooked. The clone is obviously sensitive. Someone's recorded speech, harvested and diarized and indexed, is also sensitive, and it's the thing that would let anyone rebuild the clone from scratch.

What we'd build in from the start

Not a compliance program — five things, mostly an afternoon each:

1. Watermark every output at the synthesis boundary, so nothing can be generated unmarked. Inside the server, not in the calling code, where it can't be bypassed by a new client.
2. Attach content credentials to any file that gets written to disk or served.
3. Set a retention period on the corpus, the weights, and the cache, with an actual deletion script rather than an intention.
4. Write down the consent scope: who the audience is, what withdrawal looks like, what gets deleted when it happens.
5. Keep the subject able to hear what it says. The cheapest safeguard for the consent-can't-cover-outputs problem is that the person can review the range of what's being produced and say "not that."

Item 1 is the one that matters most, because it's the only one that protects the artifact after it leaves your control.

The technical review, briefly

For completeness, since the review covered both: the component choices held. The architecture, the speaker-isolation approach, the quality-gate design, and the decision to run everything locally all remain defensible against current practice.

Two real technical gaps surfaced alongside the governance ones: the persona model would benefit from a more efficient quantized base, and there's no streaming path, so a long reply is a long wait. Both are ordinary backlog items.

That contrast is the reason this post exists. A system can be technically sound in every component and still be missing the thing that makes it safe to exist — and the missing thing costs less than any of the optimizations we'd spent weeks on.

Key takeaways

1. Consent to be cloned is not consent to specific outputs. No one can review a generative output space in advance; the mitigation is keeping the subject in the loop, not a signature.
2. Consent for one audience is not consent for distribution. The clip doesn't carry the room with it.
3. Unmarked synthetic audio is indistinguishable from a recording, and good intentions produce identical bits to bad ones.
4. Mark at the synthesis boundary, not the client. It's the only place it can't be bypassed.
5. Use two layers. Content credentials survive copying; a watermark survives re-encoding. Platforms strip metadata.
6. EU AI Act Article 50 transparency obligations are in force. Scope for private projects is a lawyer's question; the marking is cheap enough that the question is more expensive than the answer.
7. The harvested corpus is as sensitive as the model. A person's recorded speech is what lets anyone rebuild the clone.
8. A retention policy nobody wrote down is not a retention policy.