How Klangfarbe verifies an AI voice agent's identity

Klangfarbe is an open protocol for verifiable identity of AI voice agents, with a working proof of concept now in build. It answers one question that ordinary phone and WebRTC calls cannot: which organization's authorized AI agent is speaking right now, and for what purpose?

This page is a public summary of the Klangfarbe whitepaper v1.0, a review draft dated 21 September 2026. Nothing described here has shipped yet; the proof of concept is in progress. Timings are MVP engineering targets, not measured results.

The problem: identity disappears at the voice

Before it speaks, an AI voice agent can carry strong identity: a workload identity such as a SPIFFE ID, OAuth client credentials, cloud workload credentials and an organization's authorization. After text to speech and a phone network, the person or system receiving the call gets only audio.

The audio cannot tell them which organization authorized the agent, which agent is speaking, whether the session is valid right now, what it was authorized to do, or whether the identity changed halfway through the call.

Voice biometrics do not help, because a synthetic voice can sound like anyone and voice similarity is not authorization. Audio watermarking can say that speech is machine-generated. It does not say whose authorized agent produced it.

The trust chain

Klangfarbe separates four questions and gives each one its own object:

1. Who is the organization? Answered by a trust framework the verifier already accepts: private enterprise trust, a federation such as OpenID Federation, an EUDI-style trusted list, vLEI, or a Klangfarbe public trust service. No single registry is required.
2. Which agent has the organization authorized? Answered by an Agent Authorization Credential, a verifiable credential that binds an agent identifier and a holder key to purposes, channels and a validity period of days or weeks.
3. Is this a valid session controlled by that agent, right now? Answered by a Session Attestation the agent signs locally for every call. It authorizes a fresh session key, narrows the purpose, and can bind to the SIP Call-ID, a WebRTC session or a verifier challenge.
4. Is this speech bound to that session? Answered by the Audio Binding: a compact, opaque session handle and a rolling authenticator carried inside the speech itself.

A verifier keeps these layers separate and reports which one failed, instead of collapsing everything into a single yes or no.

Agent Authorization Credential

The credential is the organization's statement that a specific agent may act in voice contexts. It can bind to an existing workload identity, for example spiffe://acme.example/agents/claims-assistant, so Klangfarbe adds voice authorization rather than replacing workload authentication. This follows the IETF WIMSE AI Identity Management System draft, which treats AI agents as workloads.

Required claims: issuer, agent identifier, holder key, validity period, authorization scope, status mechanism and profile version. The core credential deliberately leaves out the model vendor, TTS voice, system prompt, phone numbers and transcripts. Those change too often or raise privacy concerns.

Credentials are revoked; sessions expire.

Session Attestation

Issuing a new credential per call would add issuer round trips and confusion. Instead the agent creates a short-lived, signed attestation for each call. A new session key pair limits the damage of a compromised key and gives every call a clean expiry.

Three binding strengths are possible:

• Time-bound: a random session ID, a short lifetime and a holder signature.
• Channel-bound: a digest of the SIP Call-ID, WebRTC session or CPaaS call ID, so the attestation cannot be moved into another call.
• Challenge-bound: a fresh nonce from the verifier or call platform, the strongest live mode against replay.

Audio Binding

The audio does not carry the credential. It carries just enough to find the session evidence, prove that this speech belongs to that session, and keep proving it over time: a profile version, a resolver namespace or context, an opaque session handle, sequence information and a rolling authenticator derived from the session key.

The handle is random and non-enumerable. It never encodes customers, case numbers or phone numbers. A static handle alone would be copyable, which is why the authenticator rolls with the session key.

The binding is designed to sit inside the 300 to 3400 Hz telephone band, so it can survive G.711 as well as Opus. How many authenticated bits survive three seconds of speech after real codecs is the first thing the working PoC measures.

Resolution

A verifier recovers a small handle from the audio and still has to find the Session Attestation.

• Signaling-assisted: SIP, WebRTC or CPaaS signaling carries the issuer and session context, and the audio binding has to match it. Signaling helps locate evidence; it never creates trust on its own.
• Standalone: for a recording with no signaling, the audio carries a compact resolver namespace. A signed, cacheable, mirrorable Resolver Directory maps it to the organization's resolver. It is a bootstrap list, not a database of agents or calls.

Verification states

Klangfarbe reports structured evidence, not a single boolean. The core states are VERIFIED, NO_ATTESTATION, INSUFFICIENT_AUDIO, RESOLUTION_UNAVAILABLE, INVALID_BINDING, INVALID_SESSION, EXPIRED_SESSION, REVOKED_CREDENTIAL, UNTRUSTED_ISSUER, IDENTITY_CHANGED and VERIFICATION_INTERRUPTED.

The rule behind them: presence can prove, absence cannot disprove. Missing evidence is reported as unverified, never as human or safe, and is always kept distinct from evidence that is present but invalid.

What Klangfarbe proves, and what it does not

It can establish that a trusted organization issued a valid authorization for the agent, that the session was created by the holder of that agent's key for a specific purpose and time window, and that the observed speech carries a valid binding to that session for a measured share of the call.

It does not prove that what the agent says is true, that an unmarked speaker is human, that unmarked audio is malicious, which model produced each word, or that deepfakes without Klangfarbe can be detected.

Threats it is designed for

Replay of an old recording, copying a session handle into other speech, splicing, watermark stripping, re-synthesis, resolver spoofing, directory poisoning, compromised issuer, holder or session keys, relay into another call and mixed human and AI speech. Stripping or re-synthesis cannot be prevented by any watermark; the protocol makes sure the result is reported as unverified, not as a false identity.

Where it runs

The MVP targets WebRTC with Opus, SIP and RTP, G.711 A-law and μ-law, PSTN paths and recorded files. Live verification runs where the media stream is reachable: WebRTC apps, SIP clients, contact centers, CPaaS media streams and carrier-side processing. Recordings verify offline with a CLI such as klangfarbe verify call.wav.

MVP targets: a verified identity within 3 seconds of speech on WebRTC, within 5 seconds over telephony, re-verification every 2 to 3 seconds, and no wrong identity in the published test corpus.

Open by design

The specifications, credential profiles, resolver protocol, verifier, CLI, SDKs and test vectors are planned as open source under a permissive license. Verifying a call will never require a su.engineering account. su.engineering intends to offer managed issuance, organization verification, hosted resolvers, enterprise deployments and integrations for teams that want them.

Status

The whitepaper v1.0 is out for expert review, and a working proof of concept is being built now, over the next few months. It covers four test rigs: a live WebRTC call verified within 3 seconds, SIP over G.711 on a real phone path, offline verification of a recorded call, and an attack suite with replay, stripping, agent swap and resolver outage. After the PoC come a telecom pilot on real SIP trunks, a public open-source release, and federation with managed trust. Join as a design partner to shape what the PoC proves first.