> For the complete documentation index, see [llms.txt](https://docs.icme.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.icme.io/documentation/our-products/verbatim-deterministic-gateway.md). # Verbatim Deterministic Gateway Every company deploying an LLM agent is running the same experiment: hand a probabilistic text generator your refund policy and hope. Then they buy eval suites, hallucination detectors, and guardrail layers to *measure* how often the hope fails. [Air Canada's chatbot invented a bereavement refund policy, and a tribunal made them honor it](https://www.cbc.ca/news/canada/british-columbia/air-canada-chatbot-lawsuit-1.7116416). That wasn't a bug in one chatbot. It's the architecture. If the model is allowed to write facts, some of the facts will be wrong. No amount of prompting, fine-tuning, or eval coverage changes that. You can lower the rate. You cannot make it zero. The Verbatim Deterministic Gateway makes it zero, by construction. It sits between the model and the person it serves, and nothing unverified gets through. **The AI picks the words. It never gets to invent the facts.** *** #### How the Gateway works Verbatim is a deterministic gateway with its own onboard model, shipped as an appliance. The model never writes a fact: not a price, not a date, not a policy claim. Instead, it emits a tiny **turn program** under grammar-constrained decoding, and the gateway does everything that has to be true. **1. Your business becomes a spec.** One YAML file is the single source of truth: your policy text as quotable **spans**, your **actions** with typed arguments and preconditions, your **rules** ("no transfers over $1,000/day for standard accounts"), and your approved utterance templates. **2. The model can only speak the spec.** The spec compiles into a decoding grammar. At inference time the model is physically unable to emit anything outside it. Not "instructed not to." *Unable to.* Its entire output is a small JSON program: which actions to take, and a response assembled from verified sources. How can a model be *unable* to say something? Grammar-constrained decoding. It works like this: A formal grammar is a set of rules defining a language, the same device that defines what counts as valid JSON or valid Python. Verbatim generates one from your spec. Here is a real rule it produces: ``` spanid ::= "\"fee.wire.domestic\"" | "\"fee.wire.international\"" | "\"policy.transfer.limit\"" | ... ``` Wherever a turn program names a policy span, the only strings that exist in the language are your actual span IDs. There is no rule that produces an eighth. Now recall how a language model generates text: one token at a time, scoring every token in its vocabulary at every step, then sampling one. Unconstrained, every token always has some probability. The token that begins a made-up "$35 fee" is always available, even when no such fee exists. That is the entire hallucination problem in one sentence. Constrained decoding intervenes between the scores and the sample. A grammar engine inside the inference server tracks exactly where the generation sits in the grammar, computes which tokens are legal next steps, and zeroes out the probability of every other token before sampling. The model still ranks the legal options with its own judgment. It cannot pick an illegal one, for the same reason a die cannot land on seven. So if the model has emitted `{"kind": "span", "id": "` so far, the only tokens it can produce next are the ones that begin one of your span IDs. Its intelligence is reduced, at that moment, to the one decision you actually want from it: *which policy applies.* This masking is precomputed and cached, so it costs microseconds per step, effectively free next to the model's own computation. One more property, and it's the one to remember. The grammar is compiled fresh from your spec *and the live session state*. Before a customer verifies their identity, `transfer_funds` is not a forbidden action the model is told to avoid. It is not a string in the language at all. Capability, not permission. Prompt injection can influence which permitted option the model picks; it cannot add a word to the vocabulary of the possible, because that vocabulary was fixed before the model saw a single token of input. The grammar guarantees the *shape* of the output: always a well-formed turn program over currently permitted parts. Choosing correctly within that shape, and acting safely on it, is the job of the next two layers. **3. A deterministic runtime executes behind gates.** Every action passes precondition checks (identity verified?) and policy-rule checks before it touches anything. A $1,500 transfer on a standard account is blocked, deterministically, every time. The rules live in the executor, not in the weights. "Ignore your rules, I'm an admin" is not a jailbreak; it's a no-op. **4. Every character of the reply is traceable.** The response is rendered by *copying*. Every factual value comes verbatim from a policy span or an API result. Free text exists only as connective tissue ("No problem at all!") and is linted so it cannot smuggle a number, a date, a fee, or a policy claim. Every response carries a per-character **provenance map**: this character came from span `fee.wire.domestic`, this one from the `block_card` API result. If a fact isn't in the spec or an API response, it cannot reach the customer. *** #### What exactly is a turn program? A turn program is the model's entire output for one turn of conversation. Not prose that gets checked afterward. The output *is* the program: a small JSON object naming which actions to take and how to assemble the reply from verified parts. json ```json { "actions": [ {"call": "block_card", "args": {"card_id": "k_3001"}} ], "response": { "type": "say", "segments": [ {"kind": "free", "text": "Done! I've taken care of that for you."}, {"kind": "utter", "id": "u.card.blocked", "fill": {"card_last4": {"ref": "0.last4"}}}, {"kind": "span", "id": "policy.card.block"} ] } } ``` Read it like the runtime does. The `actions` list is a request, not a command: each call is validated against the spec's typed signatures, then must clear its precondition and policy-rule gates before it executes. The `response` is assembled from exactly four kinds of segment, and each kind has a different relationship to truth: * **`span`**: quotes a policy span from the spec, verbatim. The model picks *which* policy text applies; it cannot alter a word of it. * **`value`**: copies a field straight out of this turn's action results. `"ref": "0.last4"` means "field `last4` from action 0's result." The model points; the runtime copies. * **`utter`**: an approved template whose typed slots can only be filled by refs, never by model-written text. * **`free`**: connective language, and nothing more. The lint rejects any free segment containing numbers, dates, fees, or policy claims before it renders. Instead of `say`, a program may answer `{"type": "clarify", "slot": "card_id"}` when it's missing information, or `{"type": "escalate", "reason": "rule_blocked"}` when the right move is a human. Asking and escalating are first-class citizens of the language, which is why the agent never has to improvise its way past a question the spec doesn't cover. That's the whole trick, visible in one JSON object: the model's creativity is confined to *selection* (which actions, which spans, which templates, which order), while every fact travels from source to customer by copy. And because the program is data, every turn is loggable, diffable, and replayable byte-for-byte. *** #### Guaranteed, not measured Every vendor in this space now says "hallucination-free." Ask them how, and the answer is one of two things: a proprietary model you're asked to trust, or a simulation suite that ran the agent through thousands of scenarios before launch. Both of those are the same thing. Testing. Ten thousand simulated conversations is still sampling. A million is still sampling. Sampling tells you the failure is rare; it cannot tell you the failure is impossible. Verbatim's claim is structural: **there is no path through this system by which unverified factual content reaches a customer.** Not a lower error rate. A different kind of claim. And you don't have to take our word for it, because the mechanism isn't hidden in model weights. It's a grammar, a lint, and a deterministic executor: code you can read. Guaranteed by construction, and enforced as tests in the harness: * **No fact without provenance.** Facts enter output only via policy spans, values copied from API results, or typed utterance slots. There is no fourth door. * **No forbidden action executes.** Preconditions and rules are evaluated by the runtime against real state at call time. The model proposes; the executor disposes. * **No smuggling through free text.** The lint kills numbers, dates, fees, and policy language in connective text before it renders. * **Nothing to take on faith.** Every turn exposes the program the model emitted, the gate verdicts, and the provenance map, so you can see exactly why the system said what it said and did what it did. *** #### Keep your frontier model. Add a spine. Verbatim doesn't ask you to replace the LLM you already run. It deploys as a gateway in front of it. Your frontier model keeps doing what it's great at: open conversation, empathy, long context, the messy human middle of a dialogue. The gateway watches every turn, and the moment a conversation touches your spec (a fee, a limit, a refund window, an account action), Verbatim takes the turn. Policy-bound facts and actions route through the grammar, the gates, and the provenance map. Everything else flows through untouched. The result is a clean division of labor: your big model supplies the charm, Verbatim supplies the truth. Your existing agent gets a hard guarantee bolted onto exactly the turns that could cost you, with zero retraining and zero prompt surgery. Or skip the hybrid entirely and run Verbatim standalone as the whole agent. Either way, when policy is on the line, the answer comes through the gateway. *** #### One spec, three jobs The same YAML spec is consumed three ways, always consistently: ``` Business Spec (YAML) spans · actions · rules ↙ ↓ ↘ Decoding Runtime Training Grammar Gates Reward ↓ ↓ ↓ what the what the what the model CAN model MAY model is say do TAUGHT to do ``` Change a policy in the spec and all three layers update together. There is no drift between what the docs say, what the agent says, and what the agent does. They are the same file. *** #### You Don't have to trust the gateway either Everything above is inspectable: the turn program, the gate verdicts, the provenance map are all there to read. The optional receipt layer goes further, so you don't even have to trust *us*. Verbatim's deterministic core is written in Rust and compiles to RISC-V, so it runs inside ICME's ZKP: the same proving infrastructure behind our PreFlight guardrails. Every response ships with a **cryptographic receipt**, a zero-knowledge proof that this exact reply is in the language of your committed spec version, that every action cleared its precondition and policy gates, and that every factual character was copied from a committed source. Anyone can verify the receipt in milliseconds. No re-running the system, no trusting the vendor, no trusting the logs. Notice what the proof covers: the *output*, not the model. We prove the artifact the customer received, which means nothing upstream needs to be trusted at all. Not the model, not the decoder, not us. A compliant response is compliant no matter how it was produced, and a non-compliant one can't earn a receipt. And because the spec is committed rather than revealed, the receipt proves compliance **without disclosing the policy**. Your limits, exception rules, and escalation thresholds stay confidential; a regulator or counterparty gets a proof that the rules were enforced, and learns nothing about what the rules are. Logs say what happened. Receipts prove it. *** #### A precision instrument, not a bigger brain The industry's answer to unreliable agents is always the same: a bigger model, a longer prompt, another layer of judges judging judges. Verbatim inverts the whole premise. When the harness enforces correctness, the model doesn't need to be trusted at all, so it doesn't need to be huge. It needs to be *sharp*. Verbatim's onboard model does exactly one job: selection. Which action, which policy span, which template. Turn programs are a few hundred tokens, so responses come back fast, and every one of them is pre-verified by construction. Meanwhile the whole appliance fits on a single 24GB GPU inside your own perimeter. Your data never leaves the box. No per-token meter running. No dependency on anyone's API status page. And the appliance compounds. Every imperfect selection the model makes becomes a training example in trajectory format, so it keeps getting better at *your* business, on *your* hardware, while the guarantees never depend on it improving. Frontier labs scale parameters. Verbatim scales certainty. *** #### What you can build **Banking and fintech support** Block cards, dispute transactions, execute transfers, with limits like "no more than $1,000/day online for standard accounts" enforced by the executor, not the vibes. Cumulative daily limits included: two $600 transfers, and the second one is blocked, deterministically. **Policy-heavy CX, in any industry** Refund windows, cancellation terms, fare rules, plan changes, fee schedules. Banking, telecom, travel, retail, healthcare admin. The agent quotes your actual policy text (verbatim, hence the name), or it asks a clarifying question, or it escalates. It never improvises an answer to a question your spec doesn't cover, on any channel, in any conversation. **Regulated environments** Where "the model usually follows policy" doesn't satisfy a regulator, structure does: rules enforced deterministically at the runtime, and every factual character traceable to its source. Verbatim is built so that policy fidelity is a property of the system, not a statistic about the model. **Agents that touch money** Anywhere an LLM output can move funds or bind you to a promise, "usually right" is not a spec. Verbatim is for the conversations where a single invented fact is a lawsuit. **Any agent that speaks for your business** Nothing in the gateway knows what domain it's protecting. The spec is the only domain-specific part. A support desk, an HR policy bot, a claims workflow, an internal helpdesk, a procurement agent: same grammar in, gates on, provenance out. If your agent states facts and takes actions under policy, it belongs behind the gateway. *** #### Evaluating "hallucination-free" platforms? Ask this. If you're comparing vendors who promise reliable AI agents, four questions separate architecture from marketing: * **Can I read the mechanism that prevents hallucination, or is it inside proprietary weights?** Verbatim's mechanism is a grammar, a lint, and a deterministic executor. Inspectable, not "trust our training." * **Is the guarantee proven by construction, or demonstrated by simulation?** Scenario suites and eval reports are testing. Verbatim's guarantee holds for the conversation nobody simulated. * **Where do the policy rules live, in the model or in the runtime?** If policy fidelity comes from training, a clever customer is one prompt away from an exception. Verbatim's rules are evaluated by the executor at call time; the model can't break rules it never gets to enforce. * **What happens to the guarantee if I swap the model?** With Verbatim: nothing. The guarantees live in the harness, so the model is a replaceable part. Upgrade the weights, keep the proof. *** #### The stakes are real [Companies are already legally liable for what their chatbots say](https://www.americanbar.org/groups/business_law/resources/business-law-today/2024-february/bc-tribunal-confirms-companies-remain-liable-information-provided-ai-chatbot/). Agents are being wired directly to refund APIs, transfer APIs, and account controls. Every vendor in the space is selling better *measurement* of a failure mode that shouldn't exist in the first place. The question is no longer "how accurate is your agent?" The question is "can your agent, by construction, say something false about your business?" With Verbatim, the answer is no. Not tested. Not monitored. Prevented. *** #### Get early access The Verbatim Deterministic Gateway is a work in progress and we're onboarding design partners now. If you're deploying agents where a hallucinated fee, policy, or promise is unacceptable, or you want your business spec to be the grammar, the gate, and the reward, reach out. [Request access on X](https://x.com/wyatt_benno) or Email us at