The model was speaking fine and lying in JSON

The first report came in as a shrug. An operator said a call "just kind of ended" mid-sentence, the caller still talking, the agent gone quiet, then the session closing itself out like nothing happened. No error the operator could see. No angry customer, even, just a job half-done that someone had to redial. I run Dumbly, a voice intake copilot on the OpenAI Realtime API that sits on roughly 300 to 400 real intakes a day across 20-odd operators, so "just kind of ended" is the kind of sentence that ruins your afternoon. It's vague. It's rare enough to feel random. And it's the caller's time, not mine, so I don't get to file it under "flaky, will look later."

I did the obvious thing first and assumed it was audio. A call that drops mid-sentence feels like a plumbing problem, so I chased plumbing: a connection that dropped, a session that timed out, voice-activity detection deciding the caller had gone quiet when they hadn't. Every one of those is plausible enough to keep you busy for a good while, and they did. I spent longer than I want to admit combing through the wrong layer. I was staring at connection timings and turn-detection settings while the actual failure was sitting one level up, in the part of the transcript I'd been scrolling straight past.

Here's the thing about a voice agent: the voice is the part everyone watches, so it's the last part you suspect. The mouth was working perfectly. The model sounded calm and competent right up to the second it vanished.

What actually broke was a tool call. When the model wants to do something in the real world, look up a record, write down an answer, it emits a function call whose arguments are supposed to be a clean JSON object matching the schema I gave it. Most of the time they are. This time they weren't. The arguments came back as something a JSON parser refuses to accept, close enough to look right at a glance but not actually valid. My handler tried to parse it, the parse threw, and because of how I'd wired the error path, the whole session tore down instead of recovering. The caller heard silence. I heard nothing at all, because the failure looked like a clean disconnect in every dashboard I owned.

Finding it meant giving up on the audio theory and actually reading the raw events instead of the tidy version my dashboards showed me. Once I logged the arguments string before parsing instead of after, the malformed payload was right there, ugly and obvious, the way these things always are once you stop looking where the light is good.

Then came the question of whose fault it was, which is where I got humble. My first instinct was that I'd written the schema wrong, or that my prompt was ambiguous enough to invite garbage. So I rewrote both, tightened the descriptions, made the shape I wanted harder to misread. It helped at the margins and it didn't make the problem go away, which was the clue that the problem wasn't entirely mine to fix. The bad arguments were coming out of the model's own generation, not out of my parsing, and no amount of schema wording fully closes that gap. A model can speak a grammatical English sentence and, in the same breath, emit JSON that no parser will accept. Those are two different faculties, and only one of them was failing.

The short-term fix was defensive and I'm not proud of how long it took me to just do it. I stopped trusting the arguments string. I wrapped the parse in something that catches the throw, keeps the session alive, and gives the model another shot at the tool call instead of letting one bad token end a real person's call. A failed parse became a recoverable event rather than a fatal one. It should have been there from day one. The lesson I keep relearning is that the boundary where a model's output meets my code is exactly where I should assume the least and validate the most.

The longer-term fix wasn't mine to write, and honestly that was a relief. Malformed tool-call arguments are a well-known failure class, and the real answer lives upstream in the model API, not in ever more elaborate parsing on my side. It's the whole point of strict function calling and Structured Outputs: constrain generation to the schema so the arguments come back valid by construction instead of best-effort. Moving my tool definitions onto strict mode did more for reliability than any of my defensive code did, though I'm keeping the defensive code anyway. I'd built a voice agent before this one, an earlier stint on Twilio and ElevenLabs, and every version of this job teaches the same thing eventually. The demo is the voice. The product is everything that has to not fall over when the voice is the only part still working.