When AI Answers 911, the Governance Question Is Not Whether, It Is Where

In August a headline claimed New Orleans would "use AI to answer 911 calls instead of a human." It spread fast, because it hit the exact fear people carry about AI in public safety: that a machine will decide who lives and who waits. The agency that runs the city's 911 system, the Orleans Parish Communication District, pushed back publicly and called the story inaccurate. Both the panic and the rebuttal are worth studying, because the governance lesson lives in the gap between them.

What New Orleans actually deployed

Here is what the record shows. OPCD has used an AI triage tool since 2023, built with the emergency-communications firm Carbyne and now branded Axon 911. Its job is narrow on purpose. When someone reports a car crash, the 911 center often gets a burst of calls about that same crash. Those duplicate calls stack up in the queue ahead of unrelated emergencies, a heart attack, a shooting, a fire.

The AI only engages under a specific set of conditions: all human call-takers are already busy, and the incoming call originates within about 200 meters of an already-reported vehicle crash. In that case the AI asks one question, whether the caller is reporting that known crash. Anything other than a clear "yes," in any language, routes the caller straight to a human. Executive Director Karl Fasold has been direct about the limits: the system cannot dispatch responders, cannot prioritize calls, and does not hang up on anyone. The caller has to hang up. OPCD says it reviewed 100 percent of the tool's calls for the first three months and recorded no false positives and no false negatives.

That is a long way from "AI answers 911 instead of a human." The Shreveport Times headline that started the panic did not make the distinction, and OPCD said the reporter never contacted them. But notice something: the correction does not make the governance question disappear. It sharpens it. New Orleans did not hand 911 to an algorithm. It drew a line, and the entire safety case rests on exactly where that line sits.

The line is the whole game

Strip away the headline and a clean principle is left. The AI in New Orleans handles a task where the cost of a mistake is low and the human stays one step away. A duplicate crash report is, by definition, information the system already has. If the AI mishandles it, the caller is routed to a person. Fasold said the quiet part plainly when asked about using AI on 311 non-emergency calls versus 911: on the non-emergency side, "if it messes up, it's not a tragedy, it's an inconvenience." On the 911 side, "if an AI was handling something and it messed up, someone could die, and we're never going to take that chance."

That is a risk-tiering decision, whether or not anyone called it that. It is the same judgment every organization deploying AI has to make, in healthcare, in finance, in any consequential workflow. The question is never "AI or no AI." It is: for this specific task, what happens when the machine is wrong, and how far away is the human who can catch it?

Get that placement right and AI genuinely helps. Carbyne reported the triage system handled more than 3,500 events over a 90-day window and cut redundant calls by more than 30 percent among the calls it triaged, which is dispatcher time returned to the calls that actually need a person. Get the placement wrong, push AI one tier deeper than the failure cost allows, and you have built the thing the headline was afraid of.

The failure mode nobody put in the demo

There is a second lesson buried in the New Orleans design, and it is the one I would press hardest in any review. The AI checks whether a caller is within 200 meters of a known crash and asks if that is why they are calling. Reasonable. But proximity is a correlation, not a fact about need. Someone can be standing at the scene of a fender-bender and calling about something else entirely, a person collapsing on the sidewalk, a fight escalating, a second crash the system has not logged yet.

New Orleans handled this correctly by design: anything but a clear "yes" goes to a human, and the system never terminates the call. That is the safeguard that makes the deployment defensible. It is also exactly the kind of safeguard that erodes quietly under pressure. When staffing is short and the queue is long, the temptation is always to widen the AI's lane, to let it handle "just a bit more." OPCD said it is testing whether the tool could handle other call clusters, but only in a test environment, not live. That discipline, the refusal to expand the machine's authority faster than the evidence supports, is the governance control that matters most and the one that shows up in no product demo.

What this looks like as a rule, not a headline

Public safety makes the stakes vivid, but the pattern is general. Every serious AI deployment needs three things decided before go-live, not after an incident.

First, an honest risk tier for each task, driven by one question: what is the cost of the model being wrong, and is that cost reversible? A duplicate-call filter and an autonomous dispatch decision are not the same tier and cannot share the same guardrails.

Second, a human who is genuinely in the loop, not nominally. "Routes to a human on anything but yes" is real oversight because the default is escalation. A system where a human theoretically can override but never actually sees the decision is not oversight, it is paperwork.

Third, a standing rule against scope creep. The New Orleans deployment is defensible because its authority is fixed and narrow, and expansion has to earn its way through testing. Most AI governance failures are not a bad initial decision. They are a good initial decision that quietly grew.

Under all three sits a commitment worth naming. This year Pope Leo XIV devoted his first encyclical, Magnifica Humanitas, to safeguarding the human person in the age of AI, and argued for "discrediting the assumption that technical power automatically confers the right to govern." One need not share the faith to see the engineering translation: the person on the other end of the call is the point, and the machine's job is to get them to the right human faster, never to become the wall between them and help. New Orleans, at least in how it drew its line, built toward that. The headline missed it entirely.

If you are working out where your own line belongs, the structure I use in my own analysis is public. My AI Governance Toolkit includes a four-tier risk classification and a human-oversight standard you can adapt to your context, free, no sign-up. A template does not govern anything. People drawing careful lines do. But it beats starting from a blank page two weeks before a board asks where your line is.