The AI on the case file is not the one you bought

The sanctioned product is sitting in a procurement folder. The AI on the case file is sitting in someone's phone.

An officer pastes a witness statement into ChatGPT because the report is due before end of shift. A detective drops a screenshot of a CAD note into Gemini and asks it to clean up the timeline. A records clerk feeds a draft supplement into Claude and takes the rewrite. None of that went through IT. None of it is on the approved-software list. The department cannot say where the prompt went, whether the vendor retained it, or who else can see it.

The demo is not the risk. The risk is case data already leaving through personal AI accounts the inventory does not list.

A health system has the same shape with PHI. A nurse or a coder pastes into a consumer chatbot that has no Business Associate Agreement. The buyer gate is different. The failure is the same.

What it looks like on a shift

Shadow AI in public safety is not a secret lab. It is paste, screenshot, and draft.

Someone copies three paragraphs from a case file because the writing is messy and the chatbot is fast. Someone photographs a whiteboard from a briefing and asks the model to turn it into bullets. Someone dumps an interview outline in and asks for a tighter narrative. The output comes back clean. The work gets done. Nobody files a ticket.

That is why it spreads. The official path, if there is one, is slow or missing. The consumer tool is already on the phone they used to look up a statute. You do not need a policy violation to get here. You need a deadline.

I spent 13 years in law enforcement. I know what a shift does when the report is due and the sanctioned system is clunky. People use what is in their pocket. That is not a morality play. It is how the work gets out the door. The problem for a founder, a CEO, a board, or a COO is what happens next. Criminal Justice Information, or something close enough that a reviewer will treat it that way, is now sitting in a consumer chat history the agency does not own.

Consumer tiers fail the buyer gate

ChatGPT, Gemini, and Claude as personal or consumer accounts are not a CJIS-eligible system. They are not a HIPAA system either. There is no CJIS authorization. There is no BAA. There is no FedRAMP boundary the city or the health system signed. The contract, if any, is between the vendor and a private email address.

A buyer will not argue model quality with you. They will ask questions the consumer tier cannot answer.

Where did the prompt go. Consumer products route through vendor infrastructure the agency does not control. Region, sub-processors, and default logging follow the terms the employee clicked, not the agency's.

Is it retained, and for what. Training use, abuse review, and vendor logs are not an agency records schedule. If you cannot point to a written retention rule the agency owns, you cannot answer a discovery request or a CJIS question with evidence.

Who can see it. The employee can. The vendor can, under its own policies. The agency often cannot. There is no agency-owned audit trail that says which case number went in, which user sent it, and what came back.

Same model on an enterprise contract with the right clauses is a different animal. Same model on a personal free account pointed at a case file is unmanaged disclosure. The technology is not the tell. The account, the contract, and the log are.

What a city, county, or health system will ask

They will not start with your architecture diagram. They will start with a list.

Name the tools and the tiers. Consumer, team, enterprise, on-prem. If you cannot tell them apart on paper, you do not have an inventory. "We use AI" is not an answer. "These named systems, at these tiers, under these contracts" is.

Who is allowed to act. Which roles may paste case data or PHI into a model. Which roles may only use a sanctioned workspace. Which roles may not use generative AI on records at all. If privilege is "use your judgment," the buyer hears that you have not decided.

Where prompts go. Region, vendor, sub-processors, training, retention, logging. If the answer is a help-center article the employee found, that is not the agency's answer.

What happens before anything becomes a record. Human review is a named step: a person reads the output before it is filed, dispatched, billed, or sent to counsel. If the chatbot draft can land in RMS, the EHR, or a court packet without that step, you have given the model a write to the official record.

Those four questions are the gate. A polished demo does not clear them. A named inventory and a privilege line do.

What good looks like on paper

Good is boring on purpose. A reviewer should be able to forward a short packet without you on the call.

A named inventory. One row per tool, per use, per tier. Who owns it. What data class it can touch. Whether CJIS, a BAA, or another buyer framework actually covers that row. Personal accounts used for work get their own rows. Hiding them is how you fail the first question.

A privilege matrix. Who may prompt, who may approve, who may never point the tool at a case file or a chart. Autonomy scored against action class. Draft a report is not the same as file a report. Summarize a note is not the same as send a message to a patient or a victim. Read, draft, and write to the record are different privileges. Score them that way.

A writeup a reviewer can forward. Not a slide that says you take security seriously. A short document that answers inventory, privilege, data flow, human review, and stop. If a city attorney, a privacy officer, or a board member has to call you to decode it, it is not buyer-ready.

None of that requires a named product. It requires someone to sit still long enough to write down what is already running.

If you need that written down for a public safety buyer or a health system, that is the work. Named inventory, privilege matrix, autonomy scored against action class, and a packet a reviewer can forward. Most packs ship in 30 to 45 days once access and inputs are in hand. contact@carbene.ai