Procurement Checklists for High-Stakes AI

Procurement is where a lot of AI risk is quietly accepted. A system is evaluated for features and price, a contract is signed, and its limitations and failure modes become the buyer's problem, often before anyone has examined them. For high-stakes AI, the purchase decision is one of the most important governance moments there is, because it is the point where risk enters the organization on terms that are still negotiable. A good procurement process treats it that way.

The ProblemProcurement treats AI like ordinary software

Traditional procurement asks whether a product has the features, meets the price, and passes security review. AI systems need more, because their behavior is learned, probabilistic, and liable to change, and their failures fall on the people the buyer serves. Evaluated with an ordinary checklist, an AI system's most important properties, how it behaves on hard cases, where it is biased, how it degrades, what it was trained on, go unexamined. The organization ends up owning risks it never assessed, discovered only after deployment when they are far more expensive to address.

Why It MattersThe buy decision locks in risk

Once a system is purchased and integrated, the leverage to demand evidence, transparency, or changes drops sharply. Before signing, the buyer can require what they need; after, they are dependent on a vendor whose incentives may not align with theirs. This is why the procurement moment matters so much: it is when the organization can still set terms, insist on testing, and walk away. Skipping that scrutiny does not remove the risk; it simply accepts it blindly and permanently. For high-stakes AI, a weak procurement process is a governance failure that everything downstream inherits.

The TeraSystemsAI PerspectiveProcurement as the first governance gate

Our view is that procurement should be the first serious checkpoint in a system's lifecycle, not a rubber stamp before deployment. That means evaluating an AI system on evidence rather than assurances: how it performs on the buyer's own representative and high-stakes cases, what is known about its data and limitations, and how it behaves when conditions change. It means writing the organization's real requirements, around performance, transparency, oversight, and support, into the decision and the contract. Approached this way, procurement stops being a place where risk sneaks in and becomes the place where it is first examined and bounded.

Practical ImplicationsWhat to require before you sign

In practice, a procurement checklist for high-stakes AI asks for evidence of performance on cases like yours, not just published benchmarks; documentation of training data, intended use, and known limitations; and the results of testing you run yourself where possible. It requires transparency about how and when the system changes, since a model can shift beneath you, and commitments around monitoring and support. It defines accountability, who is responsible when the system errs, and it preserves an exit, so dependence does not become a trap. None of this slows a good vendor down; it simply ensures that the risk entering your organization is understood before you agree to carry it.