What are acceptance criteria?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Acceptance criteria are observable conditions used to decide whether an agreed requirement has been met. A requirement states what a product must do or a condition it must satisfy. Acceptance criteria make that statement concrete enough to check, rather than leaving completion to someone’s impression that the result looks reasonable.
For a booking tool, a requirement might say that users can cancel a reservation. The criteria could specify that the reservation changes to canceled, the seat becomes available again and the user receives confirmation. They should also say what happens if the reservation has already been canceled. That is part of the behavior, not an optional technical detail.
An acceptance test is a particular check of the criteria using defined inputs and expected results. For example, cancel a known test reservation, reopen it and verify its status. The test needs a reliable way to inspect what actually changed. A generated message saying “canceled successfully” is not enough if the underlying record remains active.
AI-enabled behavior may require criteria that allow more than one acceptable wording. A summary could be judged by whether it preserves the main findings, includes important qualifications and adds no unsupported facts. It would be inappropriate to require every acceptable summary to use exactly the same sentence. The test should match the task.
Criteria should also cover refusal, uncertainty and recovery where relevant. When a source is missing, the expected behavior may be to say that the answer cannot be established from the supplied material. Guessing a plausible answer should not count as successful completion merely because the output is fluent.
Passing these checks is different from demonstrating benefit. A cancellation function can meet its specification while the overall service remains confusing. A summary can satisfy its content requirements without saving the reader time. These broader questions require additional evidence from use.
For a clinic handoff tool, acceptance criteria might require that each unresolved task shows an owner, a source and a current status; that uncertain patient matches are held for review; and that a failed save never appears as a completed update. Fictional cases can test each behavior, including interruptions and duplicate records. Passing those checks establishes the tested behavior. Whether handoffs improve or patients reach care sooner requires separate evidence from the service using the tool.
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy