What is trustworthy AI?
The AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Trustworthy AI is AI that merits confidence for a specified use. The important word is merits. A friendly interface, a recognized brand or a persuasive demonstration can produce trust without providing evidence that the system deserves it.
Trustworthiness has several dimensions. NIST’s AI Risk Management Framework groups these around validity and reliability, safety, security and resilience, accountability and transparency, explainability and interpretability, privacy, and fairness with harmful bias managed. These are qualities to examine, not a certification automatically obtained by using the terminology.
Validity asks whether the system measures or performs what it claims to. Reliability concerns dependable performance under stated conditions. A tool can be reliable at producing the wrong kind of answer, so both matter. Safety concerns harm; security concerns threats and unauthorized interference. Neither can be inferred from accuracy alone.
The qualities can involve trade-offs. More detailed information may help explain an output while creating a privacy concern. A stricter threshold may reduce false alarms while missing more cases. The appropriate choice depends on purpose and consequences, and should be documented rather than hidden behind a single score.
Trustworthiness also changes over time. A previously suitable model may encounter a different population, an altered workflow or a new data format. Continued confidence needs continuing evidence and a way to challenge or stop the system.
For a clinical operations assistant, justified confidence might mean that it reliably identifies a missing document, shows its evidence, respects access limits and records the action taken. Staff should know when its answer is uncertain and how to correct it. Patients need a meaningful route to resolve a mistake. Calling the tool trustworthy adds little unless these properties are demonstrated. The useful question is not whether people like or believe the AI, but whether their reliance on it is appropriate to the task and supported by evidence.
Sources: NIST — Artificial Intelligence Risk Management Framework (AI RMF 1.0, 2023); WHO — Ethics and Governance of Artificial Intelligence for Health (2021)
To learn more, take a look at the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy
Lead with AI in Health Care
The AI Pragmatist is a monthly newsletter from ALA faculty, with new explainers & essays from the Advanced Learning Academy