What is algorithmic bias?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Algorithmic bias refers here to systematic patterns in a system’s data, behavior or use that can disadvantage some people or groups. An algorithm is a procedure for carrying out a computation. Bias can arise in the information used to build the system, the objective it is designed to pursue or the way its outputs are applied.
Consider a recruitment tool trained on earlier hiring decisions. If those decisions favored candidates from a narrow set of schools, the system may learn that pattern even when the stated aim is to identify ability. A model—the learned component making predictions—can reproduce a historical preference without anyone explicitly programming a rule to favor those schools.
This is not only a question of including enough examples from each group. The target itself may be poorly chosen. A target is the outcome the model is taught to predict. Past hiring decisions, successful job performance and a manager’s rating are different targets. None should silently substitute for another merely because it is easier to measure.
A proxy is an indirect measure used in place of something else. Postcode might act as a proxy for circumstances the team did not intend to use. Removing an explicit personal characteristic does not necessarily remove related information elsewhere in the data. That is why fairness cannot be established simply by deleting one column.
The surrounding process matters as well. A reasonably accurate tool may be available only to people who use a particular language or device. A score may be treated as a decision rather than one piece of evidence. Those choices can create unequal consequences even when the model’s immediate calculation is technically correct.
Feedback can reinforce the pattern. If a recommendation determines who receives an opportunity, later records may contain outcomes mainly for those already selected. Training another model on those records can reproduce the earlier restriction. Whether this feedback loop occurs, and how strongly, depends on the setting.
There is also a separate statistical meaning of bias: a systematic tendency for an estimate to differ from the quantity being estimated. This entry concerns disadvantage and unequal outcomes. Stating the intended meaning prevents these uses from being confused.
The course’s recursive care-law discussion connected representation in data with access to benefit. As an illustration, a follow-up system built mainly from completed visits could underrepresent people who struggled to reach the service in the first place. Directing additional support only toward those most likely to attend could then reinforce that pattern. Examine whose needs are missing, which groups receive help and whether the intervention closes or widens the gap. Better prediction of existing attendance is not automatically a fairer way to organize access.
Source: NIST — Towards a Standard for Identifying and Managing Bias in Artificial Intelligence.
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy