What is exploratory data analysis?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Exploratory data analysis, usually shortened to EDA, means examining a collection of data to understand what it contains before drawing strong conclusions or building a model. A dataset is the collection being examined. EDA uses summaries, charts and direct checks to reveal patterns, gaps and possible mistakes.
Suppose a spreadsheet contains delivery records. Before predicting delays, you would want to know what each row represents, which dates are recorded and how a delay is defined. A row might represent an order, a parcel or an attempted delivery. Confusing those units can make every later calculation misleading.
Start with simple questions. How many records are there? Which values are missing? Are identifiers repeated? Do dates occur in a sensible order? A missing value is not automatically equivalent to zero. It might mean not recorded, not applicable or unknown, and those possibilities can affect interpretation.
Even an ordinary column such as age can contain impossible values. A value outside any plausible range may reflect a typing error, an unusual coding convention or a mismatch between units. Automatically deleting it may remove evidence of a wider problem. Domain knowledge—understanding the subject represented by the data—helps interpret what the number means.
Charts can reveal patterns hidden by an average. A histogram shows how values are distributed across ranges. A time plot shows how a measure changes over successive dates. Neither chart explains a pattern on its own, but both can suggest questions that a single summary misses.
Exploration differs from confirmation. Finding an apparent relationship while examining many possibilities is a reason to investigate, not automatic proof of a general rule. Where a consequential claim depends on the pattern, it may need separate data and an appropriate analysis designed to test that claim.
AI can help write analysis code or suggest checks. Code is the set of instructions executed by a computer. The results still need verification against the actual records, particularly when an assistant may misunderstand a column or invent an explanation.
For psychiatric records, first establish whether a row represents a person, an assessment or a visit, and whether repeated encounters belong to the same patient. Examine missing assessments, inconsistent dates and how care settings were recorded before interpreting any model score. The impossible-age example from the course was an accessible warning about this larger issue: apparently tidy columns can conceal misunderstood information. EDA helps establish what the data can answer, not simply which charts can be produced.
Source: NIST/SEMATECH — What Is EDA?.
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy