What does validation mean in product development?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Here, validation means asking whether the proposed product is suitable for its intended use. That requires realistic tasks, representative users, documented assumptions and evidence from a sufficiently stable version. This is the product-development meaning of the term; it is broader than checking a model.
A prototype can show that an idea is understandable. Acceptance tests can show that specified behaviors occur. Validation asks a broader question: does the resulting product serve its intended purpose under conditions that matter to its users? The answer can depend on accuracy, usability, burden and how the product fits existing work.
Imagine an assistant that drafts replies to routine inquiries. Tests may show that it produces correctly formatted drafts. Validation would also examine whether staff can recognize its mistakes, whether important cases are overlooked and whether checking the drafts consumes more effort than writing them directly.
The version needs enough stability for observations to be interpretable. If the instructions, model and interface change constantly during the evaluation, it becomes difficult to say what was actually tested. A model is the trained component generating responses. Its version and configuration are part of the product arrangement being examined.
Representative users are people whose needs and circumstances meaningfully reflect those of the intended audience. A successful test with the development team alone may miss unfamiliar vocabulary or assumptions about access and experience. Realistic tasks should include important exceptions, not just the easiest examples.
The word also has other meanings. In machine learning, a validation dataset is a collection of examples used during development to compare or adjust candidate models. That does not, by itself, validate a whole product for use. Ask what kind of validation is being claimed.
For the psychiatric care-pathway idea, a promising retrospective prediction would be only one part of validation. The team would also need to examine whether the output reaches clinicians at the relevant moment, represents available care options and helps rather than distorts their decisions. A model that reproduces past admission choices has answered a narrower question than whether people should use the product. Changes in setting or workflow can reopen that question. Validation supports a defined use under defined conditions, not a permanent approval of “the AI.”
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy