What is a pilot?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
A pilot is a limited trial of a product or process in its intended setting. It has defined participants, responsibilities and questions to examine. It is meant to produce evidence for a further decision, rather than to become routine operation simply because a small group has started using it.
Suppose a team wants to introduce an AI assistant for preparing internal reports. A pilot might involve one team, a restricted set of documents and review of every report before circulation. The purpose could be to understand whether the assistant reduces preparation work without increasing correction time or introducing unsupported claims.
The boundaries should be explicit. Which tasks are included? Who can use the system? What information may be supplied? Who responds when it fails? When does the trial end, and what evidence will determine whether it expands? Without those decisions, “pilot” can become a label for unstructured use.
A pilot differs from an earlier prototype test. A prototype is a working draft that may simulate parts of the final system. A pilot examines a more concrete arrangement within a real or closely representative setting. That exposes issues such as handoffs, workload and support that may not appear in a demonstration.
It also differs from broad production use, where people depend on a service for ongoing work. A pilot may involve real users and real consequences, so its limited scale does not remove the need for safeguards. The level of oversight should reflect what the trial can affect.
Evidence from a pilot has limits. A highly supported early team may succeed under conditions that cannot be maintained elsewhere. An improvement after introduction may also reflect changes in staffing or workload. The findings should distinguish feasibility—whether the arrangement can operate—from stronger claims about effectiveness and wider applicability.
A proposed pilot for a referral-support system could begin with one intake team, a defined group of cases and read-only briefs that do not change the existing patient-facing process. Staff could compare the briefs with the original records and record omissions, false alerts and time spent correcting them. That would inform a decision about a more active role; it would not yet demonstrate improved patient outcomes. A pilot is useful when its boundaries and evidence make the next decision clearer, including a decision not to expand.
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy