What is a reasoning model?
The AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
A reasoning model is a model designed or trained to spend additional computation working through a problem before producing its final answer. The label is used mainly for language models, and its exact meaning varies between providers. It describes an approach to producing answers, not a guarantee that the model reasons correctly.
A model may generate intermediate steps, consider alternative approaches or check parts of a proposed solution. This extra work happens during inference: using an already trained model on a particular task. The model’s learned settings do not ordinarily change simply because it spends longer answering.
Some products let users choose a reasoning-effort setting. More effort may improve results on difficult problems, but it can also increase response time and cost. For a simple extraction task, additional reasoning may offer little benefit. The right comparison is performance on the work that actually needs doing.
A displayed explanation should be treated as something to check, not as a complete recording of the process that caused the answer. Research has shown that generated reasoning can omit influences that affected a model’s conclusion. Some products display only a summary of intermediate work. In either case, a long and orderly explanation can contain a mistaken assumption.
Prompting advice is also model-specific. Some reasoning models work well with clear instructions and do not need an added request to “think step by step.” It is usually more useful to specify the task, constraints and evidence required than to insist on a theatrical display of thought.
For a research team comparing several studies, a reasoning model might help reconcile different populations, endpoints and assumptions. The team should still verify the papers and calculations. A second model can offer criticism, but its agreement is not independent confirmation, especially when both use the same sources. In clinical work, model price, response length and apparent sophistication cannot substitute for validation of the intended use. More computation is a resource to evaluate, not a reason to suspend judgment.
Sources: OpenAI — Reasoning Models; OpenAI — Reasoning Best Practices; Turpin and colleagues — Language Models Don’t Always Say What They Think (2023)
To learn more, take a look at the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy