What does temperature mean in AI?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Temperature is a setting that changes how an AI language model chooses among possible next pieces of text. A language model is software trained to recognize language patterns and generate responses. Temperature influences how strongly the system favors likely continuations over less likely ones. It does not measure physical heat.
To understand the setting, start with tokens: the small pieces of text a model processes, which can be words, word fragments or punctuation. When preparing a response, the model assigns different likelihoods to possible next tokens. The software then chooses one and continues. Choosing among these possibilities is called sampling.
A lower temperature makes the differences between those likelihoods more pronounced. The most likely options become more dominant. A higher temperature makes the distribution less concentrated, so less likely options have a greater chance of being chosen. The model is still working from its learned patterns; the setting changes how choices are drawn from them.
Suppose you ask for names for a fictional bookshop. At a lower temperature, repeated responses may favor familiar phrases. At a higher setting, the names may vary more and include unusual combinations. That can be useful for exploring alternatives. It can also produce awkward or irrelevant suggestions. More variation is not automatically more creativity.
Temperature is not an accuracy setting. A model can strongly favor a false answer, in which case reducing the temperature may make it repeat the same error more consistently. Raising the temperature does not cause the system to check additional sources or acquire missing knowledge. Those require other features of the application.
A public health team drafting alternative subject lines for a vaccination-reminder campaign might welcome variation in wording, while the approved clinic dates, eligibility criteria and instructions must not vary at all. Where a model exposes a temperature setting, compare its effect on that particular task rather than treating a low value as a safety lock. Many modern reasoning models manage sampling internally and disable manual temperature adjustments altogether, and even a setting of zero does not always guarantee identical results across runs.
Sources: Hugging Face — Generation; Hugging Face — Generation Strategies.
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy