What is quantization?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Quantization is a way of representing numbers using a smaller set of possible values. In AI, it often means storing a model’s internal numbers with less numerical precision so that the model needs less memory. A model is the mathematical component of software that has learned patterns from examples and uses them to produce outputs.
An everyday analogy is rounding a measurement. Recording a distance as 3.14 meters takes less detail than recording it as 3.141592 meters. The rounded version may be perfectly adequate for some purposes and inadequate for others. AI quantization makes a related trade-off, although the actual methods are more involved than ordinary decimal rounding.
Computers store numbers using bits, the basic storage units that can each hold a zero or a one. More bits allow a richer range of values or finer distinctions between them. Representing a set of model numbers with eight bits each rather than sixteen can roughly halve the storage for those numbers. Other parts of the application still require memory.
The numbers being compressed often include weights: internal values adjusted during training, the process through which the model learns from examples. Quantization can also affect temporary numbers used while the model calculates an answer. It changes how numbers are represented, not necessarily how many learned values the model contains.
Developers may quantize a model after training or prepare it during training to tolerate the reduced precision. Some approaches preserve greater precision for particularly sensitive calculations. The aim is to reduce resource requirements while retaining enough accuracy for the intended use.
A quantized model may fit on a smaller computer or respond more efficiently. Faster operation is not guaranteed, however. It depends on whether the computer’s hardware and software can perform the lower-precision calculations efficiently. The amount of additional work needed to convert between formats also matters.
A healthcare team considering a smaller model for a local clinical-note summary prototype could compare quantized and higher-precision versions on the same synthetic records. Do they preserve medication dosages, negation, dates and missing information equally well? Measure memory use and response time alongside those errors. Quantization reduces precision in the model’s internal numbers; it is not permission to round or simplify the clinical facts being summarized.
Sources: Hugging Face — Quantization Overview; Dettmers and colleagues — LLM.int8().
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy