What are context management and compaction?
The Harvard Chan AI Glossary
In our classes, we often pause to explain the ideas behind AI and the language used to describe it. These posts grew out of those conversations. They’re for anyone who would like to understand AI better, with no technical background assumed.
Context management means selecting and organizing the information an AI model receives for its current task. Compaction is one technique within that work: shortening accumulated material, often by summarizing it, so that the process can continue without carrying every earlier detail into each new step.
A model is the trained component that interprets information and generates responses. It has a context window, meaning a limit on the amount of information it can work with at one time. Long conversations, documents and results from connected tools can fill that window. The application must then decide what to preserve, shorten or leave out.
Imagine planning a website over several conversations. Early messages contain discarded ideas, while later messages settle the audience, required features and budget. A compact summary should preserve the agreed decisions and unresolved questions, rather than giving every suggestion equal weight. Otherwise, an abandoned idea can reappear as though it were a requirement.
Before continuing a long task, save agreed decisions in a specification. A specification is a written description of what should be built and how it should behave. This creates a more dependable record than hoping a long conversation will always be supplied in full. The record itself still needs to be kept current.
Compaction is not lossless memory. A summary may omit an exception, change a qualification or blur the difference between a proposal and an approval. For example, “consider emailing customers” must not become “email customers.” Original records should remain available when the details matter.
Context management can also involve retrieving only relevant passages, excluding repeated material and separating current task progress from long-term reference information. These techniques aim to make the right information available, not merely to minimize text length.
Suppose a clinical shift handoff is compacted from several progress notes into one brief. “Consider holding the anticoagulant if platelets fall further” must never be compacted into “anticoagulant held.” The summary needs to preserve conditions, timestamps, ownership and what remains unresolved, while keeping original records linked for verification. Saving tokens is not an improvement if compaction turns a conditional contingency plan into a completed order.
If you would like to find out more about AI and improve your leadership skills, consider the AI in Health Care Certificate of Specialization from the Advanced Learning Academy at Harvard T.H. Chan School of Public Health. You can take an individual course or work toward the full certificate.
AI is a fast-moving field. This explanation reflects our class discussions and related material at the time of writing. Terminology, capabilities and tools may change.
For suggested additions or corrections, please email me at tpanch@hsph.harvard.edu.
Dr. Trishan Panch
Program Director, AI in Health Care
Harvard T.H. Chan School of Public Health — Advanced Learning Academy