Dictionary term
Distillation
Also called knowledge distillation · model distillation
Distillation trains a smaller 'student' model to reproduce the outputs of a larger 'teacher' model, transferring much of the teacher's ability into a model that is cheaper and faster to run.
In plain English
Distillation is teaching a small AI to copy a big AI, so you get much of the quality at a fraction of the cost.
Analogy
Like a senior expert training an apprentice on hundreds of worked cases until the apprentice can handle most of them alone.
Why it matters
Many small, fast models are built this way, and teams use it to turn an expensive model's outputs on one narrow task into a cheaper model for high-volume use. Check the teacher model's terms of service first, because some providers restrict using their outputs to train other models.
In practice
Meta describes its open-weight Muse Glimmer model as distilled from its larger Muse Spark model and built to run on consumer hardware.
Related terms
Sources
Browse all terms →Last verified