Knowledge distillation

Also known as: distillation, model distillation

Training a smaller student model to mimic a larger teacher’s outputs or internal signals, preserving task quality while shrinking size and inference cost.

← Back to full glossary · View on index