Model Distillation (Knowledge Distillation)

A technique where a smaller 'student' model is trained to mimic the behavior of a larger 'teacher' model. This creates faster, cheaper models that retain much of the teacher's performance. Used to create efficient models for edge devices or cost-sensitive applications.