Explanation-enhanced knowledge distillation (e2KD) is a method to faithfully distill teacher models into students by additionally optimizing the similarity of teacher and student explanations. We show that e2KD consistently improves accuracy and student-teacher agreement, ensures that students learn from teachers to be right for the right reasons, and is robust across architectures, data amounts, and works even with pre-computed explanations.