A lot of ink has been spilled on the question of what will ultimately win: the scientific method, an approach to learning about the world by coming up with theories and testing those theories against ...

MIT introduces Self-Distillation Fine-Tuning to reduce catastrophic forgetting; it uses student-teacher demonstrations and needs 2.5x compute.