Post-Training

On-Policy Distillation: Closing the Gap Between RL and SFT

• whackur
• Ai
• July 10, 2026

The two standard post-training methods each leave a gap. Supervised fine-tuning (SFT) has the student imitate sequences a teacher already produced, but training happens on states the student may never actually visit, so errors compound over long generations. Reinforcement learning (RL) samples from the student’s own rollouts, which fixes that mismatch, but the reward is usually a single bit or two per episode. A post Thinking Machines Lab published in October 2025 proposes combining the two: sample trajectories from the student, then have a strong teacher score every token in that trajectory.