Distillation

Why did Anthropic publish its report on alleged Kimi-to-Claude routing?

Why did Anthropic publish its report on alleged Kimi-to-Claude routing?

If sensitive information was reaching Claude, why would Anthropic reveal the route? A model provider may gain more from limiting unauthorized use and alerting customers to a data-handling problem than from continuing to observe the inputs. Answering a competitor’s users consumes compute and may produce training material for that competitor. Publication also reaches users and enterprise buyers whom account bans alone cannot inform. These incentives explain why disclosure can make sense; they do not establish how Anthropic made its internal decision.

Read More
On-Policy Distillation: Closing the Gap Between RL and SFT

On-Policy Distillation: Closing the Gap Between RL and SFT

The two standard post-training methods each leave a gap. Supervised fine-tuning (SFT) has the student imitate sequences a teacher already produced, but training happens on states the student may never actually visit, so errors compound over long generations. Reinforcement learning (RL) samples from the student’s own rollouts, which fixes that mismatch, but the reward is usually a single bit or two per episode. A post Thinking Machines Lab published in October 2025 proposes combining the two: sample trajectories from the student, then have a strong teacher score every token in that trajectory.

Read More
Categories
Tags