Learning path › Foundations
INTERACTIVE COURSE · 25 MIN R
The Alignment Studio
Teach a policy to Explore the ideas behind modern LLM post-training. Run tiny experiments, inspect the math, and build intuition for what the algorithms actually optimize.
✦ Your sandbox is ready
Progress saves in this browser
FIELD NOTE Think like an optimizer At every step, ask: what signal is the model receiving, and what unintended behavior could maximize it?
These are simplified, deterministic toy simulations—not training runs. They isolate one mechanism at a time.
CHECK YOUR INTUITION A policy ratio is far above the PPO clip range. Does clipping force the new policy back to the reference?
Yes — clipping is a hard constraint on the policy. No — it clips the surrogate objective's contribution.