Measure What Matters, When It Matters