RL or SFT? New Research Teases Apart What Actually Drives Reasoning in LLMs

The debate over how best to teach large language models to reason — whether through reinforcement learning or plain supervised fine-tuning — is one of the most consequential questions in AI today. A new preprint on arXiv wades directly into that fight, attempting to isolate the contributions of each technique in a controlled setting.

What the Research Does

Rather than treating RL and SFT as black boxes, the authors break down the training pipeline into discrete stages, measuring how each method affects model performance on reasoning benchmarks. The headline result: while supervised fine-tuning gets models most of the way there on standard tasks, reinforcement learning appears to provide a measurable boost in more complex, multi-step reasoning — though at a cost in training stability and compute.

The study is careful to separate the effects of data quality, reward design, and training dynamics, which are often conflated in earlier work. By varying these factors independently, the authors pinpoint where RL's advantage actually emerges — and where it doesn't.

Why It Matters for Practitioners

For developers building with LLMs, this research has immediate practical implications. If you're training a model to generate code or solve logic puzzles, the choice between RL and SFT isn't just academic — it affects everything from dataset size and reward design to wall-clock training time.

The findings suggest a hybrid approach may be optimal: SFT for broad competence and RL as a targeted fine-tuning step for specific, high-value reasoning behaviors. That's a workflow that tooling vendors can already support, and it's likely what we'll see more of in production pipelines.

Open Questions

Several questions remain open. Does RL's advantage hold for smaller models? How sensitive is the result to the choice of reward function? And can RL methods be made more sample-efficient so they don't require the massive compute budgets that only the largest labs can afford?

These are exactly the questions that will shape the next generation of AI devtools — the kind that promise to make reasoning models cheaper and more accessible.