Hugging Face lifts a 350M model from 22.6% to 29.7% in 100 GRPO steps
Hugging Face published a recipe on 3 September 2026 that takes a 350M parameter model from 22.6% to 29.7% on a structured-output benchmark, using about 500 training samples and 100 reinforcement learning steps. The guide fine-tunes Liquid AI’s LFM2.5-350M with Group Relative Policy Optimization, or GRPO, through the TRL library, then scores it on the IFStruct benchmark.
Authors Leonie Monigatti, Ben Burtenshaw and Sergio Paniego say the whole run is small enough for a free-tier Colab or Kaggle GPU. That matters because the usual counter-argument to task-specific tuning is cost, and the training here is sized for a 16 GB card. The evaluation ran separately on a MacBook Pro, served through llama.cpp. That machine had an Apple M5 Max and 36 GB of unified memory.
The mechanism is a LoRA adapter rather than a full weight update. It targets the LFM-specific module names, because LFM2.5 uses a hybrid attention and convolution architecture. It trains roughly 6M parameters, about 1.66% of the model.
Three reward functions score every completion. They ask whether the output parses in the requested form, carries the expected number of top-level fields, and validates against the row’s JSON Schema. Those three are combined as a weighted sum, with schema validation weighted at 2.0 against 1.0 for format and 0.5 for field count.
The training data is Nvidia’s Nemotron-RL structured-outputs set, which pairs each prompt with a target JSON Schema and an expected field count. That distribution doesn’t match the evaluation. So 40% of prompts had a fenced-code-block instruction appended, and a disjoint 20% were converted into top-level-array tasks. The result shows up in exactly those slices.
| IFStruct group | Base | GRPO-tuned | Change |
|---|---|---|---|
| Overall | 22.6% | 29.7% | +7.1 |
| JSON | 18.0% | 31.9% | +13.9 |
| YAML | 27.2% | 27.5% | +0.3 |
| Wrapper key | 28.5% | 29.7% | +1.2 |
| Bare list | 16.6% | 29.7% | +13.1 |
YAML barely moved, which is the honest reading of the table. The gains sit in JSON output and in bare-list structure, the two things the prompt augmentation was built to teach. A score that rises where you aimed and stays flat elsewhere beats one that drifts up everywhere. It tells you the reward signal did the work, not the sampling.
IFStruct itself comes from Liquid AI, which released it on 30 June 2026 as an open benchmark for output validity and schema following. Scoring is binary, so a sample passes only if every requested structural constraint is satisfied.
The evaluation harness runs six checks. They cover parsing, code fencing, commentary, top-level shape, item count and the schema itself. Liquid AI reports that LFM2.5-350M passes 21.10% of the test set out of the box and 44.90% after its own RL training. That beats Qwen3.5-4B at 36.25% and granite-4.0-h-tiny at 38.75%.
A cheap, task-specific reward signal can make a small model substantially more reliable about form.
Hugging Face, “Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps”
Three caveats sit inside the post, and they’re worth keeping. The authors state plainly that this pipeline isn’t the one Liquid AI used to train its RL model, so 29.7% and 44.90% are not the same experiment. The 22.6% baseline is their own local BF16 measurement rather than the 21.1% Liquid AI reports, and they use the local figure so both runs share a serving stack. The tuned score also still trails Qwen3.5-2B, which the Hugging Face post and the IFStruct table both put at 33.15%.
The wider point is about which kind of fine-tuning you reach for. A separate paper, Kaplan et al. at CoLM 2026, finds that supervised fine-tuning on new factual information can increase hallucinations. The damage lands on knowledge the model acquired in pre-training. Teaching form is a different job from teaching facts, and a reward that only scores structure never asks the model to store anything new.
What’s worth watching is whether the recipe holds on other small models and other schema benchmarks. One run on one 350M checkpoint isn’t a general finding.
Liquid AI notes that IFStruct scores structure and not content, so anyone optimising against it needs a separate quality signal in the reward. That caveat is the reason a 29.7% pass rate is a claim about form, not about whether the output is any good. If you’re weighing this against a full tuning run, our guide on when fine-tuning earns its keep covers the trade.
Get the daily rundown
One email each weekday with the AI news that matters, every claim linked to its primary source.
Free, one email each weekday, unsubscribe in one click. We never sell or share your address.
