Apple tests GRPO in 11 languages, finds transfer and severe regressions

Apple researchers trained nine language models, at sizes up to 8 billion parameters, with GRPO in 11 different languages, then measured what carried over. The study, published on Apple’s machine learning research site on August 18, finds that reasoning trained in one language often lifts performance in many others. It also finds that some model and language pairs regress sharply on tasks the training never touched. The preprint went up on arXiv on August 13.

GRPO, short for Group Relative Policy Optimization, is the reinforcement learning method that the DeepSeekMath paper introduced in 2024 as a memory-saving variant of PPO. It usually runs inside reinforcement learning with verifiable rewards, or RLVR, a recipe the Tulu 3 paper named, in which the reward comes from automatically checking the model’s answer, such as an exact numeric match. Apple’s paper calls that combination “a central recipe” for improving reasoning, but notes that studies of it remain heavily English-centric.

The training data comes from the Multilingual Reasoning Gym, a companion project from the same team that procedurally generates verifiable reasoning problems in 14 languages. The 11 training languages, drawn from the MGSM benchmark, run from English, Chinese and German through Thai, Swahili, Telugu and Bengali. Italian, Korean and Portuguese stayed held out entirely, so the team could test transfer to languages the training never saw.

Setup	Figure
Base models (Qwen3, Qwen3-Base, Gemma 3, SmolLM3)	9
Training languages	11
Fully held-out languages	3
Tasks used in training	62
Unseen tasks for evaluation	30
RL steps per training language	500

Two results stand out. Rewarding a model for reasoning in the target language, rather than in English, “often leaves only a small gap” compared with English rewards, the paper reports. And the transfer is real: training in Swahili lifted Swahili evaluation scores by 32.2 percentage points on average, while the next-best training language, Telugu, delivered 20.5 points. Lower-resource languages gained the most from their own data, because transfer from other languages covered them least.

Conclusions drawn from a single model or a single training language can be misleading.

GRPO Beyond English, Apple Machine Learning Research

The catch sits in the regressions. Specific combinations of base model and training language showed what the authors call “extreme regression” on particular task types, such as difficult math problems or unseen tasks with English content. Averages hid that damage, which is why the paper keeps returning to one warning:

Some model-language combinations degrade on unseen English-content tasks or difficult math tasks, even when performance in the training language is preserved.

GRPO Beyond English, conclusion

That warning lands on a method much of the field now depends on. GRPO traces back to the DeepSeekMath project, and it keeps showing up in results we cover, most recently when a DeepSeek-first cascade beat GPT-5.6 Sol on a coding benchmark. Evaluation breadth is becoming its own argument too, as the ICML position paper on evaluating human-AI teams made from a different angle.

The thing to watch is whether labs shipping multilingual reasoning models start reporting per-language regression panels. The paper’s closing message points that way: RLVR beyond English can deliver broad crosslingual gains, but it “requires broad evaluation to detect language-specific regressions.” The result is that a single-benchmark scorecard doesn’t surface the failure mode Apple just documented.