AWS Shows Multi-Turn RL Training on SageMaker AI
- Amazon SageMaker AI’s multi-turn reinforcement learning service runs prompts through a hosted agent and trains a policy model from rewards assigned to completed episodes.
- In the Wordle example, each training row supplies a secret word rather than a target sequence of actions; the agent environment uses the answer without exposing it to the policy model.
- The walkthrough uses GRPO to update a LoRA adapter for GPT-OSS-20B, with the reward reflecting how the game ends and, to a smaller extent, play quality.
- On held-out words, the trained model’s solve rate at medium reasoning effort and temperature 1.0 was 84%, compared with 61% for the base model; training tokens cost about $339.
- The job’s validation score at temperature 0 did not represent performance at the chosen serving settings, so the authors recommend evaluating under the conditions planned for deployment.
Our take
Compared with supervised fine-tuning, MTRL avoids the need to label ideal action sequences, but it makes reward design and a dependable task environment central engineering work. The example’s first reward scheme ranked some losses above solves, illustrating why teams should test scoring against real trajectories before a full run. Budget for AgentCore runtime use separately from training tokens, and do not redeploy the agent while a training job is active.
Sources
- Multi-Turn Reinforcement Learning on Amazon SageMaker AI: A hands-on guide · AWS Machine Learning
The summary and analysis were generated by AI. The original sources are authoritative. AI content policy