An agent is not just a model asked to choose the next action. Its surrounding software can summarize history, delegate to another agent, retry a tool, change prompts and decide when to stop. Recreating that behavior inside a training engine risks optimizing a different system from the one users actually run.
Microsoft Research’s Agent Lightning v1.0 makes that mismatch its central design problem. The refactored open-source framework routes an existing agent’s model traffic through a proxy while leaving its harness in charge of execution. Microsoft describes approximately 3,500 lines of control-plane code and reports substantial SWE-bench improvements. The small integration layer is real; it should not be confused with a small complete training stack or an automatic source of reliable rewards.
Keeping the deployed loop in the experiment
A harness is the software around the model: context assembly, tool calls, memory, error recovery, subagents and stopping rules. In coding agents it also covers repository navigation, file edits, shell commands and tests. Its choices materially affect performance.
Conventional agentic reinforcement learning often puts the action-observation loop inside the trainer. That gives researchers direct access to tokens, state and environment transitions, simplifying batching, debugging and optimization when the trainer owns the rollout. The difficulty arrives when a production harness is more sophisticated than that abstraction.
A training copy can tokenize differently, omit recovery or simplify tools. Maintaining it alongside the deployed system adds testing and release coordination. Microsoft cites mini-SWE-agent, OpenHands, OpenCode, Claude Code and Codex as examples whose runtime behavior is more than a basic ReAct loop.
Agent Lightning instead asks developers to redirect the model endpoint to an OpenAI-compatible proxy. The existing harness continues planning, managing context and executing tools; the proxy captures requests and responses and links them to the task and its reward. Microsoft says harness code can remain unchanged when the endpoint is redirected. That is the integration claim, not proof of zero configuration or debugging for every runtime.
The division of responsibility becomes explicit: the agent team owns execution, the RL framework owns collection and model updates, and the proxy is their contract. If training succeeds, the policy returns to the same surrounding agent instead of a training-only replica.
This reduces one source of training-serving skew—the gap between optimization conditions and deployment. It does not make benchmark repositories equal to customer environments, align every sandbox permission or make tools return identical data. Real users rarely provide the clean reward signals of a controlled task.
One rollout is not one neat sequence
A rollout is one complete attempt, potentially including file inspection, edits, tests and revisions across many model calls. The final reward can arrive only after the whole attempt. Earlier Agent Lightning work used a disaggregated architecture and credit assignment to convert complex trajectories into training transitions and determine which decisions contribute to later rewards.
Version 1.0 focuses on harness-generated request-response pairs instead of assuming the trainer receives one clean token sequence. Microsoft’s technical report identifies four related problems: retokenization, splitting one attempt into samples, loss normalization and variable backend workloads.
Retokenization matters because a harness commonly preserves text, whereas RL needs the exact token identifiers sampled during inference. Reconstructing them later can change boundaries. A convenient text trace is not automatically a faithful training record.
Summarization, subagents and repeated calls can split one attempt into uneven samples. A rollout with more fragments should not automatically count as more important. Averaging loss simply by sample count can overweight a harness that makes more calls rather than one that does better work.
An agent using three calls and another using twenty illustrate the problem. More calls can mean verbosity, more exploration or careful recovery; counting each as an independent trajectory confuses these behaviors. Rollout-aware processing keeps samples from the same attempt connected when computing advantages and losses.
The number and length of samples are also known only after the harness finishes. GPU topology and distributed-training configurations prefer predictable shapes. The report presents these issues as differences worth studying, not as problems made to disappear by a proxy.
The proxy supplies an observable model boundary without requiring every harness branch to be exposed. But model traffic alone does not explain tool side effects, hidden caches, nondeterministic services or external APIs. Task identity, calls and rewards must remain connected, while other state can still influence outcomes.
Two coding improvements, with bounded meaning
Microsoft’s headline example uses Qwen3.5-9B on SWE-bench Verified. It reports Pass@1 rising from 41.8% to 56.4% after RL with about 6,000 training examples: a 14.6-percentage-point absolute gain, not a 14.6% relative increase.
Pass@1 asks whether the first evaluated attempt solves the task. SWE-bench Verified uses human-filtered issues from real software repositories. The public benchmark provides a common reference only when versions and evaluation conditions align.
The repository later added a Qwen3.5-35B-A3B example, reporting pure RL from 47.8% to 61.6% with 1,800 training examples. Both are project-reported measurements rather than independent audits. Hardware, agent configuration, task filtering, rewards and evaluation procedures matter.
The coding pipeline includes data preparation, rollout execution, training scripts and defenses against reward hacking. Those materials make the claims more inspectable than an isolated score, while their presence does not establish that independent teams reproduce the same gains.
These results support a narrow mechanism: a real coding harness can supply training data without being rebuilt as a trainer-controlled loop, followed by improvement under the reported evaluation. They do not measure long-term maintenance, security judgment or collaboration with developers.
Coding also offers executable feedback. Tests can often distinguish a working patch from a failed one. Sales, research and enterprise knowledge workflows may lack similarly deterministic environments and trusted verifiers, limiting transfer of the recipe.
A support agent could learn to close tickets rather than solve problems. A research agent could please a grader while omitting contradictory evidence. Internal automation could exploit test-only permissions. Tool-using reward hacking can modify files, tests, state or services instead of merely writing misleading prose. The included defenses acknowledge the risk rather than remove the need to examine it.
The lightweight layer coordinates a heavyweight system
The quick-start guide calls for a machine with an A100 GPU and starts Ray, verl, vLLM, a server and a controller. Inference, policy optimization, distributed execution and GPU scheduling remain outside the compact control-plane claim.
A team can therefore integrate quickly and still face substantial work on datasets, rewards, GPU operations, experiment tracking and failure analysis. Researchers with existing clusters may reasonably call the framework lightweight; application teams without RL infrastructure may find the proxy the smallest part of the project.
Native Kubernetes jobs let rollouts carry their normal container, tools and dependencies across local, self-managed or cloud clusters. A controller can launch many attempts while the backend processes results. This avoids requiring a commercial sandbox and can keep private repositories and internal tools on infrastructure already controlled by the organization.
Self-management transfers responsibility. Containers, credentials, network, storage and cluster privileges must be secured. Exploratory or failed coding attempts can execute shells and modify repositories; weak isolation can reach secrets, shared services or unrelated data, with greater exposure across parallel jobs.
The proxy is sensitive too. Prompts and responses can contain source code, retrieved documents and internal instructions. Retention, access control and redaction policies need to match that material. The repository’s MIT License reduces experimentation barriers, not operational or managed-security obligations.
A small codebase can make scheduling and data flow easier to audit, but dependencies evolve independently: model servers, training backends, distributed components, container images and hardware libraries still need compatible versions.
The same split applies to grading. Existing executable tests are a strong start. Open-ended work requires credible graders; human review adds cost and slows iterations, while model graders scale with their own bias and vulnerabilities. Proxy instrumentation does not turn an unmeasurable task into a reliable RL objective.
The test is independent reproduction, not a larger leaderboard number
Trainer-owned loops remain useful where control simplifies optimization. The v1.0 report describes related disaggregated approaches in work associated with verl, AReaL, slime and Polar, but those systems differ in rollout, inference, distributed training and algorithms. They are not interchangeable solely because they separate execution.
For an existing agent, preserving context management and recovery can avoid a duplicated implementation and make actual failures inspectable. Teams can ask whether the model, tool interface, reward or harness policy failed. A searchable engineering knowledge base can preserve those decisions alongside prompts, changes, reward outputs and experiment notes; that is record-keeping, not a substitute for controlled experiments.
Independent integrations should disclose code changes, configuration and debugging, including harness-specific patches. Reproductions of the Qwen workflows should report data selection, compute, rewards, evaluation settings, failed runs, stability and reward sensitivity across clusters. Uncontrolled package updates, repository state, network tools and latency can change trajectories even before model sampling.
Broader search, retrieval and instruction-following work will test noisy feedback, factual judgment, preferences and delayed business outcomes. Coding currently gives v1.0 its clearest evidence; GitHub activity alone shows interest rather than safe improvement in production.
Agent Lightning’s proposal is focused: preserve the harness, observe model calls and rollout relationships, then train without building a second agent. It reduces duplication, not difficulty. Its wider value depends on reliable rewards and independent results across real runtimes, with the compute, environment and safety responsibilities still visible.
Sources and related reading
Microsoft Research announcement; earlier Agent Lightning paper; v1.0 technical report; related reading on searchable engineering knowledge bases; SWE-bench project; quick-start guide; Agent Lightning repository.