Framework
How AgriManager runs on top of VeRL — and what it adds.
AgriManager is an agentic environment layer built on top of VeRL. VeRL supplies the RL training engine; AgriManager turns crop simulators into multi-turn text environments and wires them into that engine. A run is two nested loops:
Two loops#
Inner — agentic rollout. Each turn follows Observe → Think → Act. Observe: the
environment renders the simulator's state into a text prompt (observation, action menu, objective).
Think: in think mode the policy writes a short reasoning block. Act: the policy names one
action; the environment parses it, steps the simulator, and returns the next observation and a scalar
reward. After N turns the episode ends as one trajectory. The loop is driven by
AgriToolAgentLoop.
Outer — training. VeRL's trainer collects a group of trajectories, runs an RL algorithm
(algorithm.adv_estimator=grpo by default; VeRL also provides PPO and others), and
updates the policy — which then drives the next rollouts.
system_prompt = env.system_prompt() # task, schema, action menu obs, info = env.reset() # first textual observation while not done: response = policy.generate(prompt) # think format: <tool_call>…</tool_call> <answer>…</answer> obs, reward, done, info = env.step(response) # env parses the answer tag prompt = reassemble(system_prompt, obs)
What AgriManager adds#
VeRL is used as an unmodified submodule — AgriManager plugs in through VeRL's own extension points (an agent loop, an interaction, and a reward function, all declared in config) and never patches it. On top of the engine, AgriManager contributes three things:
- The environment layer. A uniform adapter that turns external crop simulators (WOFOST, Cycles and DSSAT, each through its Gym wrapper) into text agent environments — render observation, parse action, compute reward. See Adding a new environment.
-
The schema
Σ_O · Σ_A · Σ_R. Config decouples what the simulator computes from what the prompt shows, so a single policy can be trained and tested across different observations, action menus, reward objectives, and simulators. See Tasks. - OOD instrumentation. Named, per-axis validation sets logged as live generalization curves during training (held-out weather, action menus, objectives, or simulators). These curves are for monitoring; reported results come from the frozen held-out test suite. See Benchmark.
Where each part of the diagram lives in code:
| In the diagram | What it does | Code |
|---|---|---|
| Environment (render) | simulator state → text prompt, per schema Σ_O·Σ_A·Σ_R |
env/<name>/prompt.py |
| Environment (step) | parse action, step the simulator, score the turn | AgriInteraction · compute_score |
| Agentic rollout | drive the N turns, emit the trajectory | AgriToolAgentLoop |
| VeRL trainer | RL algorithm + policy update (reused) | adapter/trainer/ → VeRL |