Guide / agents
Building RL environments from enterprise workflows
An RL environment built from enterprise workflows gives an agent a real task, the state of the systems around it, tools to act with, and a reward tied to an outcome that can be checked. The data that makes this work is records with recorded outcomes, the system state at the time, linked records across tools, and the rules people followed.
Anatomy
Six parts, and where the data for each comes from
| Part | Role | Enterprise data behind it |
|---|---|---|
| Task | What the agent is asked to do | The opening request of a ticket, an issue, an inbound email |
| Initial state | The world before the agent acts | Account and order records, the repository at a commit, the inbox so far |
| Actions | What the agent can call | Tool calls modelled on the systems the organisation used |
| Simulated party | The customer or colleague on the other side | Follow-up turns drawn from how real requesters replied |
| Rules | What the agent may and may not do | Written policies: refund limits, approval steps, coding standards |
| Reward | How success is judged | Recorded outcome, tests that pass, rules respected |
Which workflows make good environments
Not every process is worth simulating. The candidates that pay off share five traits. They happen often, so there are thousands of examples. They take several steps across more than one system. Their outcome was recorded, so success can be judged without a human. The rules people followed were written down somewhere. And the systems involved can be imitated with a small set of tool calls.
Helpdesk work fits almost perfectly: a customer request, account lookups, a policy on what may be granted, and a resolution code at the end. Software maintenance fits too: an issue, a repository at a known commit, and tests that decide whether a change worked. Purchasing, invoicing and order changes by email fit when the outcome was logged in a business system.
From records to tasks
- Select records whose outcome is verifiable and not trivially implied by the request.
- Reconstruct the state at the moment the task began: the account, the order, the code, the earlier messages.
- Write the task prompt from the opening request, de-identified, without leaking the answer.
- Define the tool calls the agent can make and implement them against the reconstructed state.
- Build the simulated requester from the real follow-up turns, so it supplies information only when asked.
- Set the reward: outcome match, rule compliance, and for code, the tests that the real fix made pass.
- Hold out a split of tasks, by organisation or by time, for evaluation only.
Public benchmarks show the two shapes most teams start from: SWE-bench for repository tasks checked by tests, and tau-bench for an agent serving a simulated customer under a policy. Enterprise records add what public sets cannot: private systems, real policies and requests no model has seen.
Pitfalls
- Outcome leakage. Labels, status fields or later messages in the observation give away the answer. Cut the state at the moment the task began.
- Shortcuts the real agent took. A human agent who phoned a colleague left no trace in the records. Drop tasks whose resolution depended on steps the environment cannot offer.
- Reward on the end state only. When policy matters, check the path: a refund granted without the required check is a failure even if the customer was happy.
- Identity across systems. State and dialogue must refer to the same customer. That needs tokens shared across sources, which should be agreed in the spec because it changes the privacy picture. See the de-identification checklist.
What to ask a supplier for
Ask for outcomes with a code book, the state snapshots or the records needed to rebuild state, links between records across tools, the policies and approval rules in force during the period, and enough verifiable tasks after filtering. The support ticket and source code spec sheets list the fields that matter most.
Starting points
Three environment types to start with
Service
Customer case handling
Engineering
Repository maintenance
Operations
Order and invoice changes
FAQ
RL environment questions
What is an RL environment built from enterprise data?
A simulated workplace in which an agent receives a real task, sees the state of the relevant systems, acts through tool calls, and is rewarded when it reaches an outcome that can be checked, such as a resolved case or passing tests.
Which enterprise data makes the best RL tasks?
Records with verifiable outcomes: support tickets with resolution codes, issues closed by a merged change with tests, deals with a recorded result. The outcome is what lets a grader reward the agent.
Why not generate environment tasks synthetically?
Synthetic tasks follow the patterns their author imagined. Real requests carry the long tail: ambiguous wording, missing information, exceptions to policy and tasks that span systems. Many teams combine both.
What should a supplier provide for RL use?
Tasks with outcomes and a code book, the system state around each task, linked records across systems with consistent tokens, the written policies people followed, and a held-out split for evaluation.
Does de-identification get in the way of RL environments?
It can, when the same person or account must be recognised across systems. Linked records need tokens that stay consistent across sources, which raises re-identification risk and should be agreed explicitly.
How do I stop an agent gaming the reward?
Keep the recorded outcome out of the observation, check the path as well as the end state where policies matter, and review a sample of high-reward episodes by hand.
Keep reading
Next step
Building an environment and short of real tasks?
Name the data type, languages, period, volume and intended use. We answer by email with what can be assembled and how it would be documented.