ServiceNow’s AutoSynthData Turns Agent Failures Into Training Tasks

ServiceNow CoreAI built AutoSynthData to convert enterprise-agent capability gaps into training data by using a target model’s failures and a stronger teacher’s successes to determine what the model should learn next. [1] The system generates and validates new tasks that exercise those capabilities, then shifts its curriculum toward the difficulties that remain as the model improves. [1]

AutoSynthData represents each task as a system specification, a user prompt, and a verifier within an environment defined by observable and modifiable state, available tools and APIs, and action-driven state transitions. [1] The system specification includes constraints such as system instructions, environment policies, and optional task-specific initialization, and it must remain compatible with the environment’s tools, state, and supported actions. [1] ServiceNow CoreAI illustrates the pipeline with EnterpriseOps Gym and uses that project’s released dataset. [1]

AutoSynthData treats enterprise-agent adaptation as a moving curriculum rather than a static task-generation exercise. Its central comparison is between a target model’s failures and a stronger teacher’s successes, which focuses training on capabilities the target has not yet mastered. The verifier and environment specification are consequently as important as generation because they define success and constrain the available actions. EnterpriseOps Gym provides the stated demonstration setting, but no competing task-generation pipeline is evaluated. What the sources don't address: how much AutoSynthData improves target-model performance, what it costs to operate, or whether generated tasks transfer reliably to other enterprise environments.