Skip to main content
The CRM Data Pipeline Environment is an OpenEnv-compliant benchmark that puts autonomous AI agents to work as real data engineers. Your agent receives messy, incomplete CRM records from multiple enterprise sources and must profile, clean, standardize, and merge them into an analytics-ready master dataset — all through a typed REST API.

Quick Start

Spin up the environment and run your first agent loop in minutes.

Action Space

Explore all 8 action types your agent can execute.

API Reference

Full REST endpoint reference with request/response schemas.

Task Catalogue

Three difficulty tiers — Easy, Medium, and Hard — with escalating complexity.

How It Works

The environment runs as a FastAPI server. Your agent interacts with it through three core endpoints — /reset, /step, and /state — and submits its cleaned dataset when ready. A hidden grader engine scores the final output against ground truth.
1

Start the server

Run the environment locally or via Docker. It listens on port 8080.
2

Reset to begin an episode

POST /reset initializes a fresh episode, loads messy CRM data, and returns your first observation with the task objective and available data sources.
3

Execute cleaning actions

Your agent sends POST /step with a CRMPipelineAction payload — view data, profile columns, standardize formats, handle missing values, deduplicate records, or run SQL joins.
4

Submit and get scored

Send SUBMIT_PIPELINE to trigger the grader. It compares your final dataset against hidden ground truth and returns a score from 0.0 to 1.0.

Task Difficulty Levels

Task 1 — Easy

Normalize a single web_forms dataset. Fix dates, lowercase emails, strip spaces.

Task 2 — Medium

Merge web_forms and legacy_db. Handle schema mismatches and deduplicate records.

Task 3 — Hard

3-way merge across Salesforce, Web Leads, and Legacy DB with conflict resolution rules.

Key Capabilities

Architecture & POMDP Model

Understand the environment’s state machine, reward signals, and data flow.

Writing an Agent

Step-by-step guide to building a custom agent that interacts with the pipeline.

Data Engine

Deep-dive into data sources, generation logic, and all standardization strategies.

Grading & Scoring

Understand exactly how your agent’s output is evaluated and scored.