Skip to main content
Every interaction with the CRM Pipeline environment — whether a POST /reset or a POST /step — returns a CRMPipelineObservation Pydantic object. This is the complete and exclusive window through which your agent perceives the world. The environment’s true state (the hidden ground truth DataFrames, the full mutable sources, the step counter) is never directly exposed. Instead, your agent must read this observation carefully after every action and build up a belief about what the data looks like and what still needs to be done. Understanding the semantics of each field — especially which fields update after which actions — is critical for writing an agent that makes informed, sequential decisions rather than random ones.

Full Observation Structure

The CRMPipelineObservation object returned by the server after every step looks like this:
For a Task 3 episode after reset, the same object includes populated conflict_rules:

Field-by-Field Reference

bool
required
Indicates whether the current episode has reached a terminal state. done=True is only set when a SUBMIT_PIPELINE action succeeds (i.e., step_count >= MIN_STEPS_BEFORE_SUBMIT and a valid final_source is provided). Once done=True, the grader has already run server-side and the terminal score has been computed.A done=False in response to SUBMIT_PIPELINE means the submission was rejected due to insufficient steps — the episode continues.
Optional[float]
The step reward earned by the immediately preceding action. This is not a cumulative score — it reflects only the reward signal for the single action just taken. Step rewards range from -0.15 (premature submission penalty) to +0.20 (terminal schema-match reward on SUBMIT_PIPELINE).The value is 0.0 on the initial observation returned by /reset (no action has been taken yet). See the Reward System page for the complete reward schedule.
str
required
A natural-language description of the current task’s goal. This field is set once at reset() time and never changes during an episode. It is derived from the task_id:
  • "t1" → "Normalize web_forms dataset"
  • "t2" → "Deduplicate legacy_db dataset"
  • "t3" → "Merge Salesforce, Web Leads, Legacy databases"
Your agent should parse this field at the start of each episode to determine what the top-level objective is.
Dict[str, str]
required
A dictionary mapping each expected output column name to a human-readable description of its required format. This is the contract your agent must satisfy. The grader compares your final DataFrame’s values against the hidden truth using these formats as the standard.For Task 1 and Task 2:
For Task 3 (3-way merge — fewer columns, since only the intersection matters):
schema_target is also used to compute the SUBMIT_PIPELINE step reward: 0.2 × (matching columns / max(1, total target columns)).
List[str]
required
A list of all DataFrame names currently accessible to the agent in the environment’s sources store. This list changes dynamically as the episode progresses:
  • At reset(): contains only the original source names (e.g., ["web_forms"] for t1, ["salesforce", "web_leads", "legacy_db"] for t3)
  • After MERGE_SOURCES: "merged_output" is added
  • After EXECUTE_SQL: the output_table name is added
  • After EXECUTE_SQL overwrites an existing name: the list stays the same length, but the content of that DataFrame has changed
Always check available_sources before referencing a DataFrame name in a subsequent action. Referencing a source not in this list will trigger a ValueError and return a -0.05 penalty.
str
required
A markdown-formatted table showing the first 3 rows of the most recently viewed source, produced by df.head(3).to_markdown(). This field is only updated by the VIEW_SOURCE action. All other actions leave current_view unchanged from its previous value.The initial value after reset() is "Select a source to view.". After a VIEW_SOURCE call:
Because only 3 rows are shown, current_view is best used for identifying column names and example data patterns. Use data_quality_report for statistical coverage across all rows.
Optional[str]
A per-column quality report string generated by the most recent PROFILE_SOURCE action. The report lists each column’s Pandas dtype and null count in the format:
Like current_view, this field persists across steps — it reflects the last profile taken and is not reset between steps. If you profile web_forms, then perform several STANDARDIZE_COLUMN operations, the data_quality_report still shows the pre-standardization null counts until you call PROFILE_SOURCE again.The initial value after reset() is "" (empty string). The field becomes null in the JSON if no profiling has occurred and the internal report is None.
str
required
A human-readable string describing the outcome of the most recent action. This is the primary signal for detecting success vs. failure on each step. Feedback messages follow predictable patterns:Success patterns:
  • "Viewing top 3 rows of <source>"
  • "Profiled <source>"
  • "Standardized <source>.<column> using <strategy>"
  • "Handled <N> missing values in <source>.<column>"
  • "Deduplicated <source>, removed <N> rows."
  • "Merged <source> and <source2> into 'merged_output' using <key>."
  • "Executed SQL successfully. Wrote <N> rows to '<output_table>'."
  • "Pipeline submitted with final source: <name>."
Error/penalty patterns:
  • "Error: Source '<name>' not found. Available: [...]" — invalid source name
  • "Error: Column '<name>' not found. Available: [...]" — invalid column name
  • "Merge failed: key '<key>' missing from one of the sources." — bad join_key
  • "SQL Error: <sqlite error message>" — malformed query
  • "Blocked Action: Only SELECT queries are allowed for security." — non-SELECT query
  • "Blocked: Query contains forbidden keyword." — DDL/DML attempted
  • "Early submission blocked. You must perform at least 3 cleaning steps..." — premature submit
  • "Error: <exception message>" — any unhandled exception in a handler
Your agent should parse last_action_feedback after every step to detect whether the action succeeded. Feedback strings beginning with "Error:", "Merge failed:", "SQL Error:", or "Blocked" indicate a failed action with a negative reward.
Optional[Dict[str, str]]
default:"null"
A per-column dictionary specifying which source to prefer when conflicting values exist during a merge. This field is only populated for Task 3 — it is null for Tasks 1 and 2.
These rules reflect how the hidden truth was constructed. The grader expects your final output to honor these rules exactly:
  • email must come from Salesforce (it is the canonical, lowercase email)
  • phone must come from Web Leads when a Web Leads phone is available, otherwise fall back to Salesforce
  • customer_id must use the Salesforce format (CUST_XXXX, not OLD-XXXX)
When this field is non-null, your agent must incorporate these priorities into its EXECUTE_SQL merge logic (e.g., using COALESCE(wl.phone, sf.phone) to implement prefer_web_leads with Salesforce fallback).

How Fields Update After Each Action

This table shows exactly which observation fields change after each action type, allowing your agent to know what to read after each step:
data_quality_report persists across steps and only updates when PROFILE_SOURCE is called. If you standardize a column and want to verify the transformation improved null coverage, you must call PROFILE_SOURCE again explicitly.

Using last_action_feedback to Detect Errors

The last_action_feedback field is your agent’s most actionable signal for error detection. A robust agent should parse this field after every step and adapt its strategy accordingly. Here is a pattern for programmatic feedback parsing:

Understanding done=True and Terminal State

When done=True appears in an observation, the episode has ended. The grader has already run server-side. There are no further step() calls that can change the outcome. The terminal reward embedded in the reward field of this final observation is the schema-match heuristic (0.2 × ratio) — a proxy for final quality, not the true grader score. The true grader score (the row-level accuracy against hidden truth) is computed internally and used to generate the benchmark result for the episode. To understand how that score is calculated, see the Reward System page.