> ## Documentation Index
> Fetch the complete documentation index at: https://ritika-fa61c43f.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Task Catalogue: Three Difficulty Tiers of CRM Cleaning

> Detailed breakdown of all three benchmark tasks — Easy, Medium, and Hard — with data schemas, objectives, and scoring criteria.

The CRM Data Pipeline benchmark exposes three tasks of escalating complexity, each spawned fresh on every `/reset` call using Faker with a randomised seed. No two episodes share the same data — your agent must generalise a cleaning strategy, not memorise a specific dataset. The difficulty progression mirrors real enterprise data engineering escalation: a single messy source, then two sources with conflicting records, then a three-way merge with divergent schemas and per-column conflict resolution rules.

> [!WARNING]
>
> Calling `SUBMIT_PIPELINE` before completing at least **3 action steps** in any task incurs a hard penalty of **-0.15** and the episode does **not** end. The minimum-steps gate (`MIN_STEPS_BEFORE_SUBMIT = 3`) is enforced in `server/environment.py` regardless of how clean your data already is. Always run at least one `PROFILE_SOURCE` and one `STANDARDIZE_COLUMN` before submitting.

## Task Comparison at a Glance

| Property | Task 1 — Easy | Task 2 — Medium | Task 3 — Hard |
| - | - | - | - |
| **Task ID** | `t1` | `t2` | `t3` |
| **Data Sources** | `web_forms` | `web_forms`, `legacy_db` | `salesforce`, `web_leads`, `legacy_db` |
| **Real Rows (approx.)** | \~40 base + fuzzy dups | \~40 + \~20 base | \~60 across all three |
| **Bot Rows** | 8 (web\_forms) | 8 (web\_forms) + 5 (legacy\_db) = 13 total | 5 per source (15 total) |
| **Schema Divergence** | None — single source | Moderate — same columns, different data quality | Severe — different column names, uppercase emails, old ID format |
| **Conflict Rules** | None | None | `prefer_salesforce` for email/customer\_id, `prefer_web_leads` for phone |
| **Ground Truth Key** | `web_forms` | `merged_output` | `merged_output` |
| **Grader Join Key** | `customer_id` | `customer_id` | `customer_id` |
| **Target Schema Columns** | 5 | 5 | 3 |
| **Primary Challenge** | Date/phone/email normalisation + bot removal | Cross-source deduplication | 3-way JOIN + conflict resolution + bot filtering |

> [tab] Task 1 — Easy
>
> ## Task 1: Web Forms Normalization
>
> Task 1 confronts your agent with a single source — `web_forms` — that has been heavily corrupted by the `correlate_truth()` function in `server/data_generator.py`. The source starts with 40 real customer rows whose ground-truth values are clean, ISO-formatted, E.164-formatted, and lowercase. Before your agent sees them, `correlate_truth()` runs every row through a randomised corruption pipeline and appends 8 bot rows. Your agent must undo all of that corruption.
>
> ### Input Data Characteristics
>
> The `web_forms` DataFrame your agent receives contains approximately **40 real rows** (some will have fuzzy duplicates appended, making the raw row count higher) plus **8 synthetic bot rows** injected at the end and then shuffled. The following corruptions are applied stochastically per row:
>
> **Missing Data (20% probability per field)**
>
> ```python theme={null}
> if random.random() > 0.8: dirty["email"] = None
> if random.random() > 0.8: dirty["phone"] = None
> ```
>
> Approximately 20% of rows will have `email` set to `None` and independently \~20% will have `phone` set to `None`.
>
> **String Chaos (50% probability per field)**
>
> ```python theme={null}
> dirty["name"]  = f"  {dirty['name']}  " or dirty["name"].upper()
> dirty["email"] = dirty["email"].upper()
> ```
>
> Names gain leading and trailing whitespace, or are uppercased entirely. Emails are uppercased in roughly half of all rows.
>
> **Date Chaos (tiered random)**
>
> | Probability Range | Corruption Applied |
> | - | - |
> | `> 0.85` | `"??/??/????"` — completely invalid placeholder |
> | `> 0.70` | `"2023-13-12"` — invalid month 13 |
> | `> 0.55` | `None` — date dropped entirely |
> | `≤ 0.55` | `"YYYY/MM/DD"` — valid date but `/`-separated, not ISO 8601 |
>
> **Phone Chaos (tiered random, only when phone is non-null)**
>
> | Probability Range | Corruption Applied |
> | - | - |
> | `> 0.85` | `"12345678901 ext. 42"` — extension suffix appended |
> | `> 0.70` | `"(123) 456-78901"` — US parenthetical format |
> | `> 0.55` | `"Invalid Number"` — literal string |
> | `≤ 0.55` | Raw digit string without `+` prefix |
>
> **Fuzzy Duplicate Injection (25% probability per row)**
>
> After corrupting a row, `correlate_truth()` rolls a 25% chance to append a near-duplicate: the last name is kept, the first name is replaced with a random Faker first name, and the `email` field is set to `None`. These fuzzy duplicates test your agent's `DEDUPLICATE` action.
>
> ### Bot Row Schema
>
> ```python theme={null}
> {
>     "customer_id": "BOT{5-digit-number}",   # e.g. BOT42381
>     "name":        "Support Team",           # or Newsletter Bot, Promo Sender, Deals Alert
>     "email":       "user42@promo-alerts.net",
>     "signup_date": "1970-01-01",             # epoch — suspicious sentinel
>     "phone":       "+00000000000"            # all-zeros — clear anomaly
> }
> ```
>
> Bot domains are: `promo-alerts.net`, `click-rewards.xyz`, `freewinners.info`, `bulk-mail.co`.
>
> ### Ground Truth Target Schema
>
> | Column | Type | Format Requirement |
> | - | - | - |
> | `customer_id` | string | Original `CUST_{n}` identifier, unchanged |
> | `name` | string | Leading/trailing whitespace stripped; any casing is accepted |
> | `email` | string | **Lowercase**, leading/trailing whitespace stripped |
> | `signup_date` | string | **ISO 8601** (`YYYY-MM-DDTHH:MM:SS`) or empty string `""` |
> | `phone` | string | **E.164 format** (`+{country_code}{number}`) or empty string `""` |
>
> The grader compares each column by lowercasing and stripping both the truth value and your submitted value before checking equality, so case differences in `name` will not fail rows. However, `signup_date` and `phone` must match the exact format — `2023/01/15` will not match `2023-01-15T00:00:00`.
>
> ### Scoring
>
> Your final score is computed by `evaluate_dataframes(truth_df, agent_df)` in `server/graders.py`:
>
> ```python theme={null}
> # Left-join truth to agent on customer_id
> merged = pd.merge(truth_df, agent_df.drop_duplicates(subset=["customer_id"]),
>                   on="customer_id", suffixes=("_truth", "_agent"), how="left")
>
> # Per-column exact match (case-insensitive, strip-normalised)
> for col in truth_df.columns:
>     if col == "customer_id": continue
>     s_truth = merged[f"{col}_truth"].fillna("").astype(str).str.strip().str.lower()
>     s_agent = merged[f"{col}_agent"].fillna("").astype(str).str.strip().str.lower()
>     merged[f"{col}_match"] = (s_truth == s_agent)
>
> # Row is correct only if ALL columns match
> score = merged["row_correct"].sum() / len(truth_df)
>
> # Bot/duplicate penalty
> penalty = max(0, len(agent_df) - len(truth_df)) * 0.15
> final_score = min(0.99, max(0.01, score - penalty))
> ```
>
> Leaving even one bot row in your submission costs you `0.15` of score. Failing to deduplicate fuzzy rows means extra rows are penalised.
>
> ### Recommended Action Sequence
>
> **1. Profile the source**
>
> ```json theme={null}
> {"action_type": "PROFILE_SOURCE", "source": "web_forms"}
> ```
>
> Earns `+0.02`. The quality report will show null counts per column and data types, giving your agent the context it needs to prioritise transformations.
>
> **2. Standardize email (lowercase + strip)**
>
> ```json theme={null}
> {
>   "action_type": "STANDARDIZE_COLUMN",
>   "source": "web_forms",
>   "column": "email",
>   "standardization_strategy": "LOWERCASE_STRIP"
> }
> ```
>
> Calls `df["email"].str.lower().str.strip()`. Earns up to `+0.05` based on how many values were uppercase.
>
> **3. Standardize name (strip whitespace)**
>
> ```json theme={null}
> {
>   "action_type": "STANDARDIZE_COLUMN",
>   "source": "web_forms",
>   "column": "name",
>   "standardization_strategy": "LOWERCASE_STRIP"
> }
> ```
>
> **4. Standardize phone (extract to E.164)**
>
> ```json theme={null}
> {
>   "action_type": "STANDARDIZE_COLUMN",
>   "source": "web_forms",
>   "column": "phone",
>   "standardization_strategy": "EXTRACT_NUMBERS"
> }
> ```
>
> The `EXTRACT_NUMBERS` handler strips extension suffixes, removes all non-digit characters, and prepends `+`. An all-zeros phone (`+00000000000`) from a bot row will survive this step but will be filtered in the SQL step.
>
> **5. Standardize signup\_date to ISO 8601**
>
> ```json theme={null}
> {
>   "action_type": "STANDARDIZE_COLUMN",
>   "source": "web_forms",
>   "column": "signup_date",
>   "standardization_strategy": "TO_DATETIME_ISO"
> }
> ```
>
> Uses `pd.to_datetime(errors='coerce')` — invalid dates like `??/??/????` and `2023-13-12` become `NaT` and are then written as `""`.
>
> **6. Deduplicate by email**
>
> ```json theme={null}
> {
>   "action_type": "DEDUPLICATE",
>   "source": "web_forms",
>   "deduplication_strategy": "EXACT_EMAIL"
> }
> ```
>
> Calls `df.drop_duplicates(subset=["email"])`. Fuzzy duplicates that have `email = None` are retained but will not match real customers in the grader join. Filter them next.
>
> **7. Filter bots and nulls via SQL**
>
> ```json theme={null}
> {
>   "action_type": "EXECUTE_SQL",
>   "query": "SELECT * FROM web_forms WHERE email IS NOT NULL AND email NOT LIKE '%promo-alerts%' AND email NOT LIKE '%click-rewards%' AND email NOT LIKE '%freewinners%' AND email NOT LIKE '%bulk-mail%' AND phone != '+00000000000'",
>   "output_table": "web_forms_clean"
> }
> ```
>
> Earns `+0.10`. This is your bot-removal step. Alternatively, filter on `customer_id NOT LIKE 'BOT%'`.
>
> **8. Submit the pipeline**
>
> ```json theme={null}
> {"action_type": "SUBMIT_PIPELINE", "final_source": "web_forms_clean"}
> ```
>
> Earns `0.2 × schema_match_ratio` as a step reward. The full terminal score comes from the grader.

> [tab] Task 2 — Medium
>
> ## Task 2: Legacy DB Deduplication
>
> Task 2 introduces a second data source. Your agent now has `web_forms` (the same corrupted dataset from Task 1) and `legacy_db` (a second, independently corrupted dataset containing 20 base customers plus 5 bot rows). The ground truth is a **merged, deduplicated** set of all unique customers across both sources — keyed to the `merged_output` table name.
>
> ### Data Sources
>
> **`web_forms`** — identical generation to Task 1: 40 real rows corrupted with the full chaos engine, 8 bot rows injected.
>
> **`legacy_db`** — generated by a second call to `create_base_truth(20)` followed by `correlate_truth()`. This produces \~20 base rows (with potential fuzzy duplicate appendages) plus 5 bot rows, then shuffled. The column names are identical to `web_forms` (`customer_id`, `name`, `email`, `signup_date`, `phone`) — but the data quality issues are independently randomised, so a customer that has a clean email in `web_forms` may have a corrupted email in `legacy_db`.
>
> ### Ground Truth Construction
>
> ```python theme={null}
> # From server/data_generator.py — generate_medium_task()
> df_truth = pd.concat([
>     web_forms_truth,    # 40 clean rows
>     pd.DataFrame(truth2)  # 20 clean rows
> ]).drop_duplicates(subset=["email"]).reset_index(drop=True)
> ```
>
> The hidden truth is the outer union of both sources' clean rows, deduplicated by `email`. This means:
>
> * If a customer appears in both sources (same email), only **one** row is in the truth.
> * If a customer appears in only one source, that row is still in the truth.
> * The truth is keyed to `merged_output` — your agent must create a table by that exact name.
>
> ### Ground Truth Target Schema
>
> Same as Task 1:
>
> | Column | Type | Format Requirement |
> | - | - | - |
> | `customer_id` | string | `CUST_{n}` format |
> | `name` | string | Stripped |
> | `email` | string | Lowercase, stripped |
> | `signup_date` | string | ISO 8601 or `""` |
> | `phone` | string | E.164 or `""` |
>
> ### What Makes This Harder
>
> The challenge in Task 2 is not just cleaning two sources — it is **producing the correct merged set size**. The truth contains up to 60 unique customers (40 from web\_forms + 20 from legacy\_db, minus any email-level overlap). If your agent over-deduplicates (removing real rows) or under-deduplicates (keeping both copies of the same customer), the grader's row count penalty activates.
>
> The `evaluate_dataframes()` grader still joins on `customer_id`, which means both sources' records must survive the merge with their original `customer_id` values intact — your agent cannot drop `customer_id` or change it during the merge.
>
> ### Recommended Action Sequence
>
> **1. Profile both sources**
>
> ```json theme={null}
> {"action_type": "PROFILE_SOURCE", "source": "web_forms"}
> ```
>
> ```json theme={null}
> {"action_type": "PROFILE_SOURCE", "source": "legacy_db"}
> ```
>
> Each profile earns `+0.02`. Examine the null counts for both sources before standardising.
>
> **2. Standardize web\_forms columns**
>
> Apply the same four-step standardization from Task 1 to `web_forms`: email `LOWERCASE_STRIP`, name `LOWERCASE_STRIP`, phone `EXTRACT_NUMBERS`, signup\_date `TO_DATETIME_ISO`.
>
> **3. Filter web\_forms bots via SQL**
>
> ```json theme={null}
> {
>   "action_type": "EXECUTE_SQL",
>   "query": "SELECT * FROM web_forms WHERE customer_id NOT LIKE 'BOT%' AND email IS NOT NULL",
>   "output_table": "web_forms_clean"
> }
> ```
>
> **4. Standardize legacy\_db columns**
>
> Apply the same four-step standardization to `legacy_db`: email `LOWERCASE_STRIP`, name `LOWERCASE_STRIP`, phone `EXTRACT_NUMBERS`, signup\_date `TO_DATETIME_ISO`.
>
> **5. Filter legacy\_db bots via SQL**
>
> ```json theme={null}
> {
>   "action_type": "EXECUTE_SQL",
>   "query": "SELECT * FROM legacy_db WHERE customer_id NOT LIKE 'BOT%' AND email IS NOT NULL",
>   "output_table": "legacy_db_clean"
> }
> ```
>
> **6. UNION both sources**
>
> ```json theme={null}
> {
>   "action_type": "EXECUTE_SQL",
>   "query": "SELECT * FROM web_forms_clean UNION ALL SELECT * FROM legacy_db_clean",
>   "output_table": "merged_output"
> }
> ```
>
> `UNION ALL` preserves all rows including cross-source duplicates. You'll deduplicate in the next step.
>
> **7. Deduplicate the merged table by email**
>
> ```json theme={null}
> {
>   "action_type": "DEDUPLICATE",
>   "source": "merged_output",
>   "deduplication_strategy": "EXACT_EMAIL"
> }
> ```
>
> This calls `drop_duplicates(subset=["email"])` on the combined table, keeping one row per unique email address.
>
> **8. Submit**
>
> ```json theme={null}
> {"action_type": "SUBMIT_PIPELINE", "final_source": "merged_output"}
> ```
>
> > [!TIP]
> >
> > The `UNION ALL` + `DEDUPLICATE` pattern is more reliable than `MERGE_SOURCES` for Task 2 because `DEDUPLICATE` uses Pandas `drop_duplicates()` which handles `None` emails gracefully (they are not treated as matching duplicates), whereas the `MERGE_SOURCES` outer join can produce Cartesian-product issues when the join key has many nulls.

> [tab] Task 3 — Hard
>
> ## Task 3: 3-Way Enterprise Merge
>
> Task 3 is the full-complexity benchmark. Your agent must unify three sources — `salesforce`, `web_leads`, and `legacy_db` — each with a **different column schema**, **independently injected bots**, and **explicit conflict resolution rules** that dictate which source wins when the same customer has different values across sources.
>
> ### Source Schemas and Generation Parameters
>
> The three sources are generated from the same 60-customer base truth, then individually subsampled, corrupted, and augmented with bots:
>
> **`salesforce`** — 90% sample (\~54 rows) + 5 bots
>
> | Column | Notes |
> | - | - |
> | `customer_id` | `CUST_{n}` format — canonical identifier |
> | `email` | Lowercase — this is the authoritative email |
> | `phone` | Present \~70% of the time (`None` otherwise) |
>
> **`web_leads`** — 80% sample (\~48 rows) + 5 bots
>
> | Column | Notes |
> | - | - |
> | `customer_id` | Nullable \~20% of the time — some web sign-ups don't yet have a CRM ID |
> | `email` | Lowercase — matches Salesforce email |
> | `phone` | **Fresh phone number** (independently generated via `fake.msisdn()`), present \~60% of the time |
>
> **`legacy_db`** — 85% sample (\~51 rows) + 5 bots
>
> | Column | Notes |
> | - | - |
> | `legacy_id` | Uses `OLD-{n}` prefix instead of `CUST_{n}` (e.g., `OLD-1000` maps to `CUST_1000`) |
> | `contact_email` | **UPPERCASE** — must be lowercased before any join |
> | `home_phone` | Original phone from base truth — older than web\_leads |
>
> ### Conflict Resolution Rules
>
> The `CONFLICT_RULES` dictionary from `server/data_generator.py` is exposed to your agent in every observation as `conflict_rules`:
>
> ```python theme={null}
> CONFLICT_RULES = {
>     "email":       "prefer_salesforce",   # SF email is canonical
>     "phone":       "prefer_web_leads",    # Web leads has fresher phone data
>     "customer_id": "prefer_salesforce",
> }
> ```
>
> The hidden truth is built by applying these rules:
>
> ```python theme={null}
> for r in truth:
>     resolved_phone = wl_phones.get(r["customer_id"], r["phone"])  # WL > SF
>     resolved_truth.append({
>         "customer_id": r["customer_id"],   # prefer_salesforce
>         "email":       r["email"],          # prefer_salesforce
>         "phone":       resolved_phone,      # prefer_web_leads
>     })
> ```
>
> `wl_phones` maps every `customer_id` to the fresh phone generated for the `web_leads` source. If web\_leads had a phone for that customer, the truth phone is the web\_leads phone. If not, it falls back to the Salesforce phone.
>
> ### Bot Injection Details
>
> Bots are injected into all three sources. Each source gets 5 bot rows:
>
> ```python theme={null}
> # Salesforce bots
> {"customer_id": "BOT{5-digit}", "email": "user42@promo-alerts.net", "phone": "+00000000000"}
>
> # Web Leads bots
> {"customer_id": None, "email": "user99@click-rewards.xyz", "phone": None}
>
> # Legacy DB bots
> {"legacy_id": "BOT{5-digit}", "contact_email": "BULK@FREEWINNERS.INFO", "home_phone": "+00000000000"}
> ```
>
> Bot domains across all sources: `promo-alerts.net`, `click-rewards.xyz`, `freewinners.info`, `bulk-mail.co`.
>
> Filtering bots requires checking both the `customer_id` / `legacy_id` prefix (`BOT`) and the email domain. Note that legacy\_db bot emails are uppercase (`BULK@FREEWINNERS.INFO`) — standardise `contact_email` before filtering.
>
> ### Ground Truth Target Schema
>
> | Column | Type | Format Requirement |
> | - | - | - |
> | `customer_id` | string | `CUST_{n}` format (from Salesforce, not `OLD-{n}`) |
> | `email` | string | Lowercase (from Salesforce — canonical) |
> | `phone` | string | E.164 (from Web Leads when available, else Salesforce) |
>
> There are only **3 columns** in the Task 3 truth. Your final `merged_output` must have exactly these three columns.
>
> ### EXECUTE\_SQL Strategy for the 3-Way JOIN
>
> > [!WARNING]
> >
> > Calling `SUBMIT_PIPELINE` before 3 steps in Task 3 incurs **-0.15** and the episode continues. Task 3 requires the most preparation of any task — budget at least 7-8 actions for a complete pipeline.
>
> The recommended approach uses a single `EXECUTE_SQL` action with a `GROUP BY email` aggregation over a `UNION ALL` of all three standardised sources:
>
> ```sql theme={null}
> SELECT
>     MAX(customer_id)  AS customer_id,
>     email,
>     MAX(phone)        AS phone
> FROM (
>     -- Salesforce: canonical email, customer_id, and SF phone
>     SELECT customer_id, email, phone
>     FROM salesforce
>     WHERE customer_id NOT LIKE 'BOT%'
>       AND email NOT LIKE '%promo-alerts%'
>       AND email NOT LIKE '%click-rewards%'
>       AND email NOT LIKE '%freewinners%'
>       AND email NOT LIKE '%bulk-mail%'
>
>     UNION ALL
>
>     -- Web Leads: fresh phone, filter bot domains and null customer_id
>     SELECT customer_id, email, phone
>     FROM web_leads
>     WHERE customer_id IS NOT NULL
>       AND email NOT LIKE '%promo-alerts%'
>       AND email NOT LIKE '%click-rewards%'
>       AND email NOT LIKE '%freewinners%'
>       AND email NOT LIKE '%bulk-mail%'
>
>     UNION ALL
>
>     -- Legacy DB: remap legacy_id → customer_id, lowercase email already applied
>     SELECT REPLACE(legacy_id, 'OLD-', 'CUST_') AS customer_id,
>            contact_email AS email,
>            home_phone    AS phone
>     FROM legacy_db
>     WHERE legacy_id NOT LIKE 'BOT%'
>       AND contact_email NOT LIKE '%PROMO-ALERTS%'
>       AND contact_email NOT LIKE '%BULK-MAIL%'
> ) sub
> GROUP BY email
> ```
>
> The `GROUP BY email` with `MAX(customer_id)` implements the `prefer_salesforce` rule — because `CUST_` sorts lexicographically after `OLD-`, `MAX()` picks the Salesforce-format ID. The `MAX(phone)` implements a loose approximation of `prefer_web_leads` (fresh phones generated by Faker are not necessarily lexicographically larger, so this is imperfect — the deterministic `RuleBasedBaseline` in `inference.py` achieves perfection through the specific SQL shown there).
>
> > [!NOTE]
> >
> > Remember to standardise `legacy_db.contact_email` to lowercase with `STANDARDIZE_COLUMN` **before** executing the SQL join. The SQL `LIKE` patterns above are case-sensitive in SQLite by default for ASCII characters, so an uppercase `BULK@FREEWINNERS.INFO` domain will not be caught by a lowercase `LIKE '%bulk-mail%'` filter unless the column has already been lowercased.
>
> ### Recommended Action Sequence
>
> **1. Profile all three sources**
>
> ```json theme={null}
> {"action_type": "PROFILE_SOURCE", "source": "salesforce"}
> {"action_type": "PROFILE_SOURCE", "source": "web_leads"}
> {"action_type": "PROFILE_SOURCE", "source": "legacy_db"}
> ```
>
> Profile each source first to understand null distributions. Each earns `+0.02`.
>
> **2. Standardize Salesforce columns**
>
> ```json theme={null}
> {"action_type": "STANDARDIZE_COLUMN", "source": "salesforce", "column": "email", "standardization_strategy": "LOWERCASE_STRIP"}
> {"action_type": "STANDARDIZE_COLUMN", "source": "salesforce", "column": "phone", "standardization_strategy": "EXTRACT_NUMBERS"}
> ```
>
> **3. Standardize Web Leads columns**
>
> ```json theme={null}
> {"action_type": "STANDARDIZE_COLUMN", "source": "web_leads", "column": "email", "standardization_strategy": "LOWERCASE_STRIP"}
> {"action_type": "STANDARDIZE_COLUMN", "source": "web_leads", "column": "phone", "standardization_strategy": "EXTRACT_NUMBERS"}
> ```
>
> **4. Standardize Legacy DB columns — critical step**
>
> ```json theme={null}
> {"action_type": "STANDARDIZE_COLUMN", "source": "legacy_db", "column": "contact_email", "standardization_strategy": "LOWERCASE_STRIP"}
> {"action_type": "STANDARDIZE_COLUMN", "source": "legacy_db", "column": "home_phone", "standardization_strategy": "EXTRACT_NUMBERS"}
> ```
>
> The `contact_email` column is UPPERCASE by design. Failing to lowercase it before the SQL join will cause the `GROUP BY email` to treat `ALICE@EXAMPLE.COM` and `alice@example.com` as different customers, exploding the row count.
>
> **5. Execute the 3-way UNION + GROUP BY SQL merge**
>
> ```json theme={null}
> {
>   "action_type": "EXECUTE_SQL",
>   "query": "SELECT MAX(customer_id) AS customer_id, email, MAX(phone) AS phone FROM (SELECT customer_id, email, phone FROM salesforce WHERE customer_id NOT LIKE 'BOT%' AND email NOT LIKE '%promo-alerts%' UNION ALL SELECT customer_id, email, phone FROM web_leads WHERE customer_id IS NOT NULL AND email NOT LIKE '%promo-alerts%' UNION ALL SELECT REPLACE(legacy_id, 'OLD-', 'CUST_') AS customer_id, contact_email AS email, home_phone AS phone FROM legacy_db WHERE legacy_id NOT LIKE 'BOT%') GROUP BY email",
>   "output_table": "merged_output"
> }
> ```
>
> On success, earns `+0.10`. The `merged_output` table now contains one row per unique email address with the conflict-resolved `customer_id` and `phone`.
>
> **6. Submit the pipeline**
>
> ```json theme={null}
> {"action_type": "SUBMIT_PIPELINE", "final_source": "merged_output"}
> ```
>
> The grader will compare `merged_output` against the hidden truth built from `_apply_conflict_rules()` in `data_generator.py`.
>
> > [!TIP]
> >
> > The `inference.py` `RuleBasedBaseline` for Task 3 achieves a perfect score on every episode. If your LLM agent is underperforming on Task 3, compare its SQL output row count against `len(truth_df)` using `PROFILE_SOURCE` on `merged_output` after the SQL step. Excess rows always point to bot rows that slipped through your WHERE filters.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.