CRMPipelineAction Pydantic objects sent as JSON to POST /step. Each action type maps to a concrete data engineering operation inside the environment. The environment dispatches on the action_type field, routes to the corresponding handler method, mutates the in-memory Pandas DataFrames, computes a heuristic step reward, and returns a fresh CRMPipelineObservation. All fields not required by a given action type should be omitted or set to null — the Pydantic model treats them as Optional.
The base action structure is minimal by design:
action_type is optional at the model level, but individual handlers will raise a ValueError (caught as a -0.05 penalty) if required fields are absent for that specific operation. The full field reference is at the bottom of this page.
The 8 Action Types
VIEW_SOURCE — Preview a DataFrame
VIEW_SOURCE — Preview a DataFrame
What it does: Calls Example feedback:Example current_view output:
df.head(3).to_markdown() on the named source and stores the result in the current view. This is a read-only action — it does not mutate any DataFrame. Use it to understand the raw structure of a source before deciding which transformations to apply.Required fields:source— the name of the DataFrame to preview (must exist inavailable_sources)
current_view— a markdown-formatted table of the first 3 rowslast_action_feedback—"Viewing top 3 rows of <source>"data_quality_report— not updated by this action
0.0 — purely informational, no reward signalWhen to use it: Call VIEW_SOURCE at the start of an episode to understand column names, example values, and obvious formatting problems. Also useful after a MERGE_SOURCES or EXECUTE_SQL operation to verify the shape of a newly created DataFrame.VIEW_SOURCE shows only 3 rows. For datasets of 40–70 rows, those 3 rows may not be representative of the full distribution of data quality problems. Always follow up with PROFILE_SOURCE to get statistical summaries across all rows.PROFILE_SOURCE — Generate a Quality Report
PROFILE_SOURCE — Generate a Quality Report
What it does: Computes null counts and data types for every column in the named source DataFrame, then formats the results as a markdown-style report. This updates the Example data_quality_report:Example feedback:
data_quality_report field in the observation. Like VIEW_SOURCE, this is read-only and does not mutate the DataFrame.Required fields:source— the name of the DataFrame to profile
data_quality_report— a per-column report in the format- **<col>**: type=<dtype>, nulls=<count>last_action_feedback—"Profiled <source>"
+0.02 — rewarded for gathering information before actingWhen to use it: Call PROFILE_SOURCE on every source at the start of a task. This is the primary mechanism for overcoming the environment’s partial observability — null counts reveal which columns need HANDLE_MISSING, and dtype values reveal which columns need STANDARDIZE_COLUMN. Without profiling, your agent is operating blind.STANDARDIZE_COLUMN — Apply a Deterministic Format Transform
STANDARDIZE_COLUMN — Apply a Deterministic Format Transform
What it does: Applies one of four deterministic string transformation strategies to a single column in a named source. The transformation is applied in-place to the specified column. The reward is heuristically calculated based on how many values were actually changed (a proxy for how dirty the column was).Required fields:Example feedback:
source— the DataFrame containing the columncolumn— the column to transformstandardization_strategy— one of the four strategies below
min(0.05, 0.01 + improvement_ratio × 0.1) on success (minimum +0.01, maximum +0.05), -0.05 on exception (e.g., column not found, source not found)Strategies in depth:LOWERCASE_STRIP — Converts all string values to lowercase and strips leading/trailing whitespace. Implemented as df[col].astype(str).str.lower().str.strip(). Use this on email and name columns where values may be uppercase or padded.TO_DATETIME_ISO — Parses dates using pd.to_datetime(errors='coerce') and formats them as '%Y-%m-%dT00:00:00'. Unparseable values (like "??/??/????" or "2023-13-12") become empty strings via .fillna(""). Use this on signup_date columns.EXTRACT_NUMBERS — Strips all non-digit characters from phone values using re.sub(r'\D+', '', s), then prepends a + prefix. Also handles extension stripping — values containing ext or x are split at that boundary before digit extraction. Values that produce no digits return an empty string. Use this on phone columns with mixed formats like "(415) 555-0192" or "555-1234 ext. 42".PHONE_E164 — Defined in the StandardizationStrategy enum but not separately implemented in the current handler. The standardization dispatch has no branch for PHONE_E164, so calling it returns a minimal reward (+0.01) without transforming any values. For phone normalization, use EXTRACT_NUMBERS instead — it provides the full digit-extraction and + prefix logic.HANDLE_MISSING — Fill or Drop Null Values
HANDLE_MISSING — Fill or Drop Null Values
What it does: Resolves null (NaN) values in a specific column of a named source using one of four strategies. The
DROP_ROW and FILL_VALUE strategies are fully implemented and return proportional rewards. The FILL_MEAN and FILL_MODE strategies are defined in the enum but not yet implemented in the handler — submitting them raises an internal exception that is caught as a -0.05 penalty. Use FILL_VALUE or DROP_ROW for reliable null handling.Required fields:source— the DataFrame containing the columncolumn— the column with null values to handlemissing_strategy— one of the four strategies (FILL_VALUEandDROP_ROWare implemented)
fallback_value— required only whenmissing_strategyisFILL_VALUE. Ignored for all other strategies.
+0.01 to +0.05 on success (proportional to null fraction resolved), -0.05 on exception or unimplemented strategyStrategies in depth:FILL_VALUE (implemented) — Fills all null values in the column with the literal string provided in fallback_value, as df[col].fillna(action.fallback_value, inplace=True). Use this to fill missing phone numbers with "" (empty string) or missing emails with a sentinel value.DROP_ROW (implemented) — Drops all rows where the specified column is null using df.dropna(subset=[col], inplace=True). This is a destructive operation — rows are permanently removed from the episode’s working DataFrame. Use with caution: dropping rows with missing emails may remove legitimate customers, reducing your grader score.FILL_MEAN (not yet implemented — returns -0.05) — Defined in the enum but the handler has no branch for this strategy. Submitting it raises an UnboundLocalError caught as a -0.05 penalty.FILL_MODE (not yet implemented — returns -0.05) — Defined in the enum but the handler has no branch for this strategy. Submitting it raises an UnboundLocalError caught as a -0.05 penalty.Reward formulas:DROP_ROW:max(0.01, (null_count_before / (len(df_after_drop) + null_count_before)) × 0.1)FILL_VALUE:max(0.01, (null_count_before / len(df)) × 0.05)
DEDUPLICATE — Remove Duplicate Records
DEDUPLICATE — Remove Duplicate Records
What it does: Identifies and removes duplicate rows from a named source using one of two strategies. Operates in-place and saves the deduplicated DataFrame back to the sources store. The reward is proportional to the fraction of rows removed (i.e., how many duplicates were found).Required fields:Example feedback:
source— the DataFrame to deduplicatededuplication_strategy— one of the two strategies below
(removed / start_len) × 0.2 — proportional to duplicate density, maximum 0.2 if all rows were duplicates. Returns -0.05 on exception.Strategies in depth:EXACT_EMAIL — Calls df.drop_duplicates(subset=["email"], inplace=True). Keeps the first occurrence of each unique email address and removes all subsequent rows with the same email. This is appropriate when email is the reliable unique identifier. Note that rows with null emails are treated as a single group by Pandas’ drop_duplicates — the first null-email row is kept.FUZZY_NAME_PHONE — A soft-match deduplication strategy. Creates two temporary columns: _tmp_name (lowercase-stripped name) and _tmp_phone (digits-only phone). Drops duplicates on the combined key ["_tmp_name", "_tmp_phone"], then removes the temporary columns. This catches cases where the same person appears twice with slightly different formatting (e.g., " Alice Johnson " vs "alice johnson") but the same normalized phone.MERGE_SOURCES — Join Two DataFrames
MERGE_SOURCES — Join Two DataFrames
What it does: Performs an outer join between two named source DataFrames on a specified key column, resolves column conflicts according to the Example feedback:
conflict_rule, and stores the result as "merged_output" in the sources store. The join uses pd.merge(df1, df2, on=key, how="outer", suffixes=('_s1', '_s2')) and then reconciles every pair of conflicting columns.Required fields:source— the first (primary) DataFramesource2— the second DataFrame to joinjoin_key— the column name shared by both DataFrames to join onoutput_table— the name to use for the merged result inavailable_sources
conflict_rule— one ofPREFER_S1,PREFER_S2, orCOALESCE(default behavior if omitted isCOALESCE)
+0.05 on success, -0.05 if the join key is missing from either sourceConflict rules:Both DataFrames are normalized on the join key before merging (astype(str).str.lower().str.strip()), which enables fuzzy key matching.After the outer join, any column that existed in both DataFrames (and thus received _s1 / _s2 suffixes) is reconciled:PREFER_S1— Takes the value from the first source; falls back to the second if null:merged[s1_col].combine_first(merged[s2_col])PREFER_S2— Takes the value from the second source; falls back to the first if null:merged[s2_col].combine_first(merged[s1_col])COALESCE— Takes the first non-null value from either source (functionally equivalent toPREFER_S1in this implementation):merged[s1_col].combine_first(merged[s2_col])
MERGE_SOURCES always writes to "merged_output" in the sources store, regardless of the output_table value. The output_table field is accepted by the model but the handler hardcodes the key "merged_output". After a merge, "merged_output" will appear in your available_sources list. For a 3-way merge in Task 3, use EXECUTE_SQL to perform a custom 3-table join instead.EXECUTE_SQL — Run a SELECT Query Over DataFrames
EXECUTE_SQL — Run a SELECT Query Over DataFrames
What it does: Loads all current DataFrames into an in-memory SQLite database (one table per source, named by its key in the sources store), executes your Example — Filter bots from a source:Example — 3-way merge for Task 3:Example — Rename legacy_db columns and filter:Example feedback (success):Example feedback (error):Example feedback (blocked):
SELECT query via pd.read_sql_query(), stores the result DataFrame under output_table, and closes the connection. This is the most powerful action in the space — it enables arbitrary filtering, joining, column renaming, and aggregation.Required fields:query— a valid SQLiteSELECTstatementoutput_table— the name to register the result under inavailable_sources
+0.1 on success, -0.1 on SQL error or security violationSecurity restrictions: The query is uppercased and checked for forbidden keywords before execution. Queries that do not begin with SELECT are blocked. The following keywords also block execution: DROP , DELETE , UPDATE , INSERT , ALTER , CREATE , TRUNCATE .SUBMIT_PIPELINE — Finalize and Trigger Grading
SUBMIT_PIPELINE — Finalize and Trigger Grading
What it does: Signals the end of the episode. If the For Task 2 or 3 where a merged output was produced:Example feedback (success):Example feedback (premature):
step_count is below MIN_STEPS_BEFORE_SUBMIT (3), the submission is rejected with a penalty and the episode continues. Otherwise, done=True is set, the final DataFrame is captured from the named source, a schema-match heuristic reward is computed, and the grader engine is triggered server-side to produce the terminal score.Required fields:final_source— the name of the DataFrame inavailable_sourcesthat represents your cleaned output
-0.15, done=FalseReward (successful submission): 0.2 × schema_match_ratio where schema_match_ratio = len([c for c in schema_target if c in final_df.columns]) / max(1, len(schema_target)). Maximum possible step reward is +0.20.Terminal behavior: done=True is returned in the observation. The episode is over — no further step() calls will produce meaningful state changes.The
final_source you name must exist in available_sources at the time of submission. If the DataFrame does not exist, the grader receives an empty DataFrame and returns 0.01 (the minimum score floor).Full Action Field Reference
The following table lists every field inCRMPipelineAction along with which action types require or use it.