Add Column to Evaluation Pipeline

POST /report-columns

Add Column to Evaluation Pipeline

cURL

curl --request POST \
  --url https://api.promptlayer.com/report-columns \
  --header 'Content-Type: application/json' \
  --header 'X-API-KEY: <api-key>' \
  --data '
{
  "report_id": 456,
  "column_type": "PROMPT_TEMPLATE",
  "name": "Generate Answer",
  "configuration": {
    "template": {
      "name": "qa_template",
      "version_number": null
    },
    "prompt_template_variable_mappings": {
      "question": "input_question"
    },
    "engine": {
      "provider": "openai",
      "model": "gpt-4",
      "parameters": {
        "temperature": 0.7
      }
    }
  }
}
'

201

{
  "success": true,
  "report_column": {}
}

This endpoint adds evaluation steps (columns) to an existing evaluation pipeline. Columns execute sequentially from left to right, with each column able to reference outputs from previous columns.

Important Notes

• Single Column Per Request: This endpoint only allows adding one column at a time. To add multiple columns, make separate API calls for each.
• Column Order Matters: Columns execute left to right. A column can only reference columns to its left.
• Unique Names Required: Each column name must be unique within the pipeline.
• Dataset Columns Protected: You cannot overwrite columns that come from the dataset.

Scoring

By default, only the last column in a pipeline is used for score calculation. To include multiple columns in the final score:

• Set is_part_of_score: true on each column you want to include in the score
• Columns must produce boolean or numeric values to be scored
• When multiple columns are marked for scoring, the final score is the average of all included columns

Column Types and Configuration

For the complete list of supported column types and their detailed configuration options, see the Node & Column Types documentation.

Batch Adding Columns

Since columns must be added one at a time, here’s a pattern for adding multiple columns:

import requests

columns = [
    {
        "column_type": "PROMPT_TEMPLATE",
        "name": "Generate",
        "configuration": {...}
    },
    {
        "column_type": "LLM_ASSERTION",
        "name": "Validate",
        "configuration": {...}
    }
]

for column in columns:
    response = requests.post(
        "https://api.promptlayer.com/report-columns",
        headers={"X-API-KEY": "your_key"},
        json={
            "report_id": 456,
            **column
        }
    )
    if response.status_code != 201:
        print(f"Failed: {column['name']}")
        break

Column Reference Syntax

When configuring columns that reference other columns:

• Dataset columns: Use exact column name from dataset (e.g., "question")
• Previous columns: Use the name you assigned (e.g., "AI Response")
• Variable columns: Reference by their name

Error Handling

The endpoint validates:

1. Column type is valid
2. Column name is unique within the pipeline
3. Configuration matches the column type schema
4. Referenced columns exist (for dependent columns)
5. User has permission to modify the pipeline

Common errors:

• 400: Invalid configuration or duplicate column name
• 403: Cannot overwrite dataset columns or lacking permissions
• 404: Report not found or not accessible

Authorizations

X-API-KEY · string · header · required

Body

application/json

report_id · integer · required

The ID of the evaluation pipeline to add this column to.

Required range: x >= 1

column_type · enum<string> · required

The type of evaluation or transformation this column performs. Must be one of the supported column types.

Available options: ABSOLUTE_NUMERIC_DISTANCE, AI_DATA_EXTRACTION, ASSERT_VALID, CONVERSATION_SIMULATOR, COALESCE, CODE_EXECUTION, COMBINE_COLUMNS, COMPARE, CONTAINS, COSINE_SIMILARITY, COUNT, ENDPOINT, MCP, HUMAN, JSON_PATH, LLM_ASSERTION, MATH_OPERATOR, MIN_MAX, PARSE_VALUE, APPLY_DIFF, PROMPT_TEMPLATE, REGEX, REGEX_EXTRACTION, VARIABLE, XML_PATH, WORKFLOW, CODING_AGENT

name · string · required

Display name for this column. Must be unique within the pipeline. This name is used to reference the column in subsequent steps.

Required string length: 1 - 255

configuration · object · required

Column-specific configuration. The schema varies based on column_type. See documentation for each type's requirements.

position · integer | null

Optional position for the column. If not specified, the column is added at the end. Cannot overwrite dataset columns.

Required range: x >= 0

Response

201

application/json

Column added successfully

success · boolean

Example:

true

report_column · object

Details of the created column including its ID and configuration