Configure Custom Scoring

PATCH /reports/{report_id}/score-card

cURL

weighted

curl --request PATCH \
  --url https://api.promptlayer.com/reports/{report_id}/score-card \
  --header 'Content-Type: application/json' \
  --header 'X-API-KEY: <api-key>' \
  --data '
{
  "column_names": [
    "Accuracy Check",
    "Style Check"
  ],
  "code": "weights = {\"Accuracy Check\": 0.7, \"Style Check\": 0.3}\ntotal_weight = weighted_sum = 0\nfor row in data:\n    for col_name, weight in weights.items():\n        if col_name in row and isinstance(row[col_name], bool):\n            total_weight += weight\n            if row[col_name]:\n                weighted_sum += weight\nscore = (weighted_sum / total_weight * 100) if total_weight > 0 else 0\nreturn {\"score\": score}",
  "code_language": "PYTHON"
}
'

200

{
  "success": true,
  "report": {
    "id": 456,
    "name": "My Evaluation Pipeline",
    "score_configuration": {
      "code": "return {\"score\": 85.5}",
      "code_language": "PYTHON"
    }
  }
}

Legacy Dataset, Evaluation, and Report endpoints are deprecated for new workflows. Use the Tables API for new dataset import, evaluation, scoring, recalculation, and reporting workflows.

Configure the score card for an evaluation pipeline. Use this endpoint to choose the columns that contribute to the score and, optionally, provide custom Python or JavaScript scoring code.

Behavior Notes

• Custom code receives a data variable containing evaluation rows and must return an object with at least a score key from 0 to 100.
• If no custom code is provided, PromptLayer uses default scoring over the selected columns.
• Updating a blueprint affects future runs; completed runs recalculate their score after the score card changes.

Related

• Create Evaluation Pipeline
• Score Card
• Node & Column Types

Authorizations

X-API-KEY · string · header · required

Path Parameters

report_id · integer · required

ID of the evaluation pipeline to update.

Body

application/json

Score-card configuration payload.

column_names · string[] · required

Column names to include in the score calculation.

code · string | null

Optional custom Python or JavaScript scoring code. The code receives data and must return an object with a score key.

code_language · enum<string> · default: PYTHON

Language used by the custom scoring code.

Available options: PYTHON, JAVASCRIPT

Response

200

application/json

Score card updated.

success · enum<boolean> · required

Available options: true

report · EvaluationPipelineSummary · object · required

Show child attributes