Skip to main content
POST
/
v1
/
ai
/
eval
/
runs
Start an eval run
curl --request POST \
  --url https://{instance}.omniapp.co/api/v1/ai/eval/runs \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "prompt_set_id": "550e8400-e29b-41d4-a716-446655440000",
  "description": "Re-running after switching to gpt-4o for query generation",
  "run_config": {
    "branch_id": "440e8400-e29b-41d4-a716-446655440006"
  }
}
'
{
  "job_count": 12,
  "run": {
    "branch_id": null,
    "branch_name": null,
    "completed_at": null,
    "created_at": "2025-01-15T10:00:00.000Z",
    "description": null,
    "id": "660e8400-e29b-41d4-a716-446655440001",
    "is_archived": false,
    "model_id": "880e8400-e29b-41d4-a716-446655440003",
    "prompt_set_id": "550e8400-e29b-41d4-a716-446655440000",
    "results": [
      {
        "agentic_job": {
          "conversation_id": "770e8400-e29b-41d4-a716-446655440002",
          "id": "990e8400-e29b-41d4-a716-446655440004",
          "state": "COMPLETE"
        },
        "cost": 0.0021,
        "error_reason": null,
        "id": "aa0e8400-e29b-41d4-a716-446655440005",
        "prompt": "What are the top 5 products by revenue?",
        "score": 0.9,
        "scoring_cost": 0.0004,
        "timing_ms": 4321
      }
    ],
    "run_number": 3,
    "status": "RUNNING"
  }
}

Authorizations

​
Authorization
string
header
required

Can be either an Organization API Key or Personal Access Token (PAT).

Include in the Authorization header as: Bearer YOUR_TOKEN

Body

application/json
​
prompt_set_id
string<uuid>
required

The prompt set to execute.

Example:

"550e8400-e29b-41d4-a716-446655440000"

​
description
string | null

Optional human-readable description for the run. Omit or pass null to leave it unset. Max 1024 characters.

Maximum string length: 1024
Example:

"Re-running after switching to gpt-4o for query generation"

​
run_config
object

Per-run configuration. Optional — omit if no overrides.

Response

Run created and jobs enqueued.

​
job_count
integer
required

Number of per-prompt agentic jobs created for this run (one per prompt that fanned out successfully). Enqueue onto the work queue happens after creation and is best-effort, so this count reflects jobs created, not necessarily those successfully enqueued.

Example:

12

​
run
object
required

The newly created run with its initial results.