OpenAI · via openai · published window from October 3, 2026

gpt-6.1-sol: stability, speed and price

site endpoint id gpt-6.1-sol

Last successful scored check: November 7, 2026 2:21 AM UTC

Core pass ratetied #1 / 3

100.0%

deterministic checks passed, latest measured day · median 100.0%

Speed#2 / 3

1.6s

median latency per canary call · p95 15.3s · median endpoint 1.6s

Cost#1 / 3

$0.115

observed mean per scored run · probe mix differs by endpoint · median $0.579

Behaviour fingerprint435 runs

Change detected

24,229 responses · last adj. p 0.00067 · Nov 7 2:21 AM UTC

Summary

gpt-6.1-sol passes 100.0% of the core deterministic checks over its latest measured day, tied #1 (2 share it) of 3 among recently checked endpoints (median 100.0%). On the hard probes (code execution, constraint logic, state tracking) it passes 86%, #2 of 3 measured on them.

Its median canary latency is 1.6s (p95 15.3s), #2 of 3 fastest, in the upper half; the median endpoint answers in 1.6s. Its observed mean cost per scored run is $0.115, #1 of 3 cheapest (median $0.579; the probe mix differs by endpoint).

Raven Alpha has 24,229 fingerprint responses over 435 runs in this endpoint’s selected series (1 other partition of history exist and are not shown here); the last recorded state is “change detected” at 2026-11-07T02:21:14.673Z with an adjusted p of 0.00067.

4 incidents on record, 1 open.

Written from the numbers on this page; every figure is in results.json.

Measurement record

Provider: OpenAI
Route: openai
Model requested: gpt-6.1-sol
Model id returned: gpt-6.1-sol
Scored calls in window: 9,573
Fingerprint responses: 24,229
Core score change: +0.0 pts across the published window
Status: ● Drift detected
Suite: hard-v2 + starter-v2 · methodology v0.2

Capabilities over time

Latest measured day by capability

Structured JSON100.0%

Schema-compliance regressions

Instruction following100.0%

Exact-format instruction drift

Arithmetic, answer only100.0%

Wrong arithmetic, or a model that starts showing its working when told to give only the number

Long context100.0%

Silent context truncation

Tool calling100.0%

Tool-use format regressions

Over-refusal100.0%

New safety layers refusing benign requests

Max-tokens respect100.0%

Truncation and limit changes

Code execution (hard)60.0%

Reasoning depth or reasoning-budget cuts

Constraint logic (hard)96.7%

Multi-step deduction regressions

State tracking (hard)100.0%

Long-horizon bookkeeping errors

Incidents

● Opengpt-6.1-solConfidence: Low

Raven Alpha (behaviour fingerprint): Responses to fixed prompts no longer match this model’s own history (1344 recent vs 7961 baseline responses). Biggest shifts: output length on “Pick a day of the week. Reply with one word.”; reasoning tokens on “Pick a day of the week. Reply with one word.”; whether it reasoned at all on “all prompts together”

Change began ≈ Oct 12, 4:40 PM UTC · adjusted p 6.7e-4 · still open · Permalink

✓ Resolvedgpt-6.1-solConfidence: Low

Raven Alpha (behaviour fingerprint): Responses to fixed prompts no longer match this model’s own history (168 recent vs 5385 baseline responses). Biggest shifts: visible output length on “Name one programming language. Reply with only its name.”; whether it reasoned at all on “Name one programming language. Reply with only its name.”; output length on “Name one programming language. Reply with only its name.”

Change began ≈ Oct 9, 2:28 AM UTC · adjusted p 1.6e-3 · resolved Oct 13 · Permalink

✓ Resolvedgpt-6.1-solConfidence: Medium

Raven Alpha (behaviour fingerprint): Responses to fixed prompts no longer match this model’s own history (1297 recent vs 2968 baseline responses). Biggest shifts: output length on “Name one animal. Reply with one word.”; reasoning tokens on “Name one animal. Reply with one word.”; visible output length on “Name one animal. Reply with one word.”

Change began ≈ Oct 7, 8:53 PM UTC · adjusted p 2.0e-3 · resolved Oct 10 · Permalink

✓ Resolvedgpt-6.1-solConfidence: Low

Raven Alpha (behaviour fingerprint): Responses to fixed prompts no longer match this model’s own history (1344 recent vs 1008 baseline responses). Biggest shifts: whether it reasoned at all on “Name one animal. Reply with one word.”; visible output length on “Name one animal. Reply with one word.”; reasoning tokens on “Suggest a name for a pet cat. Reply with only the name.”

Change began ≈ Oct 5, 11:31 PM UTC · adjusted p 9.3e-3 · resolved Oct 7 · Permalink

Raven Alpha: what it answers

40 runs, 2,240 responses, Nov 4 2:32 PM UTC to Nov 7 2:21 AM UTC. Each row is one fixed prompt with no right answer; the detector tracks the distribution of answers, not any single one. Filled cells: that run's most common answer was the overall favourite. Hollow cells: it was something else (normal for prompts with many good answers).

Prompt	Most common answers (share of all responses)	Median tokens
Pick a random integer between 1 and 50. Reply with only the number. int50 · short answer	96%37 1%27 1%24 4 distinct answers in 80 responses	4
Pick a number between 1 and 1000. Reply with only the number. int1000 · short answer	40%437 20%427 18%637 13 distinct answers in 80 responses	14
Name one color. Reply with one word. color · short answer	100%Blue	4
Name one animal. Reply with one word. animal · short answer	76%Elephant 24%Otter	17
Name a city in Europe. Reply with only the city name. city · short answer	63%Prague 29%Paris 6%Vienna 4 distinct answers in 80 responses	5
Give me one random English noun. Reply with one word. noun · short answer	100%Lantern	18
Flip a coin. Reply with only heads or tails. coin · short answer	55%heads 45%tails	4
Pick a random letter of the alphabet. Reply with only the letter. letter · short answer	76%R 14%Q 6%M 4 distinct answers in 80 responses	4
Name one fruit. Reply with one word. fruit · short answer	98%Apple 3%Mango	4
Suggest a name for a pet cat. Reply with only the name. cat · short answer	95%Mochi 5%Luna	16
Name one programming language. Reply with only its name. lang · short answer	100%Python	4
Name a chemical element. Reply with one word. element · short answer	76%Oxygen 23%Carbon 1%Hydrogen	16
Give one adjective that describes the ocean. Reply with one word. ocean · short answer	100%Vast	18
Name a famous scientist. Reply with only the name. scientist · short answer	94%Albert Einstein 6%Marie Curie	16
Pick a day of the week. Reply with one word. weekday · short answer	41%Thursday 35%Wednesday 24%Tuesday	15
Name one musical instrument. Reply with one word. instrument · short answer	96%Piano 3%Violin 1%Guitar	5
Write one sentence about rain. rain · free text	31%Rain pattered softly against the… 30%Rain taps softly on the… 23%Rain taps softly against the… 9 distinct answers in 80 responses	31
Explain what a variable is in programming, in two sentences. variable · free text	78%A variable is a named… 23%A variable is a name…	74
List three uses for a brick. brick · free text	36%1. Building a wall or… 33%1. Building a wall. 2… 16%1. Building walls or garden… 6 distinct answers in 80 responses	39
*What is 17 23?** mult · free text	68%17 × 23 = 391… 33%391…	22
Write one sentence about the sea. sea · free text	50%The sea shimmered beneath the… 43%The sea shimmers beneath the… 4%The sea glimmers beneath the… 6 distinct answers in 80 responses	31
Complete this sentence: The old house at the end of the street house · free text	100%The old house at the…	43
Describe a cat in exactly five words. catfive · free text	56%A graceful feline with whiskers… 21%A graceful, curious, whiskered companion… 6%A graceful cat with whiskers… 11 distinct answers in 80 responses	42
Give me a one-line motivational quote. quote · free text	35%“Small steps still move you… 20%Small steps still move you… 13%You don’t need to see… 17 distinct answers in 80 responses	29
Say hello in a creative way, in one short sentence. hello · free text	24%Hello, bright spark—consider this a… 20%Hello, you delightful plot twist… 11%Hello, bright spark—welcome to this… 26 distinct answers in 80 responses	28
Write a haiku about autumn. haiku · free text	31%Amber leaves drift down Cool… 19%Red leaves drift softly Cool… 13%Amber leaves descend Cool winds… 17 distinct answers in 80 responses	102
Write the first sentence of a mystery novel. story · free text	45%The morning after we buried… 4%By the time the police… 3%My mother had been dead… 37 distinct answers in 80 responses	99
Give one tip for sleeping better, in one sentence. tip · free text	49%Keep a consistent bedtime and… 13%Keep a consistent wake-up time… 13%Go to bed and wake… 6 distinct answers in 80 responses	24

Raw file: gpt-6.1-sol.json (the same reference martingale check compares your endpoint against).

Each category is scored by deterministic checks only (Structured JSON, Tool calling and others). Read the methodology.