Upstage · via openrouter · published window from October 5, 2026

upstage/solar-pro4: stability, speed and price

site endpoint id upstage--solar-pro4

Historical measurements. No successful scored check in the past 24 hours. Scores, latency and cost below describe the 24-hour window ending at the last successful check, not current performance. Current rankings exclude this endpoint.

Last successful scored check: October 13, 2026 1:55 AM UTC

Core pass rate—

82.9%

historical: latest measured day

Speed—

1.3s

historical median per canary call · p95 13.3s

Cost—

$0.003

historical observed mean per scored run · probe mix differs by endpoint

Behaviour fingerprint52 runs

No change

1,452 responses · last adj. p 0.58 · Oct 13 3:25 AM UTC

Summary

upstage/solar-pro4 has no successful scored check in the past 24 hours. The last measured day ended at 2026-10-13T01:55:28.423Z; 82.9% of core checks passed. These historical measurements are excluded from current rankings.

During that historical measurement window, median canary latency was 1.3s (p95 13.3s) and the observed mean cost per scored run was $0.003. This does not describe the endpoint’s present performance.

Raven Alpha has 1,452 fingerprint responses over 52 runs in this endpoint’s selected series (1 other partition of history exist and are not shown here); the last recorded state is “no change” at 2026-10-13T03:25:01.178Z with an adjusted p of 0.58.

No incidents on record.

Written from the numbers on this page; every figure is in results.json.

Measurement record

Provider: Upstage
Route: openrouter
Model requested: upstage/solar-pro4
Model id returned: upstage/solar-pro4
Scored calls in window: 623
Fingerprint responses: 1,452
Core score change: +3.4 pts across the published window
Status: Historical · no recent check
Suite: hard-v2 + starter-v2 · methodology v0.2

Capabilities over time

Historical measurement window by capability

Structured JSON100.0%

Schema-compliance regressions

Instruction following100.0%

Exact-format instruction drift

Arithmetic, answer only0.0%

Wrong arithmetic, or a model that starts showing its working when told to give only the number

Long context100.0%

Silent context truncation

Tool calling100.0%

Tool-use format regressions

Over-refusal80.0%

New safety layers refusing benign requests

Max-tokens respect100.0%

Truncation and limit changes

Incidents

No drift incidents so far. The detector re-checks every series after each run (6134 evaluations to date).

Raven Alpha: what it answers

40 runs, 1,116 responses, Oct 6 1:43 PM UTC to Oct 13 3:25 AM UTC. Each row is one fixed prompt with no right answer; the detector tracks the distribution of answers, not any single one. Filled cells: that run's most common answer was the overall favourite. Hollow cells: it was something else (normal for prompts with many good answers).

Prompt	Most common answers (share of all responses)	Median tokens
Pick a random integer between 1 and 50. Reply with only the number. int50 · short answer	38%27 23%37 13%42 9 distinct answers in 40 responses	3
Pick a number between 1 and 1000. Reply with only the number. int1000 · short answer	38%42 10%73 10%742 13 distinct answers in 40 responses	4
Name one color. Reply with one word. color · short answer	63%Blue 38%Red	2
Name one animal. Reply with one word. animal · short answer	45%Elephant 40%Lion 8%Dog 5 distinct answers in 40 responses	3
Name a city in Europe. Reply with only the city name. city · short answer	90%Paris 8%Berlin 3%London	2
Give me one random English noun. Reply with one word. noun · short answer	13%Lantern 8%Elephant 8%Telescope 27 distinct answers in 40 responses	3
Flip a coin. Reply with only heads or tails. coin · short answer	68%Heads 30%Tails 3%Note: FlipBot does not exist. Outcome:	3
Pick a random letter of the alphabet. Reply with only the letter. letter · short answer	30%K 28%M 10%Q 11 distinct answers in 40 responses	2
Name one fruit. Reply with one word. fruit · short answer	100%Apple	2
Suggest a name for a pet cat. Reply with only the name. cat · short answer	45%Luna 40%Mochi 5%Milo 7 distinct answers in 40 responses	3
Name one programming language. Reply with only its name. lang · short answer	100%Python	2
Name a chemical element. Reply with one word. element · short answer	35%Hydrogen 28%Oxygen 25%Carbon 6 distinct answers in 40 responses	2
Give one adjective that describes the ocean. Reply with one word. ocean · short answer	93%Vast 3%Magnificent 3%Endless 4 distinct answers in 40 responses	3
Name a famous scientist. Reply with only the name. scientist · short answer	100%Albert Einstein	3
Pick a day of the week. Reply with one word. weekday · short answer	78%Wednesday 13%Monday 10%Friday	2
Name one musical instrument. Reply with one word. instrument · short answer	70%Piano 25%Guitar 5%Violin	3
Write one sentence about rain. rain · free text	15%Rain falls from the sky… 10%Rain falls gently from the… 10%Rain falls softly from the… 24 distinct answers in 40 responses	22
Explain what a variable is in programming, in two sentences. variable · free text	100%A variable is a named…	52
List three uses for a brick. brick · free text	90%Here are three common uses… 5%Here are three uses for… 3%1. Construction: Bricks are primarily… 4 distinct answers in 40 responses	128
*What is 17 23?** mult · free text	58%17 * 23 = 391… 10%17 * 23 is 391… 10%17 × 23 = 391… 10 distinct answers in 40 responses	13
Write one sentence about the sea. sea · free text	15%The sea stretches endlessly toward… 10%The sea stretches to the… 8%The sea stretches out endlessly… 22 distinct answers in 40 responses	25
Complete this sentence: The old house at the end of the street house · free text	58%The old house at the… 8%Here are a few ways… 3%...had stood silent for decades… 16 distinct answers in 40 responses	32
Describe a cat in exactly five words. catfive · free text	3%Small furry agile hunting companion… 3%A soft furry companion animal… 3%Cute furry pet purrs loudly… 40 distinct answers in 40 responses	8
Give me a one-line motivational quote. quote · free text	23%"The future belongs to those… 18%The only limit to your… 15%The future belongs to those… 13 distinct answers in 40 responses	15
Say hello in a creative way, in one short sentence. hello · free text	25%May your coffee be strong… 3%Behold, a burst of words… 3%May the sun’s first whisper… 31 distinct answers in 40 responses	16
Write a haiku about autumn. haiku · free text	8%Crimson leaves descend, Whispering winds… 8%Red leaves drift slowly, Cool… 5%Red leaves drift slowly Cool… 33 distinct answers in 40 responses	19
Write the first sentence of a mystery novel. story · free text	13%The letter arrived three days… 5%The letter arrived three years… 5%The letter arrived on a… 32 distinct answers in 38 responses	26
Give one tip for sleeping better, in one sentence. tip · free text	29%Maintain a consistent sleep schedule… 21%Keep your bedroom cool, dark… 8%To sleep better, maintain a… 15 distinct answers in 38 responses	24

Raw file: upstage--solar-pro4.json (the same reference martingale check compares your endpoint against).

Each category is scored by deterministic checks only (Structured JSON, Tool calling and others). Read the methodology.