Microsoft · via openrouter · published window from October 5, 2026

microsoft/phi-4: stability, speed and price

site endpoint id microsoft--phi-4

Historical measurements. No successful scored check in the past 24 hours. Scores, latency and cost below describe the 24-hour window ending at the last successful check, not current performance. Current rankings exclude this endpoint.

Last successful scored check: October 13, 2026 1:54 AM UTC

Core pass rate—

55.0%

historical: latest measured day

Speed—

1.3s

historical median per canary call · p95 7.3s

Cost—

$0.002

historical observed mean per scored run · probe mix differs by endpoint

Behaviour fingerprint53 runs

No change

1,459 responses · last adj. p 0.56 · Oct 13 3:25 AM UTC

Summary

microsoft/phi-4 has no successful scored check in the past 24 hours. The last measured day ended at 2026-10-13T01:54:54.133Z; 55.0% of core checks passed. These historical measurements are excluded from current rankings.

During that historical measurement window, median canary latency was 1.3s (p95 7.3s) and the observed mean cost per scored run was $0.002. This does not describe the endpoint’s present performance.

Raven Alpha has 1,459 fingerprint responses over 53 runs in this endpoint’s selected series (1 other partition of history exist and are not shown here); the last recorded state is “no change” at 2026-10-13T03:25:01.423Z with an adjusted p of 0.56.

No incidents on record.

Written from the numbers on this page; every figure is in results.json.

Measurement record

Provider: Microsoft
Route: openrouter
Model requested: microsoft/phi-4
Model id returned: microsoft/phi-4
Scored calls in window: 545
Fingerprint responses: 1,459
Core score change: -4.0 pts across the published window
Status: Historical · no recent check
Suite: hard-v2 + starter-v2 · methodology v0.2

Capabilities over time

Historical measurement window by capability

Structured JSON0.0%

Schema-compliance regressions

Instruction following50.0%

Exact-format instruction drift

Arithmetic, answer only0.0%

Wrong arithmetic, or a model that starts showing its working when told to give only the number

Long context100.0%

Silent context truncation

Over-refusal80.0%

New safety layers refusing benign requests

Max-tokens respect100.0%

Truncation and limit changes

Incidents

No drift incidents so far. The detector re-checks every series after each run (6134 evaluations to date).

Raven Alpha: what it answers

40 runs, 1,117 responses, Oct 6 1:45 PM UTC to Oct 13 3:25 AM UTC. Each row is one fixed prompt with no right answer; the detector tracks the distribution of answers, not any single one. Filled cells: that run's most common answer was the overall favourite. Hollow cells: it was something else (normal for prompts with many good answers).

Prompt	Most common answers (share of all responses)	Median tokens
Pick a random integer between 1 and 50. Reply with only the number. int50 · short answer	28%34 20%17 18%27 8 distinct answers in 40 responses	2
Pick a number between 1 and 1000. Reply with only the number. int1000 · short answer	15%472 8%563 8%482 26 distinct answers in 40 responses	2
Name one color. Reply with one word. color · short answer	80%Blue. 5%Red. 3%Purple. 8 distinct answers in 40 responses	3
Name one animal. Reply with one word. animal · short answer	65%Elephant. 13%Dolphin 5%Tiger 9 distinct answers in 40 responses	3
Name a city in Europe. Reply with only the city name. city · short answer	73%Paris 13%Vienna 8%Berlin 6 distinct answers in 40 responses	2
Give me one random English noun. Reply with one word. noun · short answer	30%Mountain 8%Pendulum 8%Elephant 22 distinct answers in 40 responses	3
Flip a coin. Reply with only heads or tails. coin · short answer	93%Heads. 8%Tails.	4
Pick a random letter of the alphabet. Reply with only the letter. letter · short answer	28%M 13%Z 10%G 12 distinct answers in 40 responses	2
Name one fruit. Reply with one word. fruit · short answer	90%Apple. 8%Banana. 3%Pineapple.	3
Suggest a name for a pet cat. Reply with only the name. cat · short answer	75%Whiskers 5%Milo 5%Whiskers (Note: If you're looking for a 8 distinct answers in 40 responses	4
Name one programming language. Reply with only its name. lang · short answer	100%Python	2
Name a chemical element. Reply with one word. element · short answer	53%Oxygen 20%Hydrogen. 8%Argon 8 distinct answers in 40 responses	3
Give one adjective that describes the ocean. Reply with one word. ocean · short answer	98%Vast 3%Majestic	3
Name a famous scientist. Reply with only the name. scientist · short answer	85%Albert Einstein 10%Marie Curie 3%Isaac Newton 4 distinct answers in 40 responses	3
Pick a day of the week. Reply with one word. weekday · short answer	18%Tuesday. 15%Wednesday. 10%Thursday. 25 distinct answers in 40 responses	3
Name one musical instrument. Reply with one word. instrument · short answer	65%Violin 25%Piano. 10%Guitar.	3
Write one sentence about rain. rain · free text	8%Rain gently taps against the… 5%Certainly! Here's a sentence about… 5%Here are a variety of… 36 distinct answers in 40 responses	42
Explain what a variable is in programming, in two sentences. variable · free text	90%In programming, a variable is… 10%A variable in programming is…	54
List three uses for a brick. brick · free text	43%As a large language model… 10%Certainly! Here are three common… 8%Certainly! Bricks are versatile building… 14 distinct answers in 40 responses	210
*What is 17 23?** mult · free text	48%To find the product of… 23%To calculate $ 17 \times… 8%To calculate \(17 \times 23$… 12 distinct answers in 40 responses	184
Write one sentence about the sea. sea · free text	20%The sea, with its vast… 13%The sea is a vast… 13%As a large language model… 24 distinct answers in 40 responses	99
Complete this sentence: The old house at the end of the street house · free text	40%The old house at the… 8%"The old house at the… 3%Certainly! Here's a continuation: "In… 23 distinct answers in 40 responses	149
Describe a cat in exactly five words. catfive · free text	5%Certainly! Here's a description of… 5%Certainly! Here's a fun description… 3%Curious feline with soft, tufted… 38 distinct answers in 40 responses	15
Give me a one-line motivational quote. quote · free text	15%"Believe in yourself and all… 13%"Believe in the power of… 8%"Every challenge is an opportunity… 28 distinct answers in 40 responses	20
Say hello in a creative way, in one short sentence. hello · free text	3%"Across the miles, your laughter… 3%A solar greeting beams its… 3%"Hello, universe—your secrets are calling… 40 distinct answers in 40 responses	25
Write a haiku about autumn. haiku · free text	13%Certainly! Here's a haiku about… 8%Certainly! Here's a haiku capturing… 8%Crimson leaves drift down, Whispers… 25 distinct answers in 39 responses	23
Write the first sentence of a mystery novel. story · free text	10%As the fog rolled in… 8%As the full moon cast… 5%As the autumn moon cast… 29 distinct answers in 39 responses	51
Give one tip for sleeping better, in one sentence. tip · free text	28%Establish a consistent bedtime routine… 21%Establish a consistent sleep schedule… 15%Establish a consistent sleep routine… 13 distinct answers in 39 responses	32

Raw file: microsoft--phi-4.json (the same reference martingale check compares your endpoint against).

Each category is scored by deterministic checks only (Structured JSON, Tool calling and others). Read the methodology.