Humanity’s Last Exam paper
Primary paper introducing HLE.
← Back to Compute College
Compute College
What is Humanity’s Last Exam?
Learn what Humanity’s Last Exam measures and why frontier academic benchmarks matter for model capability claims and AI compute demand.
Compute & Pricing LessonsLearning path
One concept connected to AI compute market decisions.
5-8 minutesRead time
A practical introduction designed to be completed in one sitting.
HLE / Reasoning / Frontier ModelsTags
Useful for developers, founders, procurement teams, and analysts tracking model-serving economics.
Plain-English definition
Plain-English definition
Humanity’s Last Exam, or HLE, is a difficult multimodal benchmark created to test frontier model capability across expert-level academic subjects with broad coverage.
Why it matters
Why it matters
A frontier capability claim becomes relevant to ComputeTape when it causes buyers to use costly models for research, analysis, coding, or agent workflows that raise inference demand.
- Capability changes matter economically only when they affect deployed workloads or buyer choices.
- Token volume, latency, retries, and throughput determine how a useful result becomes serving cost.
- A ComputeTape reader should connect model evidence to inference demand and required AI compute capacity.
Simple example
Simple example
Progress on a demanding academic test can indicate capability improvement, but it does not state the number of tokens, latency, or dollar cost needed to complete a business task.
- Use the example to compare workload economics, not as a current market quote.
- Record the task type, evaluation or workload conditions, and the cost inputs before comparing results.
- A successful result is valuable only if its latency and cost fit the intended production use.
Example figures are illustrative calculations, not current quoted market prices.
Current example
Primary source
The Humanity’s Last Exam paper introduces HLE as a multimodal benchmark at the frontier of human knowledge with broad subject coverage. Last checked: May 24, 2026.
The page explains the test and avoids unsupported frontier-model comparisons.
Market signal
How to read the market signal
When a model improves on demanding benchmarks, observe whether customers route valuable workloads to it and whether that changes model-serving demand.
- Look for adoption, routing, usage-volume, or capacity signals rather than a headline score alone.
- Compare input tokens, output tokens, latency, tool rounds, retries, and completion quality together.
- Keep sourced capability facts separate from interpretation about future AI compute demand.
Market read: this metric becomes an AI compute signal only when it changes serving volume, effective workload cost, or the capacity buyers require.
Common mistake
Common mistake
Do not equate a hard-test result with universal production readiness or make broad capability claims beyond the cited benchmark.
Practical takeaway
What you can do with this
Use HLE as one frontier evidence source and pair it with workload-specific quality, latency, price, and adoption evidence.
- Buyers: test the metric on tasks close to the workload you will pay to serve.
- Builders: measure tokens, latency, retries, completion rate, and model price on each test run.
- Analysts: require a source and an adoption mechanism before treating a model result as demand evidence.
Decision check: identify the capability measured, the serving cost driver it affects, and the buyer behavior that would make capacity demand change.
Helpful memory trick
Helpful memory trick
Hard test, not a business case by itself.
Compute College
Follow model releases as market signals
Follow model releases as AI compute market signals in the ComputeTape Morning Brief.
Compute College track
Model Costs
Continue this Compute College lesson path
Previous lesson
What is mmlu pro
Continue the Model Costs track.
Next lesson
What is a reasoning benchmark
Continue the Model Costs track.