Stockertim Dictionary · Artificial intelligence
Evaluation rubric
Also called scoring rubric, marking rubric, rubric
In short
An evaluation rubric is a written set of criteria, with a scoring rule for each, used to judge AI outputs consistently across many cases and reviewers.
- Intermediate
- Document
- Checked
In detail
Judging AI outputs by impression gives inconsistent results: one reviewer likes brevity, another detail, and the same reviewer may score differently on a Monday and a Friday. A rubric fixes the standard in writing. It lists the criteria that matter for the task, such as accuracy, completeness, tone and format, and says how each is scored, for example 0 for missing, 1 for partly met and 2 for fully met.
Good rubrics describe what each score looks like, with a short example where possible, so that two reviewers reach the same mark. Critical criteria can be pass or fail: an output with an invented figure fails however well it is written. The rubric is written before testing starts, so it measures the task rather than the outputs that happened to appear.
Rubrics make prompt comparisons fair and repeatable. The same rubric applied to two prompt versions on the same test set shows which one is better and why. Some teams use a second AI model to apply a rubric at scale, but they check its marks against a person's on a sample first.
Worked example
Harbour Foods scores AI-drafted customer replies with a rubric of 4 criteria, each marked 0, 1 or 2.
- Maximum score: 4 x 2 = 8
- One reply scores 2 for accuracy, 2 for completeness, 1 for tone and 1 for format: 2 + 2 + 1 + 1 = 6
- As a share of the maximum: 6 / 8 = 75%
The reply scores 75%, and the rubric shows that tone and format, not facts, need work.
In practice
Teams write a rubric for any AI output used repeatedly, train reviewers on it with examples, record scores in a spreadsheet and use them to compare prompts and models.
Watch out
- A rubric written after seeing outputs tends to reward whatever the model happened to do.
- An AI marker applying a rubric needs its own checks against human scores.
Learn it properly
Official sources
- NIST AI RMF Playbook · National Institute of Standards and Technology
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile · National Institute of Standards and Technology
Facts last checked on 3 November 2026.
Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.