Stockertim Dictionary · Artificial intelligence
Confidence score
Also called confidence level (model), model confidence, confidence scores
In short
A confidence score is a number a model attaches to a result to show how sure it is, often used to send low-confidence items to a person for review.
- Intermediate
- Ratio or metric
- Checked
In detail
A confidence score is the model's own estimate of how likely its answer is to be right. An invoice coding tool might suggest an account with a score of 0.9, meaning it rates that code far above the alternatives. Scores usually run from 0 to 1 or from 0 to 100.
Many classifiers compute a raw score for every possible label and then normalise the scores so they add up to 1; the confidence score is the share belonging to the chosen label. A threshold then decides routing: items at or above the threshold are processed automatically, and items below it go to an exception queue for a person.
Confidence scores matter because they decide how much work a person sees. A threshold set too low lets errors through; one set too high sends almost everything to review. Scores are also only as good as the model's calibration: a well calibrated model that gives 0.9 is right about 90% of the time on such items, but many models are overconfident, especially on unusual items.
Firms therefore test calibration on past data, sample items above the threshold and adjust the threshold with care.
Formula
Confidence score = score for the chosen label / sum of scores for all labels
- Confidence score
- the model's estimate, from 0 to 1, that its chosen result is right
- score for the chosen label
- the raw score the model gives the result it picked
- sum of scores for all labels
- the raw scores of every possible result added together
Worked example
Kite & Co Accountants' coding tool processes 1,200 invoices in a month with a threshold of 0.85.
- For one invoice the label scores are 4.5, 0.3 and 0.2, so its confidence score is 4.5 / (4.5 + 0.3 + 0.2) = 0.9
- Invoices at or above the threshold and coded automatically: 1,020 / 1,200 = 85%
- Invoices sent to review: 180 / 1,200 = 15%
- A sample of 50 auto-coded invoices finds 49 correct: 49 / 50 = 98%
Most invoices are coded automatically, 180 go to a person, and the sample supports the threshold while showing it is not error-free.
In practice
Bookkeeping, OCR, matching and screening tools show confidence scores; finance teams set thresholds, track the share routed to review and sample items above the threshold.
Watch out
- A confidence score is the model's view of itself, not an independent check of accuracy.
- Generative AI assistants often do not give a reliable confidence score; fluent wording is not one.
Learn it properly
Official sources
- AI Risk Management Framework · National Institute of Standards and Technology
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile · National Institute of Standards and Technology
Facts last checked on 3 November 2026.
Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.