Skip to content

Stockertim Dictionary · Artificial intelligence

Confidence score

Also called confidence level (model), model confidence, confidence scores

In short

A confidence score is a number a model attaches to a result to show how sure it is, often used to send low-confidence items to a person for review.

  • Intermediate
  • Ratio or metric
  • Checked

In detail

A confidence score is the model's own estimate of how likely its answer is to be right. An invoice coding tool might suggest an account with a score of 0.9, meaning it rates that code far above the alternatives. Scores usually run from 0 to 1 or from 0 to 100.

Many classifiers compute a raw score for every possible label and then normalise the scores so they add up to 1; the confidence score is the share belonging to the chosen label. A threshold then decides routing: items at or above the threshold are processed automatically, and items below it go to an exception queue for a person.

Confidence scores matter because they decide how much work a person sees. A threshold set too low lets errors through; one set too high sends almost everything to review. Scores are also only as good as the model's calibration: a well calibrated model that gives 0.9 is right about 90% of the time on such items, but many models are overconfident, especially on unusual items.

Firms therefore test calibration on past data, sample items above the threshold and adjust the threshold with care.

Formula

Confidence score = score for the chosen label / sum of scores for all labels

Confidence score
the model's estimate, from 0 to 1, that its chosen result is right
score for the chosen label
the raw score the model gives the result it picked
sum of scores for all labels
the raw scores of every possible result added together

Worked example

Kite & Co Accountants' coding tool processes 1,200 invoices in a month with a threshold of 0.85.

  1. For one invoice the label scores are 4.5, 0.3 and 0.2, so its confidence score is 4.5 / (4.5 + 0.3 + 0.2) = 0.9
  2. Invoices at or above the threshold and coded automatically: 1,020 / 1,200 = 85%
  3. Invoices sent to review: 180 / 1,200 = 15%
  4. A sample of 50 auto-coded invoices finds 49 correct: 49 / 50 = 98%

Most invoices are coded automatically, 180 go to a person, and the sample supports the threshold while showing it is not error-free.

In practice

Bookkeeping, OCR, matching and screening tools show confidence scores; finance teams set thresholds, track the share routed to review and sample items above the threshold.

Watch out

  • A confidence score is the model's view of itself, not an independent check of accuracy.
  • Generative AI assistants often do not give a reliable confidence score; fluent wording is not one.

Learn it properly

Official sources

Facts last checked on 3 November 2026.

Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.

English