Stockertim Dictionary · Technology · Artificial intelligence
Known-answer test
Also called known-answer check, hand-calculated test, test against a known result
In short
A check that runs code on a small data set whose correct result has been worked out by hand, so any difference between the two exposes a mistake.
- Beginner
- Process
- Checked
In detail
A known-answer test compares what code produces with what it ought to produce. The analyst builds a tiny input, often five to ten rows, works out the right answer independently, by hand or in a calculator, and then runs the code on that input. If the two answers agree the code passes; if not, something in the logic, the data handling or the assumptions is wrong.
The small data set is chosen to exercise the risky parts. It includes the ordinary case, the edges such as the first and last rows or a value exactly on a threshold, a missing value, a duplicate key and anything else the code is supposed to handle. A test that uses only friendly data proves little, because the faults hide in the awkward cases.
The approach suits code drafted by an AI assistant especially well. Generated code often looks plausible and runs without an error, and reading it carefully does not always reveal that a filter is inclusive where it should be exclusive or that blanks are counted as zeros. A known answer turns the question of whether the code is right into a direct comparison.
Once written, the test is worth keeping. Saved as an automated test that runs whenever the code changes, it becomes a regression test that catches a later edit breaking something that used to work. Government guidance on assuring analytical code treats such tests as a normal part of quality assurance, scaled to how much the result matters.
Worked example
A finance assistant at Kestrel Shipping Ltd asks an AI assistant for code that totals overdue invoices, meaning those more than 30 days old.
- Test data: five invoices of 100, 200, 300, 400 and 500, aged 10, 30, 31, 45 and 60 days.
- By hand, only the invoices aged 31, 45 and 60 days are overdue: 300 + 400 + 500 = 1,200
- The generated code returns 1,400, because it counts the invoice aged exactly 30 days.
The test exposes a boundary error in one run; after the condition is corrected the code returns 1,200.
In practice
Analysts keep a small test file beside each important script and rerun it after every change. Reviewers ask for the known answers along with the code, and teams that use AI assistants make a hand-checked example a condition of accepting generated code into a report.
Watch out
- A test built from the code's own output proves nothing; the expected answer must be worked out independently.
- Include edge cases such as boundaries, blanks and duplicates, or the test passes while real data fails.
Learn it properly
Official sources
- Quality assurance of code for analysis and research · Government Analysis Function
- The AQuA Book: guidance on producing quality analysis for government · HM Treasury
Facts last checked on 9 November 2026.
Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.