Stockertim Dictionary · Project management · Technology
Data cleaning
Also called data cleansing, data scrubbing, data preparation
In short
Preparing raw data for analysis by removing duplicates, correcting formats, handling missing values and recording every change made.
- Beginner
- Process
- Checked
In detail
Data cleaning turns raw records into data that can be trusted for analysis. Typical tasks include removing duplicate rows, standardising formats such as dates and postcodes, correcting obvious errors, deciding how to treat missing values, and checking that categories are used consistently. Without it, even simple counts and averages can be badly wrong.
Each decision should be recorded. Excluding records, filling in missing values or merging categories all change the result, and someone reviewing the analysis needs to know what was done and why. Keeping the raw data unchanged and applying cleaning steps in a documented script or log makes the work repeatable and auditable.
Government data quality guidance frames this in terms of dimensions such as completeness, accuracy, consistency, timeliness, uniqueness and validity. Cleaning fixes problems in one extract, but the lasting solution is usually upstream: better forms, validation at entry and clear ownership of the data, so the same errors are not created again.
Worked example
An extract has 1,250 rows. The analyst removes 50 duplicates and sets aside 100 rows with no date.
- Rows after removing duplicates: 1,250 - 50 = 1,200
- Rows used for timing analysis: 1,200 - 100 = 1,100
- Share of the cleaned rows set aside: 100 / 1,200 = 8.3%
The timing analysis uses 1,100 rows, and the report notes that 8.3% lacked a date.
In practice
Analysts typically spend a large share of any data task on cleaning, and data quality frameworks ask teams to report quality alongside results.
Watch out
- Silently deleting awkward records can bias the result.
- Cleaning a copy fixes this report but not the source; raise the issue with the data owner.
Learn it properly
Official sources
- The Government Data Quality Framework · UK Government (Government Data Quality Hub)
- Measures of location (NIST/SEMATECH e-Handbook of Statistical Methods) · National Institute of Standards and Technology
Facts last checked on 3 November 2026.
Education, not advice: the Stockertim Dictionary explains what terms mean. It never recommends buying or selling anything.