01 · What breaks
Design for the failure modes, not only the happy path.
A quality assistant that errs in the wrong direction does more damage than no assistant at all. These four failure modes are the ones to design out first.
Anomaly treated as an error
A centre whose scores jump may have a strong new facilitator, or a lenient assessor. The assistant cannot tell which. Give each exception a state such as confirmed error, valid outlier or needs field check, and never let unusual become wrong without a person's decision.
Missing values imputed into source
Filling a missing endline score with an average makes the dataset look complete and the learning gain look real. The assistant reports missing values as attrition beside the result and never writes a value into the source. Any estimate belongs in a separate, labelled analysis file.
Rules changing without versioning
If someone tightens the duplicate rule in March, January's exception counts are no longer comparable and a past report cannot be reproduced. Store each rule with a version number, effective date and author, and record which version ran on each dataset.
Teams overwhelmed by low-value alerts
Two hundred alerts a week about optional fields teach staff to ignore the queue, including the one alert that mattered. Rank exceptions by whether they change a reported number or a person's service, cap the daily list per owner and retire rules that keep producing false positives.
02 · Minimum viable model
Store every exception with its rule, its source and its resolution.
MEL back-checks are one of the candidate DozAI missions: high volume, repeated every reporting cycle, with a person deciding each case. A starting check library is below. Each programme adds its own rules and owners.
| Check | Example | Typical owner |
|---|---|---|
| Duplicate person | Same phone number and date of birth at two centres | Centre coordinator |
| Impossible sequence | Joining date earlier than the certification date | Placement officer |
| Out-of-range value | Attendance above 100%, or a score above the maximum | Person who entered the data |
| Tool mismatch | Baseline and endline taken with different assessment versions | MEL lead |
| Status regression | A learner moves from certified back to enrolled with no reason | Programme manager |
| Back-check disagreement | A second assessor's re-check differs widely from the first result | Assessment lead |
03 · Implementation sequence
Move through reviewable gates.
Start with definitions, not checks. Without written indicator definitions the assistant can only find typos, and the costly errors sit in the definitions.
Document definitions and quality rules
For each indicator, write the definition, formula and source fields, then the rules that protect it. A useful test: two people calculating the indicator separately should get the same number. If they do not, fix the definition before adding any check.
Profile current error patterns
Run the draft rules on the last two reporting cycles and count what they find by centre, form and field. Usual culprits are free-text place names, entries rushed in at month-end and records with no clear owner. This baseline shows where checks will pay off.
Run checks without changing source
The assistant reads from a copy or a frozen snapshot and writes only to its exception queue. Where data from several tools is consolidated, run the checks there; data platforms such as Dalgo, stewarded by Project Tech4Dev, are built for that pattern. Submitted assessments stay locked, with edits approved by MEL.
Prioritise consequential exceptions
Put exceptions that would change a funder figure, an eligibility result or a person's next service at the top. Cosmetic issues go into a weekly digest. Back-checks help here: for example, a different assessor re-tests about one record in ten, and only large disagreements are flagged.
Route to accountable owners
Each rule names an owner role, so an exception lands with the person who can open the source and fix it, usually a centre coordinator rather than the MEL team. The owner corrects the source record, sets the exception state and adds a reason. Overdue items go to their manager.
Learn from reviewed false positives
Every valid-outlier or rule-wrong decision is information about the rule. Review them monthly, adjust thresholds as a new rule version and keep the old cases as test examples. Share each centre's error trend with its own team, so field staff see their quality improving.
04 · Evidence
Measure whether the work became clearer.
The aim is fewer repeated errors at source, not a longer exception queue. Review these measures by centre and by rule, alongside the reports in your M&E software.
Confirmed error, corrected at source. Valid outlier, kept with a note. Needs field check, assigned to a visit. Rule wrong, sent to the MEL lead for a new rule version. Duplicate, merged by the owner with both IDs kept. The assistant proposes a state; only the owner sets it.
05 · Responsible automation
Let systems prepare. Keep consequence with people.
Systems can support
- Running versioned rules on a read-only snapshot
- Comparing related records across forms and centres
- Ranking exceptions by their effect on reported figures
- Linking each exception to its source record
People should decide
- Correcting source records
- Accepting a valid outlier
- Changing an indicator rule
- Deciding dataset fitness for reporting
06 · Common questions
Practical questions teams ask.
Questions MEL leads and data managers ask before adding an AI assistant to their quality checks.
Can AI help a nonprofit improve MEL data quality?
Yes, by finding and explaining problems, not by fixing them silently. An assistant can run documented checks, compare related records, spot duplicates and impossible dates, and prepare a ranked queue with links to each source. The data owner confirms each exception and corrects the source. A dataset becomes fit for reporting only when the MEL lead says so.
What checks should a MEL data quality assistant run?
Start with checks that protect reported numbers: duplicate people across centres, impossible date sequences, out-of-range values, assessment tool versions that differ between baseline and endline, and status changes with no reason. Add back-check comparisons where a second assessor re-tests a sample. Each check needs a written rationale, a threshold and a named owner.
Should AI fill in missing data in a monitoring dataset?
No. Writing estimated values into source records makes the dataset look complete and can make results look better than they are. Report missing values openly, for example as attrition beside a learning gain. If an analysis needs an estimate, keep it in a separate, clearly labelled file, with the method written down and the source left untouched.
How do you stop data quality alerts overwhelming field teams?
Rank exceptions by consequence, so anything that changes a funder figure or a person's service comes first and cosmetic issues wait for a weekly digest. Cap the daily list per owner, retire rules with high false-positive rates and show each team its own error trend. Staff act on short lists they have learned to trust.
What audit trail should data quality checks leave?
For each exception: the dataset version, the record link, the rule and its version, the proposed explanation, the owner, the state they chose, any correction made at source and the date. Keep rule versions with effective dates, so any past report can be reproduced with the rules that applied when it was filed.