> ## Content Index
> Fetch the complete content index at: https://www.nanosava.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# Computer Says No
- URL: https://www.nanosava.com/essays/computer-says-no/
- Published: 2026-10-28T12:00:00.000Z
- Updated: 2026-10-28T12:00:00.000Z
- Description: Three countries, three machines, and thousands of reasonable officials who believed the screen over the person in front of them.
- Author: Sava
- Tags: Emergent Stupidity

## The shortfall

In 2005, Seema Misra became the subpostmaster at West Byfleet, in Surrey. Her branch ran on Horizon, the accounting system the Post Office had been rolling out to its branches since about 1999\. Horizon kept telling her that money was missing.

She reported the losses to the Post Office helpline. She suspected staff of stealing and let them go, and the unexplained losses, she later told a jury, kept coming. Family and friends lent her money to make the accounts balance.

On January 14, 2008, the auditors came. The shortfall on the screen became the theft on the charge sheet: £74,609.84\. Her first trial, in May 2009, was aborted when she raised doubts about Horizon. At the second, an expert for her defense set out ways the system might go wrong. The prosecution's expert conceded that he could not exclude errors, but said they could not explain losses on that scale. She was convicted, and on November 11, 2010, she was sentenced to 15 months in prison.

On April 23, 2021, the Court of Appeal quashed her convictions, along with those of 38 other people. The judgment is a calm document, which makes one sentence land harder. The defendants, the court said, "were prosecuted, convicted and sentenced on the basis that the Horizon data must be correct." The Post Office had "treated what was no more than a shortfall shown by an unreliable accounting system as an incontrovertible loss."

Read it again and notice what is not in it. No villain. A basis, some data, and a *must*.

## The question

"Computer says no" entered British English as a joke. It is usually traced to the 2004 sketch show *Little Britain*, in which a bank clerk types a customer's request into her computer, waits, and refuses. We laughed because we all knew that clerk.

This part asks the unfunny version. How does an institution full of decent, capable people come to believe a machine over every person it accuses? And why did the same thing happen, in different machinery, on the other side of the world and across the North Sea?

## Not one machine

In Australia, for about five years from 2015, the government's automated debt-recovery scheme, known everywhere as Robodebt, used income averaging: it spread a year of tax-office income evenly across fortnights, compared that with what people had reported, and raised debts where the two disagreed. It raised about A$1.76 billion in debts against hundreds of thousands of people. The Royal Commission reported in July 2023 that "Robodebt was a crude and cruel mechanism, neither fair nor legal, and it made many people feel like criminals."

In the Netherlands, the tax administration's pursuit of childcare-benefit fraud wrongly branded parents as fraudsters, sometimes over nothing more than an administrative mistake made without any intent to deceive. Amnesty International puts the number falsely accused in the tens of thousands. A parliamentary inquiry titled its December 2020 report *Ongekend onrecht*, "unprecedented injustice," and found that fundamental principles of the rule of law had been violated. On January 15, 2021, the third Rutte cabinet resigned.

The machine was only part of the Dutch story; the inquiry put most of its weight on rigid law and an all-or-nothing approach to enforcement. But a machine was in it. Amnesty found that the risk-classification model used Dutch nationality as a risk factor, and that the civil servants who reviewed flagged files were given no information about why the system had scored them high. In December 2021 the Dutch data protection authority fined the tax administration €2.75 million for discriminatory processing of nationality data.

Three countries, three systems: a ledger, a formula, a score. One pattern: reasonable officials deferred to the system, and the people who doubted it were treated as the problem.

## The three mechanisms, here

[Part I](https://www.nanosava.com/essays/the-stupidity-no-one-chose/) set out three mechanisms that systems run: delegation, insulation, and alignment. Here they explain more than any individual failing does. Delegation leads.

**Delegation: what the law offered.** A system's authority is usually informal: the screen is there, so we read it. In England and Wales the law made it formal. Until 2000, computer evidence in criminal trials was governed by section 69 of the Police and Criminal Evidence Act 1984\. In 1997 the Law Commission recommended repealing it without replacement, and Parliament did. What filled the space, in the government's own description, was a common-law presumption "that a computer was operating correctly" unless there is evidence to the contrary.

A presumption is delegation in its purest form. Trusting the machine becomes the default, doubting it becomes extra work, and the extra work falls on the person with the least access to the machine. In practice, the Court of Appeal found, the Post Office "effectively sought to reverse the burden of proof," proceeding "as if it were for the accused to prove that no such loss had occurred." The machine did not need to be believed by everyone. It only needed to be presumed. Robodebt's formula and the Dutch risk score worked the same way: the system spoke first, and disagreeing took work that agreeing did not.

**Insulation: what the institution provided.** The second mechanism is distance between a decision and the inconvenient fact. Each complaint arrived alone, at a helpline built to handle it alone. One postmaster maintains that she was told by Post Office employees "that she was the only person who was experiencing problems." In internal documents, the Court of Appeal found, a postmaster who blamed a shortfall on the system could be dismissed as "jumping on the Horizon bandwagon." In 2013 a barrister advising the Post Office urged it to build a single hub collating all Horizon-related bugs and complaints, for disclosure. The response of at least some personnel, the court accepted, was "to suggest that information should not be recorded, in the hope that it would therefore not be disclosable."

The knowledge existed. The Inquiry's chair, Sir Wyn Williams, found that some Fujitsu employees knew before rollout that Horizon could show losses that were not real. The Post Office, he wrote, "maintained the fiction that its data was always accurate." In Australia, the Royal Commissioner wrote of "dishonesty and collusion to prevent the Scheme's lack of legal foundation coming to light," and judged the institutional checks and balances, from the Ombudsman to the appeals tribunal, ineffective "in presenting any hindrance to the Scheme's continuance."

**Alignment: what the institution rewarded.** The third mechanism is the reward for moving with the group, and in these records it shows up as a silence. The Court of Appeal had been shown no document in which any employee or official of Post Office Limited (POL) "made any adverse comment upon the incorrect approach," and it concluded that the Post Office's approach to investigation and disclosure "was influenced by what was in the interests of POL, rather than by what the law required." In Australia, the Royal Commissioner wrote of "the lengths to which public servants were prepared to go to oblige ministers on a quest for savings," savings the government could not achieve, the Commission found, without income averaging. Once savings are counted, doubting the formula means doubting the budget.

The Dutch inquiry found the same pull across a whole state: legislation it called *spijkerhard*, nail-hard, passed by cabinet and parliament, and administrative courts that for years contributed substantially to keeping the rigid enforcement in place. Nobody had to be bribed or bullied. Going along paid in quieter currency: a target met, a minister obliged, a file closed.

## Why nobody chose it

Line up the people in the Horizon story and look at each one alone. The helpline adviser follows a script that says the system is sound. The auditor counts the cash and compares it with the screen. The investigator has a shortfall and a contract. The prosecutor has a presumption the law provides. The judge applies the law. Each of them is doing a recognizable job, reasonably.

Then the whole begins to feed on itself. The Court of Appeal named the mechanism: "the longer that approach persisted, the more POL was able to, and did, rely upon its own past abusive conduct by asserting that no previous challenge to Horizon had succeeded." Every conviction became evidence for the next one. The same failures, the court said, "occurred in case after case, year after year."

That is emergence in an institution. Bonhoeffer thought stupidity less a psychological problem than a sociological one, something people are drawn into under the pull of power and the group. No one in the chain decided to ruin anyone; the ruin was a property of the chain.

The pattern existed only in aggregate, and nobody's job was the aggregate. It took the postmasters themselves to build one: the Justice for Subpostmasters Alliance in 2009, and later a group action of about 555 claimants. The machine did not change. The view did.

## More and more

Horizon was ordinary software, with records that could be examined and faults a skilled person could in principle find. It still took about twenty years from rollout to a High Court judgment in 2019 that found serious faults in it, and the Inquiry has since found that the original system "was capable of producing data which was false."

The cost is still arriving. As of August 28, 2026, the government reports about £1,697 million paid in redress to more than 13,500 claimants. The Ministry of Justice's final count found 620 people in England and Wales with at least one conviction quashed by the Post Office (Horizon System) Offences Act 2024\. The Inquiry's first volume, published on July 8, 2025, estimated that approximately 1,000 people were prosecuted and convicted across the UK. It recorded at least 59 people who contemplated suicide, ten of whom attempted it, and said it could not make a definitive finding on whether 13 deaths were linked to Horizon, while not ruling it out. The volumes on how it happened, and who is criticized, are still to come; in July 2026 the chair said he was "still some months away" from completing them.

The presumption still stands. A Ministry of Justice call for evidence on it closed in April 2025; when I checked its page this month, no government response had been published.

Meanwhile, risk-scoring of welfare claimants did not end with the Dutch scandal. In 2024 Amnesty asked Sweden's social insurance agency to drop a fraud risk score that, reporters found, disproportionately flagged women, people with foreign backgrounds, and low earners, and a coalition asked France's Conseil d'État to stop the family-benefits agency's. The EU's AI Act lists systems that judge eligibility for public benefits, score creditworthiness, or serve law enforcement as high-risk, though an amendment in force since July 27, 2026, delayed those obligations to December 2, 2027.

The next generation is likely to be models, and perhaps AI agents that draft decisions, choose which cases a human sees, and act on them. A model's reason for a score is a pattern spread across millions of parameters, not a line of code an expert witness can point to. My argument, not a finding: if Horizon, which could be audited, took two decades to disbelieve, a system nobody can fully explain makes a presumption of correctness more dangerous, not less. The Dutch reviewers who saw a score and no reason are a preview of the human in the loop.

None of this makes the machines the villain. A large randomized study in Jharkhand, India, found that stricter biometric ID checks in a welfare program cut corruption but cost 1.5 to 2 million legitimate beneficiaries their access at some point, harm that appears to have been "driven primarily by decisions about the way the transition was managed." The protocol mattered more than the technology.

If the trend runs for five or ten more years, the danger is not one Horizon. It is thousands of small ones, each individually defensible, each harder to see because no one can read the reason. The compounding is in the presumption: every year a system goes unchallenged, its record becomes the argument for not challenging it. We have watched that loop run once already, and a court described it.

## Noticing it

The Three Checks and the Scale Question, for anyone who signs off on decisions about people, or lives with them:

1. **The delegation check.** Could I explain this decision without the words "the system shows"? If not, whose decision is it?
2. **The insulation check.** Which complaint am I filing as a one-off? If the same complaint were arriving at a hundred desks, would anyone be counting, and would I want to know?
3. **The alignment check.** Would I still trust this output if my colleagues did not? Would I say "I think the machine is wrong" out loud, in the meeting?
4. **The Scale Question.** If everyone in my position deferred to the screen exactly as I am doing, what would the institution become? We no longer have to imagine. We have three answers.

**One practice: the second sentence.** Before you act on a system's verdict about a person, write two sentences. The first says how the system could be wrong in this case. The second names one fact, from outside the system, that supports the verdict. If you cannot write the second sentence, you are not deciding; you are relaying. And if you are on the receiving end, ask for it: "What evidence, apart from your system, supports this?"

## The close

Seema Misra's convictions stood for more than ten years. The clerk in the sketch never learns anything, which is why it is funny. Institutions can learn, but in all three countries the learning came from outside: from the people who were not believed, organizing, and from inquiries with the power to read everything. The cheaper lesson is one we can practice at our own desks. The next time a screen and a person disagree, treat it as a question, not an answer.

**This week's sighting.** Where did you see an institution believe a system over a person this week, in your field? No names and no shaming, just the mechanism. Reply in the comments (free subscribers can comment) or write to hello@nanosava.com. Every sighting is read, and the best ones, anonymous unless you say otherwise, shape later parts.

## Sources

1. Court of Appeal (Criminal Division), *Hamilton & Others v Post Office Limited*, 2021 EWCA Crim 577, judgment of April 23, 2021 (paras 20, 82 to 88, 91, 129 to 137, 198 to 209). [https://caselaw.nationalarchives.gov.uk/ewca/crim/2021/577](https://caselaw.nationalarchives.gov.uk/ewca/crim/2021/577?ref=nanosava.com)
2. Sir Wyn Williams, "Post Office Horizon IT Inquiry: Final Report, Volume 1" (HC 1119), July 8, 2025 (paras 1.3 to 1.12, 3.8 to 3.12, 3.24, 3.174, 3.196). [https://www.postofficehorizoninquiry.org.uk/volume-1-post-office-horizon-it-inquirys-final-report](https://www.postofficehorizoninquiry.org.uk/volume-1-post-office-horizon-it-inquirys-final-report?ref=nanosava.com) (paragraph text also mirrored at [https://postofficeinquiry.dracos.co.uk/report/volume-1/](https://postofficeinquiry.dracos.co.uk/report/volume-1/?ref=nanosava.com))
3. Sir Wyn Williams, "Progress Update from Sir Wyn Williams," Post Office Horizon IT Inquiry, July 8, 2026\. [https://www.postofficehorizoninquiry.org.uk/progress-update-sir-wyn-williams-0](https://www.postofficehorizoninquiry.org.uk/progress-update-sir-wyn-williams-0?ref=nanosava.com)
4. UK Government, "Post Office Horizon financial redress data as of 28 August 2026," GOV.UK, updated September 11, 2026\. [https://www.gov.uk/government/publications/post-office-horizon-financial-redress-and-legal-costs-data-for-2026/post-office-horizon-financial-redress-data-as-of-28-august-2026](https://www.gov.uk/government/publications/post-office-horizon-financial-redress-and-legal-costs-data-for-2026/post-office-horizon-financial-redress-data-as-of-28-august-2026?ref=nanosava.com)
5. Ministry of Justice, "Summary: quashed convictions management information, 24 September 2026," GOV.UK, 2026 (data as of September 15, 2026). [https://www.gov.uk/government/publications/post-office-horizon-system-offences-act-2024-quashed-convictions-management-information/summary-quashed-convictions-management-information-24-september-2026](https://www.gov.uk/government/publications/post-office-horizon-system-offences-act-2024-quashed-convictions-management-information/summary-quashed-convictions-management-information-24-september-2026?ref=nanosava.com)
6. Ministry of Justice, "Use of evidence generated by software in criminal proceedings: call for evidence," GOV.UK, January 21, 2025 (closed April 15, 2025; no outcome published as of October 11, 2026). [https://www.gov.uk/government/calls-for-evidence/use-of-evidence-generated-by-software-in-criminal-proceedings/use-of-evidence-generated-by-software-in-criminal-proceedings-call-for-evidence](https://www.gov.uk/government/calls-for-evidence/use-of-evidence-generated-by-software-in-criminal-proceedings/use-of-evidence-generated-by-software-in-criminal-proceedings-call-for-evidence?ref=nanosava.com)
7. "Computer says no," Wikipedia (origin of the catchphrase in *Little Britain*, 2004). [https://en.wikipedia.org/wiki/Computer\_says\_no](https://en.wikipedia.org/wiki/Computer%5Fsays%5Fno?ref=nanosava.com)
8. Prime Minister of Australia, "Final report of the Royal Commission into the Robodebt Scheme," media statement, July 7, 2023 (quoting the report, p. xxix). [https://pm.gov.au/media/final-report-royal-commission-robodebt-scheme](https://pm.gov.au/media/final-report-royal-commission-robodebt-scheme?ref=nanosava.com)
9. Michelle Grattan, "Robodebt royal commissioner makes multiple referrals for prosecution, condemning scheme as 'crude and cruel'," The Conversation, July 7, 2023 (quoting the commissioner's preface). [https://theconversation.com/robodebt-royal-commissioner-makes-multiple-referrals-for-prosecution-condemning-scheme-as-crude-and-cruel-209318](https://theconversation.com/robodebt-royal-commissioner-makes-multiple-referrals-for-prosecution-condemning-scheme-as-crude-and-cruel-209318?ref=nanosava.com)
10. Tweede Kamer, "Grondbeginselen rechtstaat geschonden," press release of the Parlementaire ondervragingscommissie Kinderopvangtoeslag, December 17, 2020\. [https://www.tweedekamer.nl/nieuws/persberichten/grondbeginselen-rechtstaat-geschonden](https://www.tweedekamer.nl/nieuws/persberichten/grondbeginselen-rechtstaat-geschonden?ref=nanosava.com)
11. NOS, "Commissie: ongekend onrecht in toeslagenaffaire, beginselen rechtsstaat geschonden," December 17, 2020\. [https://nos.nl/artikel/2361021-ondervragingscommissie-kinderopvang-ouders-ongekend-onrecht-aangedaan.html](https://nos.nl/artikel/2361021-ondervragingscommissie-kinderopvang-ouders-ongekend-onrecht-aangedaan.html?ref=nanosava.com)
12. Rijksoverheid, "Kabinet-Rutte III" (resignation of January 15, 2021). [https://www.rijksoverheid.nl/regering/over-de-regering/kabinetten-sinds-1945/kabinet-rutte-iii](https://www.rijksoverheid.nl/regering/over-de-regering/kabinetten-sinds-1945/kabinet-rutte-iii?ref=nanosava.com)
13. Amnesty International, "Dutch childcare benefit scandal an urgent wake-up call to ban racist algorithms" (on the report *Xenophobic Machines*), October 25, 2021\. [https://www.amnesty.org/en/latest/news/2021/10/xenophobic-machines-dutch-child-benefit-scandal/](https://www.amnesty.org/en/latest/news/2021/10/xenophobic-machines-dutch-child-benefit-scandal/?ref=nanosava.com)
14. Pinsent Masons, "Dutch tax authority handed record fine for discriminatory data processing," Out-Law News, December 13, 2021\. [https://www.pinsentmasons.com/out-law/news/dutch-tax-authority-record-fine-discriminatory-data-processing](https://www.pinsentmasons.com/out-law/news/dutch-tax-authority-record-fine-discriminatory-data-processing?ref=nanosava.com)
15. Amnesty International, "Sweden: Authorities must discontinue discriminatory AI systems used by welfare agency," November 27, 2024\. [https://www.amnesty.org/en/latest/news/2024/11/sweden-authorities-must-discontinue-discriminatory-ai-systems-used-by-welfare-agency/](https://www.amnesty.org/en/latest/news/2024/11/sweden-authorities-must-discontinue-discriminatory-ai-systems-used-by-welfare-agency/?ref=nanosava.com)
16. Amnesty International, "France: Discriminatory algorithm used by the social security agency must be stopped," October 16, 2024\. [https://www.amnesty.org/en/latest/news/2024/10/france-discriminatory-algorithm-used-by-the-social-security-agency-must-be-stopped/](https://www.amnesty.org/en/latest/news/2024/10/france-discriminatory-algorithm-used-by-the-social-security-agency-must-be-stopped/?ref=nanosava.com)
17. European Commission, "AI Act" (regulatory framework page, application timeline and AI Omnibus), accessed October 11, 2026\. [https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai](https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai?ref=nanosava.com); Regulation (EU) 2024/1689, Annex III, points 5 and 6\. [https://artificialintelligenceact.eu/annex/3/](https://artificialintelligenceact.eu/annex/3/?ref=nanosava.com)
18. Karthik Muralidharan, Paul Niehaus, and Sandip Sukhtankar, "Identity Verification Standards in Welfare Programs: Experimental Evidence from India," NBER Working Paper 26744, 2020, revised 2021; published in *The Review of Economics and Statistics*, 2025\. [https://www.nber.org/papers/w26744](https://www.nber.org/papers/w26744?ref=nanosava.com)
19. Federal Court of Australia, *Prygodicz v Commonwealth of Australia (No 2)*, 2021 FCA 634, judgment summary, June 11, 2021 (debts of at least A$1.763 billion raised against about 433,000 people). [https://www.judgments.fedcourt.gov.au/judgments/Judgments/fca/single/2021/2021fca0634/summaries/2021fca0634-summary](https://www.judgments.fedcourt.gov.au/judgments/Judgments/fca/single/2021/2021fca0634/summaries/2021fca0634-summary?ref=nanosava.com)