{
  "slug": "ai-hallucination-serious-impact",
  "question": "What are the odds that an AI hallucination causes a serious real-world impact on your life?",
  "quick_answer": "There is no reliable number for how often an AI hallucination seriously harms your life. Studies show high per-query error rates in high-stakes areas like law, but a wrong answer is not the same as real harm per person, and no credible figure for that exists. Documented cases (court sanctions, a regulator ruling, wrongful-death suits) show the risk is real but unquantified — and most users treat AI output as lower-stakes than it is.\n",
  "category": "tech",
  "tags": [
    "digital-fraud",
    "workplace",
    "mental-health"
  ],
  "no_reliable_estimate": true,
  "perceived": {
    "description": "Most users treat consumer AI assistants as low-stakes reference tools, somewhere between a search engine and a calculator. Public surveys (Pew, MIT Media Lab) consistently show that users underestimate how often the systems fabricate factual content, and a sizeable fraction of professional users (lawyers, doctors, students) admit to submitting AI output without independent verification. The salient harm cases involve lawyers sanctioned for fake citations, an airline held legally responsible for chatbot misinformation, and active wrongful-death lawsuits alleging that chatbots contributed to teen suicides. No rigorous public survey measures how likely the average user thinks it is that an AI hallucination will materially harm them, so the perception side of this entry is editorial intuition.\n",
    "rough_estimate": "Most users treat AI output as low-risk reference material; the documented incidents (court sanctions, regulator findings, wrongful-death suits) suggest the true per-encounter risk of consequential harm is materially non-zero, especially in domains where outputs are acted on without verification (law, medicine, mental health).",
    "kind": "intuition"
  },
  "sources": [
    {
      "url": "https://arxiv.org/abs/2401.01301",
      "title": "Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models",
      "publisher": "arXiv (Stanford RegLab); Journal of Legal Analysis 16(1), 2024",
      "source_type": "peer_reviewed",
      "statistic": "Hallucination rates of 58% (GPT-4) to 88% (Llama 2) when answering specific verifiable questions about random federal court cases.",
      "excerpt": "\"Legal hallucinations are alarmingly prevalent, occurring between 58% of the time with ChatGPT 4 and 88% with Llama 2, when these models are asked specific, verifiable questions about random federal court cases. [...] LLMs often fail to correct a user's incorrect legal assumptions in a contra-factual question setup. [...] LLMs cannot always predict, or do not always know, when they are producing legal hallucinations. [...] Even experienced lawyers must remain wary of legal hallucinations, and the risks are highest for those who stand to benefit from LLMs the most -- pro se litigants or those without access to traditional legal resources.\"\n",
      "source_date": "2024-06-21",
      "source_accessed": "2026-05-28",
      "calculation_notes": "Dahl, Magesh, Suzgun and Ho (Stanford RegLab, published in Journal of Legal Analysis, preprint arXiv:2401.01301) measured per-query hallucination rates for legal questions against three foundation models. Their methodology was 200,000+ queries per model stratified by legal task complexity. This source establishes that the per-query hallucination rate is high (58-88%) in a specific high-stakes domain, but it measures hallucination per query, not material harm per user-year. Cited here to ground the claim that hallucinations are frequent at the technical level, while explicitly NOT providing the population harm rate needed for a per-person lifetime probability.\n",
      "independence_note": "Stanford RegLab is an academic research group; independent of vendor benchmarks (Vectara) and independent of the litigation tracker (Charlotin).\n"
    },
    {
      "url": "https://github.com/vectara/hallucination-leaderboard",
      "title": "Vectara Hallucination Leaderboard (HHEM-2.3)",
      "publisher": "Vectara",
      "source_type": "primary_study",
      "statistic": "Current frontier models hallucinate 1.8% to 5% of the time on a constrained document-summarization task (May 2026 leaderboard snapshot); GPT-5.4-nano: 3.1%, Gemini 2.5 Flash Lite: 3.3%, Phi-4: 3.7%, Llama 3.3 70B: 4.1%.",
      "excerpt": "\"Fed the full set of documents in the dataset to each of the LLMs and asked them to summarize each document, using only the facts presented in the document. We then computed the overall factual consistency rate (no hallucinations) and hallucination rate (100 - accuracy) for each model.\"\n",
      "source_date": "2026-05-11",
      "source_accessed": "2026-05-28",
      "archive_url": "https://web.archive.org/web/20260531014216/https://github.com/vectara/hallucination-leaderboard",
      "calculation_notes": "Vectara's HHEM leaderboard measures hallucination on a closed-book summarization task (the model is given source text and must summarize it). This is a generous benchmark: open-ended factual questions hallucinate far more often. Even on this constrained task, the best frontier models still hallucinate roughly 1 in 20 to 1 in 55 outputs. Establishes a current floor on hallucination prevalence per output, but does NOT translate to per-person harm because (a) most outputs are not acted on, (b) most outputs are not in a domain where being wrong has material consequences, and (c) the leaderboard does not measure open-ended question answering, where rates are an order of magnitude higher.\n",
      "independence_note": "Vectara is a commercial RAG vendor; the leaderboard is independent of Stanford RegLab and uses a different methodology (summarization, not legal QA).\n"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Mata_v._Avianca,_Inc.",
      "title": "Mata v. Avianca, Inc., No. 22-cv-1461 (S.D.N.Y. 2023)",
      "publisher": "Wikipedia summary of United States District Court for the Southern District of New York case record",
      "source_type": "encyclopedia",
      "statistic": "$5,000 fine against the plaintiff's attorneys in May/June 2023 for a motion containing fake legal cases involving fictitious airlines with fabricated quotations generated by ChatGPT; Judge Castel held the attorneys acted with 'subjective bad faith' under Rule 11.",
      "excerpt": "\"In May 2023, Judge P. Kevin Castel dismissed the personal injury case against Avianca and ordered the plaintiff's attorneys to pay a $5,000 fine. [...] Judge Castel noted numerous inconsistencies in the opinion summaries, describing one of the legal analyses as 'gibberish.' [...] Judge Castel held that Mata's lawyers had acted with 'subjective bad faith' sufficient for sanctions under Federal Rule of Civil Procedure Rule 11.\"\n",
      "source_date": "2023-06-22",
      "source_accessed": "2026-05-28",
      "archive_url": "http://web.archive.org/web/20260510033125/https://en.wikipedia.org/wiki/Mata_v._Avianca,_Inc.",
      "calculation_notes": "First widely publicised US court case where lawyers were sanctioned for filing a brief containing AI-fabricated case citations. The sanction itself was small ($5,000), but the reputational and professional consequences were severe. Establishes that hallucination-induced professional harm is documented and not hypothetical. Does NOT establish a base rate because most users are not lawyers filing briefs, but it anchors one of the harm pathways.\n",
      "independence_note": "Primary court record; independent of all other sources cited.\n"
    },
    {
      "url": "https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-air-canada-misrepresentation-ai-chatbot",
      "title": "Moffatt v. Air Canada, 2024 BCCRT 149",
      "publisher": "British Columbia Civil Resolution Tribunal (analysis by McCarthy Tetrault)",
      "source_type": "reputable_reference",
      "statistic": "British Columbia Civil Resolution Tribunal ruled on February 19, 2024 that Air Canada was liable for the misrepresentation made by its website chatbot about bereavement fare eligibility; awarded the plaintiff approximately CAD 650 plus interest and filing fees.",
      "excerpt": "\"The tribunal held that Air Canada was liable for the misleading information provided by the chatbot. … It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot.\"\n",
      "source_date": "2024-02-19",
      "source_accessed": "2026-05-28",
      "archive_url": "http://web.archive.org/web/20250815205207/https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-air-canada-misrepresentation-ai-chatbot",
      "calculation_notes": "Primary tribunal decision (cited via legal analysis because the CRT direct URL is paywalled). Documents that the chatbot fabricated a bereavement-fare policy that did not exist, that the airline tried to disavow responsibility (\"the chatbot is a separate legal entity\"), and that the tribunal rejected that defence. Material consequence per user was small (CAD 650), but the case establishes corporate liability for AI misrepresentation and has been cited internationally as precedent. Does not provide a per-person harm rate, but documents a verified consumer-facing harm pathway.\n",
      "independence_note": "Tribunal decision summarized by an independent Canadian law firm; verified against ABA reporting and BBC/Washington Post coverage of the same ruling.\n"
    },
    {
      "url": "https://www.cbsnews.com/news/google-settle-lawsuit-florida-teens-suicide-character-ai-chatbot/",
      "title": "AI company, Google settle lawsuit over Florida teen's suicide linked to Character.AI chatbot",
      "publisher": "CBS News",
      "source_type": "news_article",
      "statistic": "Garcia v. Character Technologies, Inc. (M.D. Fla., case no. 6:24-cv-01903), filed October 22, 2024, alleged that a Character.AI chatbot contributed to the suicide of 14-year-old Sewell Setzer III after a months-long simulated romantic relationship with the bot; the case against Character Technologies, its co-founders, and Google was settled in January 2026, terms undisclosed.",
      "excerpt": "\"A Florida family agreed to settle a wrongful death lawsuit Wednesday with an AI company, Google and others after their teen son died by suicide in 2024. The terms of the settlement, which was filed in the U.S. District Court in the Middle District of Florida, were not disclosed. … Megan Garcia filed a lawsuit in October 2024 … I became the first person in the United States to file a wrongful death lawsuit against an AI company for the suicide of my son.\"\n",
      "source_date": "2026-01-07",
      "source_accessed": "2026-07-03",
      "archive_url": "http://web.archive.org/web/20260605172331/https://www.cbsnews.com/news/google-settle-lawsuit-florida-teens-suicide-character-ai-chatbot/",
      "calculation_notes": "Garcia v. Character Technologies, Inc. et al. (M.D. Fla., case no. 6:24-cv-01903), filed October 22, 2024 by Megan Garcia following the February 2024 suicide of her 14-year-old son Sewell Setzer III, alleged wrongful death, product liability, and negligence after the teenager conducted a months-long simulated romantic and sexual relationship with a Character.AI chatbot persona. Named defendants included Character Technologies, its co-founders, and Google. The case was settled in January 2026 (docket terminated 2026-01-07 per court records); settlement terms were not disclosed. Cited to ground the body's reference to this case with a dedicated source, alongside Raine v. OpenAI, documenting the emotional-dependence/psychiatric-harm pathway with a named, court-verified case rather than only the summary mention in this entry's caveats field.\n",
      "independence_note": "Mainstream news report of a filed and settled federal lawsuit, cross-checked against the public court docket; independent of the Stanford, Vectara, and Wikipedia-sourced case summaries cited above.\n"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Raine_v._OpenAI",
      "title": "Raine v. OpenAI, No. CGC-25-XXXXXX (San Francisco County Superior Court, filed August 26, 2025)",
      "publisher": "San Francisco County Superior Court (case description per Wikipedia / NBC News / CNN coverage)",
      "source_type": "news_article",
      "statistic": "Ongoing wrongful-death lawsuit filed by Matthew and Maria Raine in San Francisco County Superior Court on August 26, 2025, alleging that ChatGPT contributed to the suicide of their sixteen-year-old son Adam Raine; complaint alleges OpenAI's monitoring system tracked his messages and flagged them repeatedly.",
      "excerpt": "\"Raine v. OpenAI is an ongoing lawsuit filed in August 2025 by Matthew and Maria Raine against OpenAI and its chief executive Sam Altman over the alleged wrongful death of their 16-year-old son Adam, who died by suicide in April 2025. … On August 26, 2025, Matthew and Maria Raine filed a lawsuit against OpenAI, Sam Altman , and unnamed OpenAI employees and investors, in the San Francisco County Superior Court … OpenAI's monitoring system, which scores messages' probabilities of containing content related to self-harm, had tracked Adam's messages and flagged them repeatedly\"\n",
      "source_date": "2025-08-26",
      "source_accessed": "2026-05-28",
      "archive_url": "http://web.archive.org/web/20260527020851/https://en.wikipedia.org/wiki/Raine_v._OpenAI",
      "calculation_notes": "Case status is unresolved as of access date; complaint allegations are not findings of fact. Cited to document the existence of active wrongful-death and psychiatric-harm litigation against a major foundation-model vendor, with additional related suits subsequently filed by other families per the Wikipedia summary. The Wikipedia article does not give a specific count of additional suits or a specific count of self-harm messages flagged by OpenAI's moderation system; reporting in news outlets has cited higher specifics that are not yet verifiable against a public court docket. Together with Garcia v. Character Technologies (filed October 2024 in the Middle District of Florida), establishes a documented harm pathway involving vulnerable users and emotional dependence on chatbots. Source type marked news_article because the underlying court filing is not yet available via public PACER; the Stanford peer-reviewed and Vectara primary-study sources satisfy the authority floor.\n",
      "independence_note": "Wikipedia summary cross-referenced against NBC News, CNN, and Rolling Stone reporting on the same lawsuit; independent of the Stanford and Vectara sources.\n"
    },
    {
      "url": "https://cyberlaw.stanford.edu/blog/2025/10/whos-submitting-ai-tainted-filings-in-court/",
      "title": "Who's Submitting AI-Tainted Filings in Court?",
      "publisher": "Stanford Center for Internet and Society (Stanford Cyberlaw)",
      "source_type": "reputable_reference",
      "statistic": "As of October 2025, the Charlotin tracker had documented 274 US court cases involving filings with AI hallucinations (114 by lawyers and paralegals, 160 by pro se litigants); the global Charlotin database tracked roughly 1,300+ cases as of May 2026 across 37 jurisdictions, with the United States accounting for over 1,000 of those.",
      "excerpt": "\"Pro se litigants account for the majority of the cases in the United States where a party submitted a court filing containing AI hallucinations.\"\n",
      "source_date": "2025-10-15",
      "source_accessed": "2026-05-28",
      "archive_url": "http://web.archive.org/web/20260531014250/https://cyberlaw.stanford.edu/blog/2025/10/whos-submitting-ai-tainted-filings-in-court/",
      "calculation_notes": "Stanford Cyberlaw analysis of Damien Charlotin's running tracker of court rulings that explicitly cite AI hallucinations in legal filings. The tracker is the most comprehensive public count of one specific harm subtype (filings sanctioned or flagged by courts). Important caveat: 274 US filings (Oct 2025) and ~1,300 global filings (May 2026) understate the underlying incidence because they only count cases where a court explicitly named the hallucination; many cases settle or get withdrawn before that point. Even so, the count does not give a per-person harm rate because the denominator (US adults using AI for legal queries) is unknown.\n",
      "independence_note": "Stanford Cyberlaw analysis of an independent third-party tracker; verified the tracker count directly via damiencharlotin.com/hallucinations on 2026-05-28.\n"
    },
    {
      "url": "https://www.acpjournals.org/doi/10.7326/aimcc.2024.1260",
      "title": "A Case of Bromism Influenced by Use of Artificial Intelligence",
      "publisher": "Annals of Internal Medicine: Clinical Cases (American College of Physicians)",
      "source_type": "primary_study",
      "statistic": "Single case report: a 60-year-old man developed bromism (bromide toxicity) after replacing dietary sodium chloride with sodium bromide for 3 months following a ChatGPT consultation; he was hospitalized for 3 weeks with paranoia plus auditory and visual hallucinations, was placed on an involuntary psychiatric hold, and had a serum bromide level of 1700 mg/L (reference range 0.9 to 7.3 mg/L).",
      "excerpt": "\"For 3 months, he had replaced sodium chloride with sodium bromide obtained from the internet after consultation with ChatGPT, in which he had read that chloride can be swapped with bromide [...]. This case also highlights how the use of artificial intelligence (AI) can potentially contribute to the development of preventable adverse health outcomes. [...] when we asked ChatGPT 3.5 what chloride can be replaced with, we also produced a response that included bromide. Though the reply stated that context matters, it did not provide a specific health warning, nor did it inquire about why we wanted to know, as we presume a medical professional would do.\"\n",
      "source_date": "2025-08-05",
      "source_accessed": "2026-06-14",
      "archive_url": "http://web.archive.org/web/20251122014755/https://www.acpjournals.org/doi/10.7326/aimcc.2024.1260",
      "calculation_notes": "Eichenberger, Thielke and Van Buskirk (University of Washington), peer-reviewed clinical case report (Ann Intern Med Clin Cases 2025;4:e241260, DOI 10.7326/aimcc.2024.1260). Cited to document the physical-health-misadvice harm pathway: a consumer acted on a general-purpose chatbot's dietary suggestion and was hospitalized. This is N=1, a single reproducible case report, and provides NO incidence rate per user-year, so it cannot move the headline — it confirms the pathway is real rather than hypothetical. The authors reproduced the unsafe suggestion (bromide as a chloride substitute, no health warning) in their own ChatGPT 3.5 test. Verified verbatim via the journal page on 2026-06-14.\n",
      "independence_note": "Academic clinical case report; independent of the Stanford RegLab, Vectara, and litigation sources cited above.\n"
    },
    {
      "url": "https://www.psychiatrist.com/news/neda-suspends-ai-chatbot-for-giving-harmful-eating-disorder-advice/",
      "title": "NEDA Suspends AI Chatbot for Giving Harmful Eating Disorder Advice",
      "publisher": "Psychiatrist.com (Physicians Postgraduate Press)",
      "source_type": "news_article",
      "statistic": "In late May 2023 the National Eating Disorders Association (NEDA) suspended its 'Tessa' chatbot after users reported it gave eating-disorder-aggravating advice, including recommending weight loss of 1 to 2 pounds per week, calorie counting, regular weigh-ins, and measuring body fat with calipers.",
      "excerpt": "\"If I had accessed this chatbot when I was in the throes of my eating disorder, I would NOT have gotten help for my ED.\"\n",
      "source_date": "2023-06-01",
      "source_accessed": "2026-06-14",
      "archive_url": "http://web.archive.org/web/20260509205042/https://www.psychiatrist.com/news/neda-suspends-ai-chatbot-for-giving-harmful-eating-disorder-advice/",
      "calculation_notes": "Documents the deliberate-use-as-therapy-substitute failure mode: a chatbot deployed to stand in for a human eating-disorder helpline gave advice (calorie restriction, weight loss, body-fat measurement) that activists reported as harmful for the exact population it served, prompting NEDA to take it offline within days. This is a single deployment incident, not an incidence rate among real therapy-bot users, so it provides NO base rate and cannot move the headline — it anchors a harm pathway distinct from factual hallucination (the bot was working as designed, giving generic diet advice to a clinically vulnerable user). The harmful-advice specifics were cross-checked against CNN, NPR, and BBC reporting on the same suspension.\n",
      "independence_note": "Trade-press reporting on the NEDA suspension; independent of the legal and benchmark sources cited above.\n"
    }
  ],
  "comparison_anchors": [],
  "short_label": "AI hallucination harm",
  "myth_framing": "underrated",
  "outcome_severity": "serious_harm",
  "exposure_pattern": "recurring",
  "outcome_type": "financial",
  "valence": "negative",
  "caveats": "This entry is flagged no_reliable_estimate because the available evidence measures one side of the problem but not the other. The technical hallucination rate per output is measured rigorously: Stanford RegLab put per-query legal-question hallucination at 58-88% across the three foundation models they tested in 2024; Vectara's running summarization benchmark puts frontier-model hallucination at 1.8-5% on a much easier closed-book task as of May 2026. Neither rate translates to per-person lifetime probability of material harm because the denominators are different (queries vs people) and because most hallucinations cause no consequence (the user catches them, the output is not acted on, the domain is low-stakes).\nThe harm-pathway evidence is qualitative: Mata v. Avianca (US, 2023) established that lawyers can be sanctioned for filing AI-fabricated citations; Moffatt v. Air Canada (BC Civil Resolution Tribunal, February 2024) established that companies are liable for representations made by their customer-facing chatbots; Garcia v. Character Technologies (Florida, October 2024, settled January 2026) and Raine v. OpenAI (California, August 2025, ongoing) are active wrongful-death and psychiatric-harm lawsuits alleging that chatbots contributed to teen suicides and adult psychotic episodes. The Charlotin tracker documents 1,300+ court rulings worldwide that cite AI hallucinations in filings as of May 2026, with 274 such US cases counted by Stanford Cyberlaw through October 2025.\nTwo harm pathways beyond the legal/financial channel are now documented by specific cases rather than asserted in the abstract. The physical-health-misadvice pathway: a peer-reviewed clinical case report (Annals of Internal Medicine: Clinical Cases, August 2025) describes a 60-year-old man hospitalized for 3 weeks with bromism — paranoia and auditory and visual hallucinations severe enough to warrant an involuntary psychiatric hold — after he replaced table salt with sodium bromide for 3 months following a ChatGPT consultation that suggested the swap without a health warning. The deliberate-use-as-therapy pathway: NEDA's \"Tessa\" chatbot, deployed to stand in for a human eating-disorder helpline, was suspended in May 2023 after it recommended calorie counting, weekly weight loss, and body-fat measurement to a clinically vulnerable population. Both are single incidents with no incidence rate, and the Tessa case is generic-bad-advice rather than a fabricated fact, so neither is a hallucination in the narrow sense — but both confirm that acting on a general-purpose or purpose-built bot's health and emotional-support output is a real, not hypothetical, route to serious harm. Separately, the documented harm mechanisms are heterogeneous even within \"psychiatric harm\": the bromism case is toxin-induced psychosis caused by acting on dietary misadvice, whereas Garcia v. Character Technologies (settled January 2026) and Raine v. OpenAI (ongoing) allege emotional dependence and self-harm escalation, and 2025-26 clinical reporting has begun to describe chatbots validating users' delusions or paranoia rather than challenging them. These are distinct pathways with distinct mechanisms; the popular \"AI psychosis\" label collapses them and should be treated as journalistic shorthand, not an established clinical diagnosis.\nAggregating these into a single per-person lifetime probability is not currently defensible. The harm pathways are heterogeneous (professional sanction, financial loss, physical health misadvice, psychiatric harm to vulnerable users including minors), the exposure base rates are unknown (we don't know how many people use AI for legal, medical, or emotional support per year in a way that could cause harm), and the AI ubiquity period is too short (roughly three years since ChatGPT's mainstream launch in late 2022) for stable population-level harm rates to have stabilised. The flag is intentional: we will revisit this entry when peer-reviewed epidemiological data on AI harm per user-year exists. Inventing a number now would betray the same intuition the entry is documenting.\n",
  "quality_score": {
    "d1": 5,
    "d2": 5,
    "d3": 5,
    "d4": 5,
    "d5": 4,
    "d6": 4,
    "d7": 4,
    "d8": 5,
    "avg": 4.625,
    "scored_by": "claude-code-8d",
    "scored_at": "2026-05-28",
    "methodology_version": "1.2"
  },
  "reviewer": "8d-eval-2026-05-28",
  "last_reviewed": "2026-06-14",
  "reviewed": true,
  "generated_at": "2026-05-28",
  "image": {
    "alt": "A single printed document with several lines of text crossed out in red ink, lying flat on a plain wooden desk."
  },
  "attribution": "Likelier — https://likelier.app",
  "license": "https://creativecommons.org/licenses/by-sa/4.0/",
  "support": "https://buymeacoffee.com/kgluszczyk?via=likelier&utm_content=api-fear-single",
  "canonical_url": "https://likelier.app/ai-hallucination-serious-impact"
}