AI hallucinations are infiltrating expert work-and entering the permanent body of knowledge

Key Points:

• A study led by Columbia University’s Maxim Topaz found over 4,000 fabricated references in nearly 3,000 biomedical papers, with the incidence of fake citations rising sharply since AI tools became widespread in research in 2024.
• The rate of fabricated references in biomedical literature has increased more than 12-fold over three years, reaching one in 277 papers in early 2026, posing risks to the integrity of medical evidence chains that inform clinical guidelines and patient care.
• AI hallucinations, where models generate plausible but false information, affect experts across fields, as demonstrated by cases like Steven Rosenbaum’s book containing AI-generated inaccurate quotes and widespread AI use among physicians, journalists, and legal professionals.
• Most fabricated references appear unintentionally due to AI errors, but the academic publishing system lacks consistent verification processes, with 98.4% of papers containing fake citations remaining unretracted, exacerbating science’s reproducibility crisis.
• Topaz emphasizes that AI is not inherently problematic but calls for integrating verification mechanisms into workflows to prevent unverified AI-generated content from entering the permanent scientific and professional record.