Humans Are Reading Copilot Prompts - And They're Horrified

Key Points:

• Human contractors hired to review and improve Microsoft’s Copilot AI chatbot are frequently exposed to lewd, sexually explicit, and disturbing image editing requests, including upskirt photos and sexualized depictions of women and children.
• These contractors evaluate whether AI-generated images meet user prompts, such as altering breast size or skirt length, rather than moderating or filtering offensive content, leading to concerns about the nature of their work and exposure to harmful material.
• Internal documents and contractor discussions reveal repeated exposure to pro-anorexia content, sexualized images of minors, and other unsettling prompts, highlighting the emotional toll on workers who must judge image quality based on subjective intuition.
• Unlike traditional content moderation, these contractors review private user uploads and prompts to improve AI output quality, raising privacy and ethical questions about the use of human reviewers in AI systems.
• Microsoft acknowledged using customer data to enhance products and enforce conduct policies, while previous abuses of Microsoft AI tools for nonconsensual image generation have been publicly reported and addressed.