OpenAI's 'Project Lilly' Exposes Human Review of ChatGPT Conversations
A new report reveals that OpenAI employs hundreds of contractors under 'Project Lilly' to manually read and rate ChatGPT transcripts, including those containing personal details. Reviewers, paid over $50 an hour, assess response quality and flag issues like AI-speak or sycophancy. The practice raises privacy concerns as many users are unaware their chats are reviewed by humans.
Project Lilly's prompt reviewers earn over $50 an hour for what one contractor describes as repetitive work, with guidelines that shift frequently and sometimes contradict themselves. Their primary task involves scoring anonymized ChatGPT exchanges on response quality, penalizing traits like excessive emojis, patronizing tones, or sycophantic language, while also flagging obvious factual errors. However, the project does not conduct deep factual verification, suggesting separate evaluation teams exist for that purpose.
Despite anonymization efforts, OpenAI acknowledged that personal details can slip through, particularly in shorter conversations. Reviewers also receive a "user memories summary" containing context about users' interests, questions, and potentially location data. Notably, this human review process operates independently from safety checks designed to identify users who might harm themselves or others, and many users remain unaware their private exchanges are being read by contractors.
This revelation could significantly erode public trust in AI chatbots, particularly among users who treat these tools as confidants or therapists. Individuals sharing sensitive personal information may feel betrayed, potentially leading to reduced usage or demands for clearer disclosure policies. The practice could also invite regulatory scrutiny regarding data privacy, as many jurisdictions require explicit consent for human access to personal communications. Companies may need to balance model improvement against user expectations of confidentiality.