ChatGPT Project Lily: do humans read your ChatGPT chats?

The Morse teamPublished October 19, 20269 min read

Project Lily is the reported internal codename for an OpenAI program in which outside contractors read real ChatGPT prompts, sometimes whole conversations, and grade ChatGPT's replies so the model improves. 404 Media reported it on September 14, 2026, two California users sued over it two days later, and as of October 19, 2026 none of the lawsuit's claims has been tested in court. OpenAI's own help center does say that "trusted service providers" may access your content "to improve model performance (unless you have opted out)."

For this page we read the 47-page complaint, the court docket, and OpenAI's privacy policy, terms, and help articles on October 19, 2026. Each claim below is marked as reported (by 404 Media), alleged (in the complaint), or stated (by OpenAI on its own pages).

What is Project Lily?

It's a human review program, according to 404 Media's investigation, which was based on leaked internal documents and real prompts the outlet saw. Joseph Cox reported that OpenAI is hiring "hundreds of contractors" to read "a massive stream of real users' ChatGPT prompts," and that what they review "can include whole conversations between users and the chatbot."

The free part of the story adds three details:

• Usernames are hidden. The contractors "don't see ChatGPT usernames."
• Filtering isn't perfect. OpenAI says it tries to remove personal information before prompts reach reviewers, "but the company acknowledged sensitive details can still get through."
• OpenAI isn't alone. "Anthropic confirmed to 404 Media it is also using human review to improve its models."

A person who works with the prompts, asked whether users know humans read their chats, told 404 Media: "No. I don't think they would imagine some contractor somewhere [...] is analyzing the conversations."

What do the reviewers allegedly see?

The complaint in Vredenburgh v. OpenAI OpCo gives the fullest public account, citing 404 Media and OpenAI's own reviewer guides. These are allegations, not findings. According to paragraph 90, a reviewer:

1. picks a task from a dashboard,
2. is shown a real user's prompt, which "can comprise entire conversations,"
3. writes a short summary of what the user seems to want,
4. reads four ChatGPT replies and highlights at least three passages as aligned or misaligned with how the model should behave,
5. scores each reply from one ("unacceptable, unusable") to seven ("would be hard to meaningfully improve"), and
6. writes a rationale, from a few sentences to a paragraph.

The complaint adds that reviewers are hired through a staffing firm, Crossing Hurdles, that internal guidance tells them to mark down replies that are sycophantic, anthropomorphize the model, misuse emoji, or end with "engagement bait," and that the guide tells reviewers to escalate tasks containing personal information. Paragraph 95 says the reviewer's screen "may also display a summary of the user's prior interactions with ChatGPT," which can show what the user has used it for "and, in some cases, where in the world the user lives." Decrypt, citing 404 Media, calls this a "user memories summary."

The filter OpenAI runs first is a version of OpenAI Privacy Filter, an open-weight model the company released on April 22, 2026 and says it uses, fine-tuned, "in our own privacy-preserving workflows." OpenAI's announcement says: "Like all models, Privacy Filter can make mistakes. It can miss uncommon identifiers or ambiguous private references." A name can be masked while the story around it still points to one person.

Does OpenAI say humans read ChatGPT chats?

Yes, but only in one help article. OpenAI's page How OpenAI handles data in consumer services has a section headed "Authorized human access to content," which says:

A limited number of authorized OpenAI personnel, as well as trusted service providers that are subject to confidentiality and security obligations, may access user content only as needed for these reasons: (1) investigating abuse or a security incident; (2) to provide support to you [...]; (3) to handle legal matters; or (4) to improve model performance (unless you have opted out).

The same page says OpenAI "may also send select portions of content to trusted service providers [...] for data annotation and safety purposes," and, right after the human-access paragraph: "Please do not enter sensitive information that you would not want reviewed or used." When the complaint quoted that human-access answer "in full," as it read on September 15, the sentence about sensitive information wasn't part of it.

The documents you agree to say less. As of October 19, 2026:

• OpenAI's privacy policy, still dated September 10, 2026, lists the vendors that receive personal data (hosting, customer service, cloud, content delivery, support and safety, email, web analytics, payments, search and shopping, IT, and age verification). No annotation or human-evaluation firm is named.
• OpenAI's help page on how your data is used to improve model performance, updated October 18, says OpenAI takes "steps to reduce the amount of personal information in training datasets." The words "human" and "review" don't appear on it.
• The transparency and content moderation page says OpenAI uses "a combination of automated technologies and human review to monitor activity" for content that may break the law or its policies.

The lawsuit is built on that gap. The complaint says OpenAI disclosed human review for flagged content and for business admins in the documents users agree to, but put human review for model improvement only in a help article. Paragraph 84 says that when 404 Media asked where users had been told, OpenAI couldn't point to it until after publication.

OpenAI has also reorganized that article since the suit was filed. The complaint calls article 7039943 the "Data Usage for Consumer Services FAQ" and says the human-access answer was the ninth of eleven questions. Today the same article number is titled "How OpenAI handles data in consumer services," shows an update in late September 2026, and gives human access its own heading.

What does the Project Lily lawsuit claim?

Alexandra Vredenburgh of Carlsbad, a paying subscriber, and Paul Coutu of San Francisco filed a proposed class action on September 16, 2026 in the Northern District of California, represented by Almeida Law Group. The complaint makes eight claims: California's Unfair Competition Law, Consumers Legal Remedies Act, and False Advertising Law, fraudulent omission, intrusion upon seclusion, invasion of privacy, the California Consumer Privacy Act, and unjust enrichment.

It proposes a nationwide class of US ChatGPT users, a California subclass, and subclasses for people who paid, and it leaves out Enterprise, Business, Team, Edu, and API customers. Among the orders it asks for:

• no chat sent to an outside reviewer without "affirmative, express consent" through a standalone disclosure,
• "Improve the model for everyone" set to off by default for every new consumer account,
• a notice in the ChatGPT window, where you type, that third-party reviewers may read and score conversations, and
• deletion of the reviewers' summaries, scores, and rationales, and deletion or retraining of any reward or preference model built from them.

The plaintiffs also want damages and restitution. Subscribers, the complaint says, "paid a price premium for a service whose privacy characteristics were materially misdescribed."

The docket so far: a dated timeline

We built this from the CourtListener docket for case 3:26-cv-10527, read October 19, 2026, plus two OpenAI dates the complaint relies on.

The stipulation and the order aren't free to read on CourtListener, so we can't tell you what they changed. The docket shows no answer, motion to dismiss, or ruling as of October 19. And Decrypt's report says OpenAI was "served on September 2," but the proof of service on the docket says September 21.

What could happen next?

OpenAI's Terms of Use, effective January 1, 2026, send disputes to "final and binding arbitration" and say "Class arbitrations, class actions, and representative actions are prohibited," unless you opted out within 30 days. The same terms leave it to the courts in San Francisco to decide whether that class action waiver holds and to hear requests for public injunctive relief, which this complaint makes. As of October 19, 2026, nothing on the docket shows OpenAI has raised arbitration. This isn't legal advice.

Does turning off "Improve the model for everyone" stop human review?

It stops the model-improvement kind, by OpenAI's own wording: access for that purpose happens "unless you have opted out." It doesn't touch the other three reasons (abuse or security, support, and legal matters). OpenAI's data controls page puts the switch under Settings, then Data controls, and says it covers "new conversations" only. If you rate a reply with a thumbs up or down after opting out, "the entire conversation associated with that feedback may be used to train OpenAI models."

A temporary chat is not used to improve OpenAI's models while it stays temporary, though "OpenAI may keep a copy of a temporary chat for up to 30 days for safety purposes." For every step and exception, see our guide to whether ChatGPT trains on your data.

What can you do now?

• Switch off "Improve the model for everyone" before your next personal conversation, not after it.
• Don't rate replies in chats you'd rather no one read.
• Use a temporary chat, which also keeps the conversation out of memory.
• Clear memories you wouldn't want summarized. A memory summary is what can allegedly hint at where you live.
• Write as if a stranger might read it. De-identification removes usernames, not the details of your story.

How Morse keeps your chats out of a review queue

A program like Project Lily needs a pool of real conversations to draw from, and on ChatGPT that pool is the history OpenAI keeps on its servers, tied to accounts with memories attached. Morse doesn't build that pool. Our servers don't keep your messages or log what you write, and your chat history is stored only in your browser, encrypted, so we have no archive to sample, annotate, or hand to contractors. Morse doesn't train AI on your chats, so there's no improvement program to feed and no switch to find.

Who else sees a message depends on the mode you pick for each chat:

• E2EE, for anything you'd never want a reviewer to read. Your browser checks the secure enclave's cryptographic proof before anything leaves your device, then encrypts the message so only that enclave can open it. Morse's servers relay ciphertext they can't read, and the reply is decrypted only in your browser. The guarantee rests on the enclave hardware and attestation working as designed. Here's how end-to-end encrypted AI chat works.
• Private, the default, for chats no host should keep. Open-weight models like DeepSeek, Kimi, and GLM, plus xAI's Grok, run on hosts that agreed to keep nothing. Where a provider confirms zero retention on each reply, Morse checks it and refuses a reply that doesn't.
• Anonymous, to use GPT, Claude, or Gemini without the provider knowing it's you. Our proxy removes your account, email, and IP address and sends the message under Morse's own account. The provider still sees the message and may keep it for a limited time to check for abuse, under its own policy, so leave names and places out.

Because your history lives on your device, deleting a chat removes it for good. Morse can't restore it either, so save what you need first. And since we don't store chats, we usually have very little to give authorities, as our privacy policy explains.

To start a chat in E2EE, sign up: end-to-end encrypted chats come with Pro, at $19 a month.