New York's AI chatbot warning, tip by tip: what each app's settings actually cover

The Morse teamPublished November 10, 20269 min read

The New York AI chatbot warning is a consumer alert the state Department of State's Division of Consumer Protection issued on November 9, 2026, telling New Yorkers to keep Social Security numbers, account numbers, health information, legal documents, and passwords out of AI chatbots, and to opt out of having future chats saved or used for training (New York Department of State). The list of what not to share holds up. The opt-out advice covers less than it sounds like it does. As of November 2026, only Gemini has one switch that stops both training and saving, and none of the four training switches (ChatGPT, Gemini, Claude, and Grok) stops staff review for safety or retention for legal reasons.

We read the release, the Stanford study it cites, and each company's own help pages for ChatGPT, Gemini, Claude, Meta AI, Microsoft Copilot, and Grok on November 10, 2026, then checked every tip against the settings those pages describe. The release is careful about browsers. It says private or incognito browsing only limits what stays "on your device," and that your information "may still be transmitted to external parties, including AI platforms."

What did New York's consumer protection division say about AI chatbots?

It said to assume the worst. Quoting a Stanford study, the alert tells readers "to assume that any information given to an AI chatbot will be collected and used for training and could be viewed by a human reviewer or hacked" (New York Department of State).

Secretary of State Walter T. Mosley tied the risk to money, saying "sharing sensitive information can increase risk of identity theft and other financial harms for users." Jenson Jacob, the state's acting chief information officer, gave the list in one sentence: never share "Social Security numbers, driver's licenses, passport numbers, account passwords, PINs, credit card details, health information, or other personal, sensitive information."

The release then gives two tips. The first is a "What Not to Share" list: personal identifiers (it names SSN, name, address, and birth dates), financial information, health information such as diagnoses and lab results, legal documents, and usernames or passwords. The second is "Opt-out of having your future data saved," by "adjusting the privacy and training settings directly within each platform."

Is the Stanford study behind the warning still accurate?

Partly. The study is "User Privacy and Large Language Models," by Jennifer King and four co-authors, first posted on September 5, 2025 (arXiv). It coded the privacy policies of Amazon, Anthropic, Google, Meta, Microsoft, and OpenAI and found that all six "appear to employ their users' chat data to train and improve their models by default, and that some retain this data indefinitely."

Stanford's own write-up adds that "some developers allow humans to review users' chat transcripts for model training purposes" (Stanford HAI, October 15, 2025). The paper names Google and OpenAI as the two whose policies discuss human review for training, and it marks Meta as "Not Specified" on whether there's a way to opt out (arXiv).

The policies it read are older than the alert suggests. Most of the paper's sources are marked "Accessed: 2025-05-23," eighteen months before New York quoted it. Two things have changed since. Microsoft released a new Copilot app on August 18, 2026, and says that in it, prompts, responses, and file contents "aren't used to train foundation models" (Microsoft). Meta now offers Incognito chats in the Meta AI app, where "Only you can see the chat. Meta can't access it" (Meta). Assuming the worst is still safe advice, though training by default no longer describes every app.

Where is the opt-out in each app, and what does it leave out?

Each app's own wording, read November 10, 2026.

Gemini is the only app where the state's two steps, stop training and stop saving, are one switch. In ChatGPT, Claude, and Grok they're two different things, and the second is a separate chat mode you pick each time. Our survey of which AI companies train on your data has the same switches for nine more apps.

Which of New York's tips does a setting back up?

Does opting out of AI training stop chatbots from saving your chats?

Only in Gemini. OpenAI says turning off "Improve the model for everyone" leaves "Regular chats" in your history, and a deleted chat is scheduled for permanent deletion within 30 days, "unless the chat was already de-identified and disassociated from your account or OpenAI must retain it longer for security or legal obligations" (OpenAI). Anthropic says a deleted Claude chat is "Deleted from our back-end storage systems within 30 days," but "we may retain chats and coding sessions as required by law, to resolve disputes, or as necessary to combat violations of our Usage Policy" (Anthropic).

The state's word "future" is accurate. Anthropic says that once you turn its setting off, your chats are left out of future training, but your data "will still be included in model training runs that are already in progress, or in models that have been trained" (Anthropic). Turn the switch off before the sensitive chat, not after it.

Google's setting goes further than the others and still stops short. With Keep Activity off, Google says "Google also does not use your future chats to improve its AI models" unless you send feedback, yet "Your Gemini settings don't control processing of your chats to create anonymized data to improve Google services" (Google).

Can a human reviewer still read your chats after you opt out?

Yes. In ChatGPT, Gemini, Claude, and Grok, turning off training leaves review for safety or legal reasons in place. Google says that "Even if your Keep Activity setting is off or you use temporary chats, Google still uses your chats to respond to you and help protect Google, our users, and the public, including with help from human reviewers" (Google). OpenAI's opt-out likewise covers only review for model improvement (OpenAI).

Google is also the most direct about what reviewers see. Its Gemini privacy hub says: "Please don't enter confidential information that you wouldn't want a reviewer to see or Google to use to improve our services" (Google). That's the state's advice in the company's own words. Anthropic limits access to "designated members of our Trust & Safety team" on a need-to-know basis (Anthropic). Meta goes furthest in one mode. A Meta AI Incognito chat runs on what Meta calls Private Processing, and Meta says it "can't access" the chat (Meta). We compare who reviews what, app by app, in who reviews AI chats, and the training-versus-review mix-up is one of our AI privacy myths.

Can an AI chatbot be hacked, and does a setting help?

The release warns that anything you enter "could potentially be shared or exposed in a data breach" (New York Department of State). A breach can only expose what the company still holds, so the settings that help are the ones that shorten retention: Gemini's 72 hours with Keep Activity off, and the 30-day temporary modes elsewhere. The training switches don't shorten anything. We list the incidents so far in AI chatbot data breaches.

How Morse keeps your chat history off its servers

Every setting in the tables above is a switch on a copy of your chat that the company already holds, under an account with your name on it. Morse is built so your chat history never sits on our servers. It's stored in your browser, encrypted, and Morse doesn't train AI on your content, sell your data, or show ads.

There's no training setting to adjust, because nothing is collected for training. There's no "future data saved" to opt out of, because our servers don't keep your messages or log what you write. Your account holds sign-in details, plan, balance, and a usage ledger with no content. A breach of our servers would find no stored chat history, because we don't keep it.

Each chat runs in the mode you pick:

• Private, for the health question or legal letter you'd rather no host kept. It's the default for a new chat. Open-weight models like DeepSeek, Kimi, and GLM, plus Grok, answer under Morse's zero data retention agreements with its inference, search, and other providers. Where a provider confirms zero retention on each reply, Morse checks it and refuses a reply that doesn't.
• E2EE, for anything no one else should read. Your browser checks the enclave's hardware attestation, then encrypts the message so only that enclave can open it, and Morse's servers relay text they can't read. This rests on the enclave hardware and attestation working as designed. E2EE runs open-weight models, without web search or image generation, and comes with Pro.
• Anonymous, for GPT, Claude, or Gemini without your name attached. Our proxy removes your account, email, and IP address and sends the message under Morse's own account, so the provider reads it without knowing it's you. The provider may keep it for a limited time to check for abuse, under its own policy.

In every mode, the history on your device is locked when you sign out and takes two keys to open, one of which never leaves the device. Because we don't store chats, we usually have very little to give authorities. Morse strips who you are but not what you type, so the state's first tip still stands: a Social Security number, a password, or a PIN doesn't belong in any chat, ours included. And since clearing your browser deletes your chats for good and Morse can't recover them, keep a copy of anything you need.

Try a chat from the Morse home page without an account, or start free with no card. End-to-end encrypted chats come with Pro, at $19 a month.