Fireworks AI Security Breach Exposes Notion AI Customer Chat Transcripts

A security breach at AI infrastructure provider Fireworks AI has exposed the chat transcripts of Notion AI customers, compromising data that was supposed to be protected by a zero-retention policy. The incident, confirmed by both companies this week, illustrates the fragile chain of custody that governs enterprise data when it is handed off to third-party language models.[1][2][7]

Notion, a popular workspace and note-taking application, integrates AI features to help users draft documents, summarize text, and query their own databases. To power these capabilities, Notion routes user prompts to external providers, including Fireworks AI, which hosts and serves open-source models.[1][7]

Under Notion's stated security architecture, these third-party vendors operate under strict zero-data-retention agreements. Prompts and responses are meant to be processed in memory and immediately discarded, ensuring that sensitive corporate data never rests on external servers.[1][7]

The Retention Failure

However, the breach at Fireworks AI revealed a critical failure in that enforcement. Attackers gained access to a Fireworks storage environment where Notion AI chat logs had been inadvertently retained, exposing the contents of user interactions.[1][2]

Notion disclosed the incident in a security advisory on November 11, stating that it was notified by Fireworks AI of a "security incident involving unauthorized access to a subset of their systems." Notion confirmed that the exposed data included "chat transcripts from a limited number of Notion AI users."[2][7]

The exposure of chat logs is particularly sensitive because users frequently feed proprietary code, financial data, and internal strategy documents into AI assistants to generate summaries or analysis. A breach of these transcripts effectively functions as a breach of the underlying workspace.[1][4]

A Widening Breach

The incident at Fireworks AI appears to be part of a broader security failure at the infrastructure provider. The Hacker News reported on November 1 that Fireworks AI had confirmed a breach affecting customer datasets and model weights, indicating that attackers had penetrated deep into the company's hosting environment.[3]

AI

Subsequent reports from eSecurity Planet and Cyber Insider characterized the Notion data exposure as either a second breach or a widening of the initial intrusion, reaching beyond raw model infrastructure to compromise live customer data flows.[4][5]

Neither Notion nor Fireworks AI has publicly quantified the exact number of users affected or the duration for which the zero-retention policy was failing. Notion stated that it has "paused all traffic to Fireworks AI" while the investigation continues, routing requests to alternative providers.[2][7]

Supply Chain Vulnerabilities

The breach highlights a structural vulnerability in the current enterprise AI ecosystem. Companies are rushing to integrate generative AI features, often relying on specialized infrastructure providers to host the models. This creates a complex supply chain where data security depends entirely on the weakest link.[1][6]

AI

When a zero-retention policy fails silently, as it did in this case, the client company—and its users—remain unaware that their data is accumulating on a third-party server until a breach occurs. The incident is likely to prompt enterprise customers to demand more rigorous auditing of how AI vendors handle temporary data.[1][4]