Fireworks incident update adds customer responses and October 6 timeline

Fireworks’ initial bulletin said the company identified unauthorized use of internal credentials on October 6, including access to evaluation-job environment variables in a limited number of customer accounts.[2] It said it began containment that day and directly notified every customer it had identified as affected.[2] At first, Fireworks said it had no evidence of unauthorized access to customer models, datasets, training data, personal information or inference traffic within its security perimeter or control.[2] Its later account described impacts to account and billing information, and to secrets, datasets and incomplete models for a limited set of accounts.[1] Fireworks said its findings remain subject to an ongoing investigation.[1]

The affected internal application exposed resource identifiers, display names, descriptions and creation times.[1] The billing information included billing and shipping addresses, invoice and payment history, account balance and credit usage.[1] Fireworks said it does not store payment card numbers and that they were not affected.[1] For certain customers, the accessed data included a subset of incomplete model weights and datasets.[1] Fireworks said it was notifying affected customers and providing impact details and recommended response actions.[1] It advised customers to rotate or revoke third-party credentials with their issuing providers, then update applications and Fireworks secrets that use them.[2]

Notion said it was notified on November 2, disconnected its systems from Fireworks and deleted its data there; it said Fireworks notified it on November 6 that malicious actors may have accessed potential customer data.[3] Notion also said it disabled its Insights feature during its investigation and stopped serving open-weight models through Fireworks.[3] Forbes reported that two customers had substantially moved their business off Fireworks, with one citing hacks and poor communication.[4] Fireworks said it disabled self-serve dataset uploads and downloads for Batch Inference and dataset-based training while investigating.[1] It said Training SDK remained available with zero data retention, and Lab-based training operated in an isolated environment with added network and storage protection.[1]

Fireworks said it revoked and replaced credentials and service accounts, removed unnecessary elevated privileges, and tightened controls against creating unauthorized credentials.[1] It also said it restricted unauthorized network pathways, isolated affected systems, reviewed access to data storage and analytics systems, and added controls against public or unauthorized access to stored data.[1] The company said it secured and enhanced logging and monitoring systems.[1] Separately, it said a third-party cloud provider hosting its R&D machines experienced a security incident on October 23, and an unauthorized party used the provider the next day to reach development machines and run a workload that caused additional customer-data impact.[1] Fireworks said it detected that activity on October 24 and had not detected further evidence of successful unauthorized access to the customer platform as of October 25.[1]