Companies Weigh Open Models as AI Data Privacy Fears Grow

Data privacy has become a decisive factor for companies selecting an artificial intelligence vendor, as executives grow wary that leading AI developers could use their proprietary information to train future models or build competing products. The tension escalated in June when Anthropic began requiring a 30-day retention policy for all chats with its most powerful models, Fable and Mythos, prompting what one advisor described as an «almost allergic reaction» across the industry.

The policy shift pushed many businesses to reconsider their reliance on frontier AI providers. A month later, Microsoft CEO Satya Nadella and Palantir CEO Alex Karp publicly warned companies against depending on models from these vendors, arguing that firms risk having their intellectual property siphoned off. Nadella said companies «pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful.»

OpenAI and Anthropic have both stated they do not train their models on enterprise data. Yet data security experts caution that AI companies may still find subtler ways to glean insights about how customers operate without directly training on processed data. «From a business standpoint, the risk of any of that data being trained on an AI model is unacceptable,» said Rob Gregory, Vice President and Chief Information Security Officer at Optiv, a B2B cybersecurity advisory firm. «We're at a very interesting crossroad.»

Recent «rogue AI» incidents, in which models scraped or published data without permission, have intensified these fears. In response, companies are increasingly pursuing what the industry calls «sovereign AI» — the ability to control their own AI stack, from owning the chips that run AI workloads to setting up proprietary cloud infrastructure and using open source models they can download and operate in their own environments. The term was historically associated with governments, but its use has now spread into the corporate world.

«Data sovereignty is something we're talking a lot about,» said Brooke Hopkins, founder of voice AI company Coval. Another executive noted that sovereign AI has been a major topic of conversation over the past few months. The appeal is clear: greater control over sensitive data and reduced dependency on a handful of dominant vendors.

However, running open source models demands more technical expertise and may mean forgoing cutting-edge capabilities in areas such as coding or financial analysis that products from OpenAI, Anthropic, and Google offer. «You're trading control for responsibility,» Gregory said. «Because there is a significant amount of responsibility when you bring that internally.»

OpenAI and Anthropic are competing directly to retain business customers and prevent them from switching to open source alternatives. After Anthropic's data retention announcement, OpenAI reiterated in an August 19 blog post that it offers full zero data retention for enterprise customers and previewed a new feature called Private Safety Processing, which adds a data storage element to ZDR and lets customers store data in their own cloud rather than on OpenAI's servers. Anthropic responded on September 1 with a similar version of ZDR that also allows businesses to store data in their own cloud accounts.

But the damage may already be done. After Anthropic announced its 30-day retention policy, firms like Booz Allen restricted their teams' use of the Fable model, according to The Information. Many businesses now access leading models through Amazon Bedrock, which lets companies use multiple AI models without providers seeing their data. Bedrock has become one of the fastest-growing AWS products, with customer spend up 170% in the first quarter and nearly 80% adoption among the Fortune 100.

«Amazon is like, 'You don't have to trust the frontier labs,'» said Randall Hunt, chief technology officer at Caylent, an AWS advisory company. «It's a more established company people have been using for decades.»

Many companies are adopting a hybrid approach, combining closed models with open source ones. «Sovereign AI basically enables you to say, 'Okay, I will continue to partner with OpenAI and Anthropic for some things, but maybe here's another set of use cases that I want to use open source models for on prem,'» one executive explained. This strategy, however, requires companies to secure and implement models responsibly on their own.

Glen Wise, CEO and co-founder of Cinder, which makes software to detect and remediate model abuse, has seen a spike in customer requests since Anthropic announced Fable would not have zero data retention. The shift signals a broader recalibration of trust between businesses and the AI companies that supply their most advanced tools.

2Views

Author

Derek Weston

Sports Writer

Derek Weston covers public affairs, politics, business, culture and daily news for Core Memo. The role focuses on verification, context, and clear explanations for readers.

All articles by this author