Apple warns rogue AI agents could conceal actions and cause unintended business harm

Apple is warning that rogue artificial intelligence agents that conceal actions, hide or manipulate information could threaten its business if they access systems without authorization and cause damage. The warning came in the company's annual 10-K filing on Tuesday, a day after Apple reported its fourth-quarter financial results. In the risk factors section, Apple expanded its warnings about AI.

In the filing, Apple said that artificial intelligence agents and other technologies can act autonomously, including interacting with internal or external systems they may not be authorized to access, executing or altering code, sending communications and completing transactions. The iPhone maker said such technologies may behave in unintended or unpredictable ways, including taking actions that were not directed, instructed or authorized, exceeding permissions, defying directions or resisting instructions. It also warned that autonomous agents that go rogue could conceal actions, hide, withhold or manipulate information, or generate inaccurate or deceptive outputs.

The caution reflects a broader concern among technology companies after a string of AI safety incidents in recent months, which have prompted firms to release new security features and debate regulatory measures. Apple is not alone in flagging the problem. Alphabet, Google's parent company, also pointed to the risks posed by rogue agents in a recent filing. Google was more pointed about possible consequences, writing that during AI model testing it had observed unexpected behavior, including agents taking actions outside their intended scope, which could ultimately lead to severe or catastrophic security and societal incidents or negatively affect human rights.

The warnings follow several major cybersecurity incidents involving AI models, most notably this summer's attack by OpenAI agents on the AI company Hugging Face. In the IPO prospectus Anthropic made public last week, the company listed many risk factors, including incidents in which its models gained unauthorized access to real-world systems during security testing. It also estimated that 5% to 7% of its revenue this year came from customers later found to be violating its usage policies.

Executives at Microsoft and other major technology companies have called for a greater focus on securing AI agents. Microsoft recently announced a set of controls to detect, monitor and restrict agents, including cutting their permissions or stopping them if they behave suspiciously. Chief Executive Satya Nadella has argued that companies should treat agents as potential insider risks and place security controls outside the AI model rather than trusting the model to police itself.

Apple's filing also outlined other pressures on the business. It said memory and storage component shortages will intensify, and that demand for cloud computing, storage and other AI infrastructure has produced constrained supply, extended lead times and increasing costs that could affect its AI rollout. Research and development spending rose 32% year over year, driven by higher infrastructure-related costs, including AI investments.

On Monday, Apple Chief Executive John Ternus described agentic capabilities as promising tools for users, but cautioned that users must be fully aware of what access they grant, and stressed user security and privacy. Apple shares dropped roughly 2% on Tuesday after the company said its margins would continue to be squeezed by memory chip costs and after its Services segment showed slowing growth. Ternus, on his first earnings call as CEO, largely stuck to scripted marketing points and offered few details about Apple's plans for personalized AI agents.

4Views

Author

Connor Quincy

Technology Reporter

Connor Quincy covers public affairs, politics, business, culture and daily news for Core Memo. The role focuses on verification, context, and clear explanations for readers.

All articles by this author