Apple’s AI Models Are Weaker than Year-Old Rival Models and Other Underdogs

Apple has announced updates to the AI models that power its Apple Intelligence features across iOS, macOS, and other platforms. However, according to the company’s benchmarks, these new models currently underperform compared to those from leading competitors, including OpenAI, Google, and Meta.

In a blog post published Monday, Apple stated that human evaluators rated the text quality generated by its latest “Apple On-Device” model as comparable to, but not better than, similarly sized models from Google and Alibaba. The Apple On-Device model, which operates offline on devices such as the iPhone, is approximately 3 billion parameters in size and supports features like summarization and text analysis.

Falls Behind Year-Old Models

Apple also introduced a more advanced model, “Apple Server,” designed to run in the company’s data centers. According to Apple’s benchmarks, human testers rated Apple Server’s text generation capabilities below those of OpenAI’s GPT-4o, a model that has been available for over a year.

Llama Beats Apple, Despite Being an Underdog

In addition to text generation, Apple evaluated its models’ ability to analyze images. Human raters preferred Meta’s Llama 4 Scout model over Apple Server for image analysis tasks. Notably, Llama 4 Scout itself does not consistently outperform leading models from Google, Anthropic, and OpenAI, highlighting the competitive challenges Apple faces in this area.

Despite these benchmark results, Apple reports that both Apple On-Device and Apple Server models offer improved tool use and efficiency compared to previous versions. The models can now understand approximately 15 languages, aided by an expanded training dataset that includes images, PDFs, documents, infographics, tables, and charts. As of this week, third-party developers can access the Apple On-Device model through the company’s Foundation Models framework.