Apple Replaces Core ML with Core AI, Bringing Free On-Device LLMs to Developers

The artificial intelligence industry has a fundamental scaling problem: every time a user prompts an app, a server somewhere burns expensive GPU compute, and the developer pays the bill. For independent creators and startups, building a highly engaging AI feature often means risking bankruptcy by API fees as usage grows.[3]

At its Worldwide Developers Conference (WWDC) in June 2026, Apple introduced a structural solution to this economic trap. The company unveiled Core AI, a ground-up rewrite of its machine learning stack designed to run large language models (LLMs) and generative AI entirely on-device.[1][2][3]

Core AI officially replaces Core ML, the framework Apple introduced nine years ago in 2017. While Core ML was revolutionary for its time, it was built for traditional, deterministic prediction tasks like image classification and object detection. It struggled to handle the autoregressive token generation, streaming responses, and multi-turn sessions required by modern generative AI.[4]

The new framework is purpose-built for Apple Silicon, orchestrating heavy AI workloads across the CPU, GPU, and the Neural Engine without requiring developers to write complex, device-specific code. By keeping inference strictly local, Core AI eliminates server dependencies, network latency, and per-token API costs.[1][2]

"Apple is making the cloud optional," notes the developer community, pointing out that local execution shifts the compute cost from the developer's cloud bill to the user's existing hardware. A simple text summarization or code generation request now travels from the user directly to the Apple Silicon chip, rather than making a round-trip to a remote server farm.[3][8]

Beyond economics, the shift fundamentally alters the privacy landscape. Because the model runs exactly where the data lives, applications can process highly sensitive information—like health records, personal messages, or financial receipts—without ever transmitting it over the internet, side-stepping massive regulatory and security hurdles.[1][7]

To make this ecosystem accessible, Apple is not forcing developers to exclusively use its proprietary models. Core AI includes a Python package called `coreai-torch`, which acts as a direct bridge to the open-source PyTorch ecosystem. Developers can convert existing open-source models into a new `.aimodel` format with just a few lines of code.[2][7]

Once converted, these models benefit deeply from Apple's unified memory architecture. Core AI utilizes zero-copy data paths and Metal 4 kernels optimized specifically for transformer architectures, allowing multi-billion parameter models to run efficiently even on the constrained thermal and battery budgets of an iPhone or iPad.[2][8]

Apple also detailed its own on-device AI architecture, revealing a 20-billion-parameter sparse model. This model uses a lazy-loaded Mixture of Experts (MoE) design, where expert selection happens per-prompt rather than per-token, minimizing the heavy data movement from storage to active memory that usually bottlenecks mobile AI.[8]

For tasks that exceed local hardware limits, Apple introduced a seamless hybrid fallback. The updated Foundation Models framework provides a unified Swift API that can automatically route complex requests to Apple's Private Cloud Compute servers, or even to third-party providers like Anthropic and Google, using a standardized Language Model protocol.[1][5]

To further entice adoption, Apple announced that developers in its Small Business Program—those with fewer than two million App Store downloads—will receive free access to Private Cloud Compute inference. This move directly challenges the API pricing models of established cloud AI giants by commoditizing server-side generation for smaller apps.[5][9]

Ultimately, Core AI signals a major strategic pivot for the tech giant. Rather than trying to be the sole provider of AI intelligence, Apple is positioning its 2.5 billion active devices as the premier arena for edge computing, giving developers the tools to build fast, private, and economically sustainable AI applications.[4][8]