Hardware Running Local Language Models in iOS 18

With the rollout of iOS 18 and Apple Intelligence, selected devices execute compact natural language models directly on the hardware rather than routing queries through external cloud servers. Running these approximately 3-billion-parameter models on-device requires tightly integrated silicon capable of executing billions of matrix operations while managing strict power constraints. The local processing pipeline relies on four primary hardware components working in tandem across Apple's system-on-chip (SoC) architecture: the Apple Neural Engine, unified memory, the GPU, and the CPU.

Apple Neural Engine (NPU)

The Apple Neural Engine (ANE) is the primary hardware accelerator dedicated to running on-device generative models. In supported chips—such as the A17 Pro, A18, and A18 Pro—the 16-core Neural Engine is optimized specifically for deep learning operations, such as matrix multiplications and activations common in transformer architectures. It processes complex inference workloads with high throughput and low latency while consuming significantly less battery power than traditional processing cores.

Unified Memory Architecture (RAM)

Memory capacity and bandwidth are critical bottlenecks for large language models. iOS 18 requires a baseline of 8 GB of unified memory to support on-device models. This unified pool allows the Neural Engine, GPU, and CPU to access the same memory address space without copying data between isolated pools. A portion of this memory is reserved to hold the quantized model weights and the key-value (KV) cache, enabling the system to generate text quickly without starving the operating system of standard application memory.

Graphics Processing Unit (GPU)

While the Neural Engine handles most repetitive neural network operations, the integrated Apple GPU assists in processing parallel computing workloads. Apple's modern GPUs feature architectural enhancements such as hardware-accelerated dynamic caching and specialized floating-point support, which assist during generation phases that require flexible, high-precision compute capacity beyond the fixed-function logic of the Neural Engine.

Central Processing Unit (CPU)

The CPU manages task orchestration, tokenization, and pipeline scheduling. Before the Neural Engine generates a response, the CPU tokenizes the input text, applies context window filters, and handles memory allocation. After inference, the CPU decodes the resulting tokens back into natural language and applies system-level safety checks before displaying the output to the user.

Supported Chipsets

This local natural language pipeline is physically integrated into:

• A17 Pro (iPhone 15 Pro and iPhone 15 Pro Max)
• A18 and A18 Pro (iPhone 16 lineup)
• M-Series Processors (M1 and later) on supported iPads and Macs capable of running identical Apple Intelligence models.