How iOS 15 Live Text Extracts Text from Photos

Live Text in iOS 15 uses on-device machine learning and optical character recognition (OCR) to detect, parse, and digitize text directly from images stored in the Photos app. Powered by Apple’s Neural Engine, the feature scans the visual elements of a photo to distinguish characters from their backgrounds, applies semantic analysis to recognize actionable information like phone numbers and addresses, and allows users to select, copy, and interact with the text as if it were written in a standard document.

On-Device Neural Processing

When an image is saved to your photo library or viewed on screen, iOS 15 leverages the Apple Neural Engine integrated into the device’s A-series Bionic processor (A12 Bionic and newer). By processing images locally on the hardware, the system eliminates the latency of cloud-based APIs, ensures total user privacy, and allows the feature to function without an active internet connection.

The Text Detection and OCR Pipeline

The text extraction process works through a multi-stage computer vision pipeline:

  1. Feature Extraction: Advanced deep learning models analyze the pixel patterns of the image to differentiate high-contrast shapes, strokes, and glyphs from background elements such as textures, lighting shifts, and objects.
  2. Text Region Localization: The software maps out bounding boxes around detected clusters of characters, identifying whether text is aligned horizontally, vertically, or distorted by perspective angles.
  3. Character Recognition: The neural network compares detected shapes against trained typographic models across multiple supported languages, converting raw image data into machine-readable digital characters.

Contextual Understanding with Data Detectors

Beyond raw character recognition, Live Text integrates with system-level Data Detectors. Once characters are recognized, the operating system assesses the structural syntax of the text strings:

  • Phone Numbers: Formatted digit sequences trigger instant calling or messaging options.
  • Email Addresses: Recognized address formats allow direct composing in Mail.
  • Physical Addresses: Locations are mapped directly to Apple Maps for routing.
  • Web Addresses: URLs become clickable links directly inside the image.

User Interaction and Instant Copying

The user experience relies on a dynamic overlay that aligns directly with the underlying image canvas. When an eligible image is opened in the Photos app, an interactive Live Text indicator appears in the bottom-right corner.

Tapping this button—or performing a standard long-press gesture directly over visible text in the image—activates native text selection handles. The recognized words are immediately mapped to the system clipboard, allowing users to copy, look up definitions, translate across languages, or paste the extracted content into any other iOS application instantly.