How Live Text Works in iOS 16 Safari

In iOS 16, Apple expanded its Live Text technology into the Safari browser, allowing users to interact directly with text embedded inside web images and paused videos. Powered by on-device machine learning, this feature automatically recognizes textual elements across web media, enabling users to copy, translate, make phone calls, convert currencies, and look up definitions directly from a webpage without requiring third-party tools.

Text Recognition in Web Images

When browsing the web in Safari, Live Text runs automatically in the background using the device's Neural Engine. When you encounter a photo, graphic, or scanned document containing text:

  • Direct Selection: You can press and hold directly on the text within the image, just as you would with standard web text. Selection pins will appear, allowing you to highlight the exact phrase or paragraph you need.
  • The Live Text Button: When viewing an image in full or tapping on it, a small Live Text indicator (a square icon with lines of text inside) often appears in the lower-right corner. Tapping this icon highlights all recognized text within the image at once for easier selection.

Text Recognition in Paused Videos

iOS 16 introduced the ability to extract text from video frames in Safari:

  1. Play any standard web video within Safari.
  2. Pause the video at the frame containing the text you want to interact with.
  3. Tap the Live Text icon in the lower-right corner of the video player, or press and hold the text on the screen.
  4. Interact with the highlighted text using the standard context menu.

Quick Actions and Contextual Data

Beyond standard copying and sharing, Live Text identifies specific data types and provides one-tap contextual actions:

  • Phone Numbers and Emails: Tapping an identified phone number allows you to initiate a call or send a message; tapping an email address opens a new draft in Mail.
  • Addresses: Selecting a physical address gives you the option to open directions directly in Apple Maps.
  • Currencies and Measurements: Live Text automatically converts foreign currencies into your local currency or converts measurement units with a single tap.
  • Translation: Highlighting foreign text provides an immediate translation option into your system's default language.
  • Web Search and Definitions: The "Look Up" function provides dictionary definitions, Wikipedia summaries, and related web results for highlighted terms.

Privacy and Performance

Live Text processes all visual data entirely on your device using the A12 Bionic chip or newer. Because the image and video analysis is performed locally, no visual data or extracted text is sent to Apple's servers, ensuring fast performance and maintaining user privacy while browsing.