How Webpage Image Translation Works in iOS 16 Safari

Apple’s iOS 16 introduced system-level image translation directly within Safari, allowing users to read foreign-language text embedded inside web images seamlessly. Powered by Apple’s Neural Engine and the expanded capabilities of Live Text, this feature identifies, extracts, and translates text directly on the webpage without requiring third-party tools or copy-pasting into a separate translation app.

The Underlying Technology: Live Text and Machine Learning

The core mechanism relies on Apple’s Live Text framework, which uses on-device optical character recognition (OCR) and machine learning models. When Safari loads a webpage containing an image with text, the operating system continuously analyzes the visual data. The Neural Engine detects legible characters, categorizes them as text elements, and identifies the source language automatically.

How Users Trigger the Feature

Users can interact with image translation in Safari through two primary methods:

  1. Direct Image Interaction: Tapping and holding an image with foreign text reveals the Live Text indicator (a small document-scanning icon) in the bottom-right corner of the image. Tapping this icon isolates the text, allowing the user to tap "Translate" from the context menu.
  2. Full-Page Translation: When a user utilizes Safari’s native webpage translation feature via the "aA" button in the address bar, iOS 16 attempts to translate both the webpage's HTML text and the detected text within visible images simultaneously.

In-Place Visual Replacement

Rather than simply displaying translated text in a pop-up window, iOS 16 can replace the original foreign text directly over the image. The software analyzes the background color, lighting, font size, and perspective of the original image text. It then overlays the translated text in a matching style, attempting to preserve the original visual layout of graphics like menus, signs, infographics, and banners.

Privacy and Language Processing

Because the OCR process is executed locally by the Neural Engine, processing image text minimizes latency and preserves user privacy. For the translation itself, Safari uses Apple's native translation engine, which supports offline translation if the corresponding language packs have been downloaded to the device, or fast cloud processing when connected to the internet.