VoiceOver Subject Descriptions in iOS 16 Photos

Apple’s iOS 16 introduces an advanced machine learning feature that isolates visual subjects from image backgrounds, and this capability is deeply integrated into the VoiceOver screen reader. This article explains how VoiceOver identifies, announces, and interacts with extracted photo subjects, providing visually impaired users with precise descriptions and contextual actions for isolated elements within the Photos app and across the operating system.

Detection and Audio Feedback

When navigating an image in iOS 16 with VoiceOver enabled, the screen reader uses on-device machine learning to scan the composition for distinct foreground objects, such as people, pets, vehicles, or plants. When VoiceOver encounters an image containing an isolatable subject, it announces that subjects are detected within the photo.

When a user performs the lift gesture—or uses VoiceOver actions to isolate the item—the system delivers a distinct haptic tap alongside an audio cue. VoiceOver then announces the status of the extraction, confirming that the foreground subject has been separated from the background.

Detailed Subject Descriptions

Rather than treating the entire photo as a single visual unit, VoiceOver provides targeted descriptions for the isolated subject.

  1. Object Classification: VoiceOver identifies what the extracted subject is, such as "person," "dog," or "flower."
  2. Visual Look Up Integration: If the subject matches data recognized by Visual Look Up, VoiceOver includes specific metadata in its spoken output, such as the exact dog breed, plant species, art piece, or landmark.
  3. Position and Orientation: VoiceOver indicates where the subject is positioned within the frame, noting its relative size and placement (e.g., "Subject centered," "Large subject on the left").

Rotor Actions and Interaction

VoiceOver incorporates the subject-lifting feature directly into the Actions rotor, making the extracted element accessible without requiring complex touch-and-hold gestures.

  • Isolate Subject: When focused on a photo, swiping up or down with one finger reveals the "Copy Subject" or "Share Subject" actions in the rotor.
  • Separation from Background Text: If an image contains both text and a central subject, VoiceOver differentiates between the two, allowing users to explore the extracted visual element independently from the image's Live Text layers.
  • Clipboard Confirmation: Upon copying the isolated subject, VoiceOver confirms the action by announcing that the subject—without the background—is stored on the clipboard, ready to be pasted into apps like Messages, Notes, or Mail.