How LiDAR Detects Doors and Signs in iOS 16

Apple introduced advanced navigation capabilities in iOS 16 by pairing the iPhone and iPad Pro LiDAR scanner with machine learning to identify doors, read signs, and calculate exact distances. Integrated into the Magnifier app's Detection Mode, this feature—known as Door Detection—is designed primarily for blind and low-vision users. It combines time-of-flight depth mapping, real-time computer vision, and on-device text recognition to map the physical environment and deliver immediate spatial feedback.

Time-of-Flight (ToF) Depth Mapping

The LiDAR (Light Detection and Ranging) scanner operates by emitting invisible pulses of light toward surrounding objects. It measures the nanosecond intervals it takes for those light pulses to bounce back to the sensor. By calculating this time-of-flight, the system builds an accurate, real-time 3D point cloud of the immediate area up to approximately 5 meters (16 feet) away. Because LiDAR relies on its own light source rather than ambient light, it can determine the precise distance to physical barriers even in dimly lit environments.

Surface and Plane Segmentation with ARKit

iOS 16 utilizes Apple’s ARKit framework to process the raw point cloud generated by the LiDAR sensor. ARKit converts these spatial data points into geometric planes and surfaces. This allows the operating system to differentiate between flat walls, recessed entryways, doorframes, and floors. By calculating the physical boundaries of these planes, the device accurately tracks how far the user is from the door surface and determines whether the door is open, closed, or partially ajar.

Machine Learning and Visual Recognition

While LiDAR calculates physical distance, standard camera input works alongside it using Apple's Neural Engine and Core ML frameworks. The system applies trained machine learning models to the camera feed to recognize visual characteristics specific to doors:

  • Door Hardware: Identifies handles, doorknobs, pull bars, and push plates to inform the user how to operate the door.
  • Access Symbols: Recognizes accessibility icons, such as wheelchair accessibility emblems.

Optical Character Recognition (OCR) for Signs

To read signs, room numbers, and notices around entryways, iOS 16 relies on the Vision framework's Optical Character Recognition capabilities. When a sign or text appears within the camera's field of view, the system extracts the text in real time. Because the LiDAR sensor provides concurrent depth information for the exact surface the text is printed on, the device can inform the user not only what the sign says, but also exactly how many feet or meters away the sign is located.

Real-Time User Feedback

Once the distance and visual data are processed, iOS 16 communicates the findings through three distinct feedback channels:

  • Audio Alerts: Spoken announcements state the distance, door attributes, and sign text. Sound pitches can also change relative to the user's proximity to the door.
  • Haptics: Vibration patterns increase in intensity and frequency as the user approaches the entryway.
  • Visual Display: High-contrast text and graphic indicators appear on the screen within the Magnifier app to highlight the door and display distance readouts.