How Apple Visual Look Up Identifies Landmarks in iOS 15

Apple introduced Visual Look Up in iOS 15, allowing the Photos app to recognize objects, plants, pets, and famous landmarks directly within a user's image library. This article explains the underlying technology behind landmark recognition, detailing how on-device machine learning, computer vision models, metadata correlation, and Apple's knowledge graph work together to identify famous structures accurately while maintaining user privacy.

On-Device Neural Engine Processing

Visual Look Up relies heavily on the Apple Neural Engine (ANE) built into modern A-series bionic chips. When a user takes a photo or views an existing one in the Photos app, iOS runs advanced computer vision algorithms locally on the device. Because the processing occurs on-device, the system can parse visual data efficiently in the background without needing to upload private photos to external cloud servers for analysis.

Feature Extraction and Visual Signatures

To recognize a landmark, the system uses deep convolutional neural networks (CNNs) trained on millions of architectural and geographic images. The model breaks the image down into mathematical representations called feature vectors or visual embeddings. These models analyze distinctive visual markers, including:

  • Architectural geometry, arches, and structural silhouettes.
  • Distinctive textures, facade patterns, and construction materials.
  • Relative scale and spatial arrangements of structural elements.

Even if a landmark is photographed from an unusual angle, partially obscured by crowds, or captured in poor lighting, the neural network compares the extracted visual signature against an embedded index of high-confidence reference signatures for world-famous sites.

Correlation with EXIF and Geolocation Data

While the visual model can identify unique structures—such as the Eiffel Tower or the Taj Mahal—purely by sight, the Photos app also uses contextual metadata embedded in the image file. If the photo contains geotagged GPS data, the algorithm narrows down the search space to landmarks within that specific geographic radius. This prevents misidentifications between architecturally similar buildings located in different parts of the world. However, if location services were disabled when the photo was taken, Visual Look Up can still rely solely on the visual classifier if the landmark's features are sufficiently distinct.

The Information Icon and Knowledge Graph Mapping

When the Photos app successfully identifies a landmark with a high confidence score, it flags the image UI. The standard "Info" (i) button at the bottom of the screen transforms into an icon featuring small stars (i*), indicating that Visual Look Up has data available.

Tapping the icon triggers a query to Apple’s Knowledge Graph (the same database powering Siri Knowledge and Maps). The device matches the recognized visual ID to a rich card displaying:

  • The official name of the landmark.
  • A brief summary sourced from reputable databases like Wikipedia.
  • Direct integration with Apple Maps, providing location information, directions, and hours of operation.