ARKit 2 Environment Reflections in iOS 12

ARKit 2 in iOS 12 significantly enhances the realism of augmented reality experiences by introducing environment texturing, a feature that dynamically estimates and reflects real-world surroundings on virtual 3D objects. By capturing image data during an AR session and using machine learning to predict unseen areas, ARKit synthesizes realistic cubemap textures. This allows virtual assets, especially those with glossy or metallic materials, to blend seamlessly into physical environments with accurate specular highlights and ambient reflections.

The Mechanics of Environment Texturing

To capture reflections, ARKit 2 relies on the AREnvironmentProbeAnchor class. An environment probe represents a specific point in 3D space where the framework samples the visual environment to create an omnidirectional texture map (cubemap).

As the user navigates the physical space, the device's camera collects image data from various angles. ARKit projects these camera frames onto an internal spherical model centered at the probe's anchor position. Because a user's camera rarely captures an entire 360-degree sphere of a room at once, ARKit utilizes machine learning to fill in the gaps. This inpainting algorithm predicts the unobserved portions of the room—such as what is behind or above the user—ensuring that the generated cubemap is complete and lacks jarring black voids.

Placement Modes: Automatic vs. Manual

Developers control how reflection probes are deployed using the environmentTexturing property within ARWorldTrackingConfiguration. ARKit 2 offers three distinct modes:

  • Automatic (.automatic): In this mode, ARKit dynamically creates, positions, and updates AREnvironmentProbeAnchor instances based on the scene's geometry and virtual content placement. As the camera reveals more of the surrounding space, ARKit automatically refines the cubemaps in real time. This mode provides the easiest implementation for high-fidelity rendering without requiring manual spatial tracking.
  • Manual (.manual): Developers explicitly define the positions and extents of probes by creating AREnvironmentProbeAnchor objects and adding them directly to the ARSession. This approach is optimal for stationary objects or specific zones in an environment where precise, static reflections are required.
  • None (.none): Disables environment texturing to conserve processing power when reflective realism is not necessary for the application.

Rendering Integration

Once the cubemap is generated, ARKit assigns it to the probe's environmentTexture property as a Metal texture.

Rendering engines, such as SceneKit, natively integrate with these probes. When using physically based rendering (PBR) materials, SceneKit automatically samples the nearest environment probe to determine how light should bounce off an object's surface. Properties such as metalness and roughness dictate how sharp or diffuse the reflection appears. A polished chrome sphere will display a clear, recognizable reflection of the room, while a brushed metal or matte plastic surface will scatter the estimated environment light into soft ambient reflections.