How iOS 12 ARKit Handles Lighting Conditions

With the introduction of ARKit 2 in iOS 12, Apple dramatically improved how augmented reality experiences adapt to real-world illumination. By using the camera feed combined with machine learning algorithms, iOS 12 estimates ambient light levels, reproduces realistic environmental reflections, and calculates directional light sources. These features ensure that virtual objects placed on physical surfaces cast convincing shadows and match the brightness and color palette of their physical surroundings.

Ambient Intensity and Color Temperature

At the foundational level, iOS 12 continuously measures the overall ambient light through the device's camera sensor. It provides developers with two key metrics: ambient intensity and ambient color temperature. Ambient intensity measures the overall brightness of the scene in lumens, while color temperature measures the warmth or coolness of the lighting in Kelvin. When an AR object is placed on a surface, the rendering engine uses these values to adjust the object's overall brightness and tint, preventing digital assets from appearing overly bright in dim rooms or unnaturally warm under cool fluorescent lighting.

Environment Texturing

The standout lighting feature introduced in iOS 12 is environment texturing. Instead of applying static reflection maps to 3D models, ARKit 2 captures imagery from the camera as the user moves the device and stitches it into a dynamic spherical cube map.

To handle unseen areas—such as regions behind the user or outside the camera's current field of view—iOS 12 uses a machine learning model trained on thousands of indoor and outdoor environments. The neural network infers and fills in the missing visual data, creating a complete 360-degree representation of the room. When a reflective or metallic virtual object is placed on a table, its surface reflects the actual room textures, ceiling lights, and nearby walls in real time.

Directional Lighting and Shadows

To anchor objects realistically to flat surfaces, iOS 12 detects the primary directional light source in the environment. ARKit estimates the orientation and intensity of dominant lights—such as a window, overhead lamp, or the sun. Rendering frameworks like SceneKit and RealityKit utilize this directional data to generate accurate drop shadows beneath virtual items. This eliminates the floating effect common in earlier AR applications, making placed items appear firmly grounded on tables, floors, or countertops.

Spherical Harmonics for Diffuse Shading

For matte and non-glossy surfaces, iOS 12 calculates spherical harmonics coefficients representing the low-frequency ambient lighting of the environment. Rather than computing sharp reflections, spherical harmonics provide smooth, diffuse light distribution across the curves and crevices of an object. This ensures that ambient occlusion and soft shading accurately represent how light scatters across complex surfaces in the physical space.