How ARKit 2 Estimates Real-World Scale in iOS 12

ARKit 2 achieves accurate real-world scale estimation by combining visual data from the iOS device camera with physical motion data from the internal Inertial Measurement Unit (IMU). This process, known as Visual-Inertial Odometry (VIO), allows the system to resolve the scale ambiguity inherent in standard monocular cameras. By integrating high-frequency sensor readings with optical feature tracking, surface detection, and pre-measured reference assets, ARKit 2 renders 3D models using standardized metric measurements (meters) that correspond directly to physical space.

Visual-Inertial Odometry (VIO) and Metric Calibration

A single camera cannot independently differentiate between a small object nearby and a large object far away. ARKit 2 overcomes this limitation through Visual-Inertial Odometry. While the camera captures optical frames at 60 frames per second to identify visual feature points, the device’s IMU (accelerometer and gyroscope) samples motion data at roughly 1000 Hz.

Because the accelerometer measures real-world physical acceleration in meters per second squared (\(m/s^2\)), ARKit uses this data to bridge the gap between pixel coordinates and real-world units. By calculating the physical distance the phone moves between frames, the framework mathematically triangulates the true distance and physical dimensions of tracked feature points in the surrounding environment.

Surface Detection and Plane Fitting

Once feature points are accurately placed in 3D metric space, ARKit 2 groups these points to identify continuous physical planes. iOS 12 expanded plane detection capabilities to recognize both horizontal surfaces (such as floors and tabletops) and vertical surfaces (such as walls and doors), as well as irregularly shaped boundaries.

When a developer places a virtual 3D asset into the scene, ARKit uses raycasting to determine where a virtual ray intersects these calibrated physical planes. Because the plane's position and orientation are tied to real metric coordinates, the engine scales the rendered 3D asset to match its exact digital dimensions (where 1 unit in SceneKit or RealityKit equals 1 physical meter).

Known Reference Images and 3D Objects

ARKit 2 introduced enhanced tracking for predetermined 2D images and physical 3D objects, which significantly refines scale accuracy:

• ARReferenceImage: Developers specify the exact real-world physical width (e.g., 0.15 meters) of an image or poster. When the camera identifies the image, ARKit instantly resets or fine-tunes its spatial coordinate system based on this known anchor.
• ARReferenceObject: ARKit 2 added 3D object scanning and detection. By scanning a physical object beforehand, the system creates a spatial point cloud with known dimensions. When that object is recognized in a live camera feed, ARKit locks its position and adjusts localized environmental scale immediately.

Spatial Persistence via ARWorldMap

In iOS 12, ARKit introduced ARWorldMap, a data structure that captures the visual feature points and anchors of a mapped area. Because the ARWorldMap retains the physical scale established during tracking, multi-user experiences and saved sessions preserve accurate dimensions. When a second iOS device loads an existing world map, it aligns its own coordinate system with the previously mapped physical measurements, ensuring virtual objects appear at the exact same size and relative distance across all participating devices.