Feature Point Extraction in iOS 12 ARKit 2

Apple’s ARKit 2, introduced with iOS 12, significantly enhanced the framework's visual inertial odometry (VIO) and computer vision pipelines to improve how digital devices interpret real-world spaces. By refining feature point extraction, ARKit 2 allowed for denser point cloud generation, robust surface recognition, persistent world mapping, and multi-user shared experiences. This article details the specific architectural and algorithmic improvements made to feature point detection and tracking in ARKit 2.

Denser and More Resilient Point Clouds

ARKit 2 improved the underlying computer vision algorithms responsible for identifying high-contrast areas, edges, and texture variations in the camera feed. The system extracted a significantly higher density of feature points compared to its predecessor, even in challenging environments with low texture or dynamic lighting. This increased density provided a more accurate representation of the physical environment, giving the system a stronger foundation for anchoring virtual content.

Persistent World Mapping

The most prominent advancement tied to feature point extraction was the introduction of ARWorldMap. In ARKit 2, feature points were no longer treated as ephemeral data discarded between sessions. Instead, the framework gained the ability to package extracted feature points, their spatial relationships, and their visual descriptors into a serialized data format. This allowed apps to save the physical state of a mapped area to disk and reload it later, enabling persistent AR experiences where virtual objects remain in the exact physical location across multiple sessions.

Accelerated Relocalization

When an AR session loses tracking—due to rapid movement, occlusion, or transitioning to background states—the framework must match incoming camera frames with previously registered points. ARKit 2 dramatically improved this relocalization process. The updated feature extraction pipeline indexed visual feature descriptors more efficiently, enabling the device to recognize known environments almost instantaneously once a familiar cluster of feature points entered the camera's field of view.

Multi-User Collaborative Tracking

Leveraging the enhancements in feature point extraction and serialization, ARKit 2 introduced shared AR experiences. Multiple iOS devices could extract local feature points and exchange the resulting spatial maps over a peer-to-peer network. By comparing and aligning feature point clusters between distinct devices, ARKit 2 allowed different users to share a synchronized coordinate system and interact with the same virtual objects simultaneously from different angles.

3D Object Detection and Spatial Scanning

ARKit 2 extended feature point analysis beyond planar surfaces to three-dimensional objects. Using the ARObjectScanningConfiguration, developers could extract and record dense feature point clusters surrounding a rigid physical object. The framework converted these points into an ARReferenceObject, which ARKit 2 could subsequently identify and track in the real world by matching live feature points against the scanned 3D profile.