How iOS 15 SharePlay Synchronizes Screen Sharing

This article examines the underlying technical architecture that enables Apple's SharePlay to synchronize screen sharing across participant devices on iOS 15. It covers how the GroupActivities framework coordinates sessions, how the operating system captures and encodes display buffers, how real-time transport protocols maintain low latency, and how receiving devices render the shared view presentation uniformly.

The GroupActivities Framework and Session Coordination

At the foundation of SharePlay on iOS 15 is the GroupActivities framework. When a user initiates screen sharing within a FaceTime call, the system initializes a GroupSession configured for display sharing. This session acts as the control plane, establishing a secure, end-to-end encrypted channel between all connected devices. Rather than handling the heavy video pipeline directly, GroupActivities handles participant state, entitlements, and control events, notifying each device when a screen-sharing session begins, updates, or terminates.

Capture and Real-Time Video Encoding

Screen sharing does not broadcast raw application state or code; instead, it streams the graphical output of the host device:

  1. Buffer Capture: The system captures the host's display using low-level compositing services (similar to ReplayKit pipelines). Frames are captured as CVPixelBuffer objects directly from the display pipeline.
  2. Hardware Encoding: These raw pixel buffers are routed immediately to the device’s hardware-accelerated VideoToolbox encoder, converting them into H.264 or HEVC video streams.
  3. Privacy Masking: The operating system automatically intercepts and masks sensitive interface elements, such as notifications and secure text fields, directly within the composited buffer before compression.

Low-Latency Transport via AVConference

Unlike standard media playback in SharePlay—which uses AVPlayerPlaybackCoordinator to synchronize playback clocks across decoupled streams—screen sharing functions as a real-time, low-latency broadcast.

iOS 15 utilizes Apple’s internal AVConference subsystem, the engine behind FaceTime calls. The encoded screen capture stream is packetized and transmitted using Secure Real-Time Transport Protocol (SRTP). The pipeline dynamically adjusts video resolution, framerate, and bitrate based on real-time network conditions and round-trip time (RTT) estimations among participants, prioritizing immediate visual responsiveness over pixel-perfect lossless playback.

View Presentation and Layout Synchronization

Once the video stream arrives at a recipient device, the system synchronizes how the view is presented:

  • Hardware Decoding and Metal Rendering: The packets are decoded via hardware and rendered onto a dedicated display layer using Metal and Core Animation.
  • Aspect Ratio and Scale Management: The receiving device's operating system reads metadata embedded in the stream detailing the source device's aspect ratio and native orientation. The presentation layer automatically applies letterboxing or pillarboxing to maintain the source aspect ratio uniformly across differing display sizes (such as sharing from an iPhone to an iPad).
  • Picture-in-Picture Integration: iOS 15 routes the active presentation into the system-level Picture-in-Picture (PiP) architecture, allowing participants to multitask while maintaining a synchronized, uninterrupted feed of the presenter's screen.