How SharePlay Screen Scaling Works in iOS 15

This article provides an overview of how screen content scaling operates during a SharePlay session on iOS 15. It covers how the operating system handles varying hardware aspect ratios, client-side interactive zooming, dynamic orientation adjustments, and adaptive resolution scaling to ensure a clear and synchronized viewing experience across different Apple devices.

Aspect Ratio and Viewport Fitting

When a user shares their screen over FaceTime via SharePlay, iOS 15 automatically fits the host’s display dimensions into the viewer’s screen bounds using standard aspect-fit scaling. Because Apple devices feature differing screen dimensions—such as the taller aspect ratios of modern iPhones compared to the 4:3 or near-4:3 aspect ratios of iPads—the system introduces black bars (letterboxing or pillarboxing). This ensures that the entirety of the presenter’s screen remains visible without cropping or geometric distortion.

Independent Client-Side Pinch-to-Zoom

Viewers are not locked into a static, scaled-down frame. iOS 15 includes independent client-side zoom controls:

• Pinch to Zoom: Viewers can pinch inward or outward on the video feed to enlarge specific areas of the presenter's screen, such as fine text, complex app layouts, or detailed images.
• Pan to Navigate: When zoomed in, viewers can drag their finger across the display to navigate around the magnified canvas.
• Isolated Interaction: These scaling and panning actions are entirely local to the recipient’s device. They do not alter the presenter's display or change the view for other call participants. Double-tapping the screen quickly resets the feed back to the standard aspect-fit scale.

Dynamic Orientation Adjustments

Screen scaling adapts in real time to physical orientation changes from both the presenter and the viewer:

• Presenter Rotation: If the presenter rotates their iPhone from portrait to landscape, the outgoing video frame adjusts its geometry immediately. The viewer's device recalculates the aspect-fit bounds on the fly to maximize the viewable area for the new orientation.
• Viewer Rotation: Viewers can also rotate their own devices to match the presenter’s orientation, minimizing empty black bars and effectively maximizing the scaled dimensions of the shared content.

Network-Based Resolution and Frame Rate Scaling

Beneath the visual layout, iOS 15 manages content scaling dynamically based on network conditions. The underlying AVFoundation and WebRTC-based architecture monitors bandwidth and latency:

• Downscaling: Under poor connection quality, the system dynamically scales down the encoding resolution and frame rate of the shared screen to prioritize low audio and video latency.
• Upscaling: When network stability improves, the stream scales back up toward native resolution, maintaining sharp interface elements and readable text.