How to Add Dictation to iOS 16 Custom Keyboards

Integrating dictation into third-party iOS 16 custom keyboards requires bridging Apple’s sandboxed keyboard extension architecture with the Speech framework. Because keyboard extensions run under strict memory limits and security constraints, developers must configure specific permissions, enable network and audio access, and implement audio capture using AVAudioEngine and SFSpeechRecognizer to insert transcribed text directly into the host application.

1. Configure Extension Permissions and Full Access

Custom keyboard extensions run in an isolated sandbox by default, with no access to the device microphone or network. To implement speech-to-text, you must first request escalated privileges:

  • Enable Full Access: In the keyboard extension's Info.plist, set RequestsOpenAccess to YES under the NSExtensionAttributes dictionary. The user must manually grant this permission in iOS Settings under their keyboard options.
  • Declare Privacy Keys: In the containing app's Info.plist, define:
    • NSMicrophoneUsageDescription: Explains why the app needs microphone access.
    • NSSpeechRecognitionUsageDescription: Explains why the app requires speech processing.

2. Request User Authorization

Before initiating any recording session, request permission from the user within the extension lifecycle or the parent application:

import Speech

SFSpeechRecognizer.requestAuthorization { authStatus in
    switch authStatus {
    case .authorized:
        // Ready to capture audio
    case .denied, .restricted, .notDetermined:
        // Handle lack of permission
    @unknown default:
        break
    }
}

3. Capture Audio and Process Speech

Once permissions are active and verified, speech recognition is handled using AVAudioEngine and SFSpeechAudioBufferRecognitionRequest:

  1. Initialize the Speech Recognizer: Create an instance of SFSpeechRecognizer with the appropriate locale (e.g., Locale(identifier: "en-US")).
  2. Configure the Audio Session: Configure the shared AVAudioSession category to .playAndRecord or .record, setting options such as .duckOthers to lower background audio.
  3. Stream Audio Buffers: Set up an AVAudioEngine input node tap to capture real-time microphone buffers and append them to the recognition request.
  4. Insert Transcribed Text: Use the recognition task’s result handler to retrieve partial or final string transcripts, then inject the text into the active field using self.textDocumentProxy.insertText(transcribedString).

4. Alternative: Container App Routing via URL Schemes

Keyboard extensions on iOS 16 operate under strict memory caps (typically between 30 MB and 50 MB). Heavy audio processing pipelines using AVAudioEngine can occasionally trigger memory warnings and crash the extension.

To avoid this, many developers implement dictation by offloading the task to the containing app:

  1. The keyboard triggers a custom URL scheme (e.g., mykeyboard://dictate) to open the main app.
  2. The main app performs the audio recording and speech-to-text transcription without keyboard memory constraints.
  3. The transcribed text is saved into a shared container using App Groups (UserDefaults(suiteName:) or a shared file directory).
  4. When the user returns to the keyboard, the extension reads the stored text and writes it via textDocumentProxy.