hex/ios/HEXShared/AudioSessionControlling.swift
Ryan Vogel 62ba0b4984
iOS: a warm session dictates in the background on iOS 27 again, and a keyboard that opens Hex brings the warm default back (#2406)
TestFlight build 129 (iPhone 17 Pro, iOS 27.3): Dictate opened Hex every time, and back
in the chat app the Hex key said "Another app is using audio. Try again."

Cause, reproduced on an iPhone 17 Pro on iOS 27.2 with build 128's code (84915a9f,
before #2393) as well as dev: a warm session's dictation fails as soon as Hex is in the
background. beginCapture calls configureAudioSession, which set the category again even
though it was unchanged (the keepalive's .playAndRecord/.measurement/same options). On iOS
27 that asks to interrupt others, which audiomxd refuses for a background app that isn't
Now Playing ("can't stop others because it is in the background", "BeginInterruption
returned an error"), so setCategory throws cannotInterruptOthers ('!int'), which Hex shows
as "Another app is using audio". The keepalive restart after a capture fails the same way
in the background, so the warm session silently ends. #2393 didn't cause this, but its
Off default meant a keyboard that opened Hex for a reason it left no record of got a one
shot each time and no warm session.

What changed
- AudioBackend.configureAudioSession sets the category only when category, mode or
  options differ (AudioSessionControlling now exposes category, mode and categoryOptions,
  which AVAudioSession already has). A Bluetooth choice, which needs HFP, still sets it.
- Hex records every keyboard launch for a dictation as "the keyboard didn't dictate by
  itself" (KeyboardSelfDictation.recordOpenedHex, reason openedHex, keeping a reason the
  keyboard wrote), before reading the warm default, so the 15-minute default, the row and
  a warm cold start come back from that launch whatever the keyboard's reason was.
- The record is cleared when the keyboard delivers a dictation it made itself (insertResult
  with source "keyboard"), not when one merely starts (replaced: the clear in the
  onLocalDictateTap closure).
- Removed: "Off where the keyboard dictates by itself: a launch that missed its recording
  warms nothing" in activateWarmSessionAndRunHandoff. Every keyboard launch now records,
  so it could no longer apply; a late handoff warms as it did before 129.
- README's warm-session paragraph says so.

Verified
- iPhone 17 Pro, iOS 27.2 (UDID 00008150-00163C5C0A47801C), builds signed for the device:
  - build 128 code: Keep mic warm 15m, then the Hex key in another app shows "Another app
    is using audio. Try again." (the report's screenshot); the next tap dictates in the
    keyboard.
  - this branch: a keyboard cold start (hex-dictation://keyboard/record) records, Done in
    the keyboard keeps the warm session (Hex key orange), the next Hex key records through
    Hex in the background with no error, and the warm session stays after it. The App Group
    then holds keyboard.handedDictationToHex = openedHex and Settings shows Keep mic warm
    at 15m.
  - control: with Hex in the back and no warm session, the keyboard dictates by itself,
    also with Safari playing audio.
- xcodebuild test on an iOS 27.2 simulator: 878 tests, 3 skipped, 0 failures. New:
  AudioSessionSetupTests.testTheCategoryIsSetOnlyWhenItChanges; updated
  testKeepMicWarmFollowsWhetherTheKeyboardDictatesItself (a late handoff records and warms)
  and KeyboardBridgeTests (recordOpenedHex keeps the keyboard's reason).
- Not checked: iOS 27.3 itself, iOS 26, and a real phone whose keyboard can't dictate by
  itself (memory, missing model); a keyboard-made dictation clearing the record was only
  unit-tested.
- python3 scripts/code-shape.py passes; git diff --check clean.
2026-11-22 13:12:24 -05:00

54 lines
2.4 KiB
Swift

import AVFAudio
/// The `AVAudioSession` calls Hex makes, so tests can check how it records and lets go.
protocol AudioSessionControlling: AnyObject {
var isOtherAudioPlaying: Bool { get }
var category: AVAudioSession.Category { get }
var mode: AVAudioSession.Mode { get }
var categoryOptions: AVAudioSession.CategoryOptions { get }
/// The inputs the current category offers.
var microphones: [Microphone] { get }
func setCategory(
_ category: AVAudioSession.Category,
mode: AVAudioSession.Mode,
options: AVAudioSession.CategoryOptions
) throws
/// Records from `microphone`, or from iOS's choice for nil.
func prefer(_ microphone: Microphone?) throws
func setActive(_ active: Bool, options: AVAudioSession.SetActiveOptions) throws
}
extension AVAudioSession: AudioSessionControlling {
var microphones: [Microphone] {
(availableInputs ?? []).map(Microphone.init)
}
func prefer(_ microphone: Microphone?) throws {
try setPreferredInput(microphone.flatMap { wanted in
availableInputs?.first { $0.uid == wanted.uid }
})
}
}
/// Lets other apps' audio that in-keyboard dictation interrupted play again.
///
/// The keyboard records through WebKit (`getUserMedia`), whose audio session lives in
/// WebKit's GPU process: WebKit sets `.playAndRecord` in `.videoChat` mode without
/// `.mixWithOthers`, and no page or app setting changes that. Going active, it interrupts
/// another app's video or music; ending, it deactivates without `.notifyOthersOnDeactivation`,
/// so the player is never told it may resume. When other audio played as the capture began
/// and nothing plays once it has ended, the keyboard activates its own session (nothing else
/// plays, so it interrupts nothing) and deactivates it with `.notifyOthersOnDeactivation`,
/// for players that honour `.shouldResume`. Whether iOS passes that on to a player WebKit
/// interrupted can only be seen on an iPhone: the simulator has no interruptions.
nonisolated enum InterruptedAudio {
static func shouldAskToResume(playingAtStart: Bool, playingNow: Bool) -> Bool {
playingAtStart && !playingNow
}
static func askToResume(_ session: AudioSessionControlling) throws {
try session.setCategory(.playback, mode: .default, options: [])
try session.setActive(true, options: [])
try session.setActive(false, options: .notifyOthersOnDeactivation)
}
}