hex/ios/HEXMobile/SpeechModelMove.swift
Ryan Vogel 7e4ee0b483
iOS: Speech model in keyboard on by default for English, downloaded only where the keyboard can use it (#2390)
* iOS: Speech model in keyboard on by default for English, downloaded only where the keyboard can use it

What changed
- KeyboardParakeetSettings.isEnabled reads on unless the person turned it off (a stored value
  wins, so a switch turned off stays off). The switch shows off and disabled under 6 GB (isOn).
  When Parakeet is used is unchanged: English (V2) or the Multilingual model's language
  (Ultra), 6 GB or more, not with Two-language dictation, Apple's engine whenever the model
  isn't ready or doesn't fit, never the GPU, one chunk at a time.
- Automatic download (KeyboardParakeetDownload.hold): only for English dictation without
  Two-language dictation or the Multilingual model, once the keyboard has measured its own
  memory limit on this iPhone and it is at least KeyboardParakeetBudget.requiredLimit (the
  keyboard writes footprint + os_proc_available_memory to the App Group at each launch and
  posts a Darwin notification when it changed), on a connection that is neither expensive nor
  constrained (NWPathMonitor: not cellular, a personal hotspot or Low Data Mode), with 2 GB
  free. Checked at launch, when Hex comes to the front, when such a connection returns and
  when the keyboard reports its limit; an automatic download stops when the connection
  becomes expensive (its files resume). Turned on by hand (a stored true, also from the
  builds where it was off by default) it still downloads at once on any connection.
- The row: "For English. Uses about 700 MB, downloaded on Wi-Fi. Until then, the keyboard
  uses Apple's speech recognition." (464 MB download plus the keyboard's 234 MB compiled
  copy), and while it waits: "Downloads after you first use the Hex keyboard", "Downloads on
  Wi-Fi", "Downloads once the iPhone has 2 GB free", "Only for English dictation"; the
  keyboard's statuses are unchanged. Simplified Chinese for all; the old detail string is
  replaced. Still under Experiments.
- Moving a legacy Parakeet folder into the App Group forgets that model's compile record
  (ParakeetCompileRecord.forget): Core ML compiles moved files again (18 s for Ultra on an
  iPhone 17 Pro, measured), which the app showed as "Loading the speech model" with a quick
  estimate. Now V2 says it's optimizing for this iPhone with the first-load estimate, and
  Ultra gets ready beside the engine that dictates, as after its download.
- KeyboardParakeetBudget: one set of numbers for V2 and Ultra (Ultra's transcription rose
  9 MB in Hex's keyboard, under V2's 16 MB); required limit 96 MB for both (was 115 MB for
  Ultra, from the probe keyboard).
- The keyboard's log calls a load cold by its time (5 s or more) instead of a per-iOS-version
  record, which called real recompiles after a re-download "warm"; the record is removed.
- README updated.

Why
- Ryan: Parakeet V2 becomes the default for English dictation in the keyboard in the next
  TestFlight build, without anyone downloading 464 MB for a keyboard that can't use it.

Verified
- Studio simulator (iPhone 17 Pro, iOS 27.2, fresh), on origin/dev 962164fc: xcodebuild test,
  857 tests, 0 failures, 3 skipped. scripts/code-shape.py passes.
- iPhone 17 Pro, iOS 27.2, Release build installed over the previous one: the keyboard wrote
  its limit (177.0 MB) at its first open; the row shows the new detail; the first open after
  the update recompiled V2 (18.2 s, logged cold) and that dictation fell back to Apple's
  engine (0.42 s); switch off deleted the files and dictated with Apple (peak 56 MB, no
  Parakeet log lines); switch on by hand downloaded at once (30 s), the keyboard compiled it
  (15.1 s, cold) and a dictation with Hex force-quit ran on V2 (56 MB before, 71.7 MB peak,
  0.32 s for 9.9 s of audio); the row then said "Ready in the keyboard".
- Not yet on a phone: the automatic download from a never-set switch (needs a fresh install).

* iOS: the warm mic's default waits for the keyboard to load the multilingual model itself

What changed
- KeyboardSelfDictation.dictatesItself ended with KeyboardParakeetSettings.isEnabled ||
  !keyboardOpensHex. Written while isEnabled meant "turned on by hand"; with Speech model in
  keyboard on by default it said the keyboard dictates by itself for everyone whose language
  the Multilingual model takes, before the keyboard had that model loaded, so Keep mic warm
  defaulted to Off and hid while Dictate still opened Hex.
- Now the keyboard dictates that language itself only once it has the multilingual model
  loaded: the keyboard records which model its "ready" state is
  (KeyboardParakeetSettings.setKeyboardState(_:multilingual:), keyboardHasMultilingualModel),
  and dictatesItself asks that instead of the switch.

Why
- People whose language the Multilingual model takes keep their warm mic until the keyboard
  really dictates by itself (#2393's intent), with the switch on by default.

Verified
- KeyboardBridgeTests.testKeepMicWarmDefaultFollowsWhetherTheKeyboardDictatesItself gains the
  cases: on by default with only Hex having the model (15 minutes), the keyboard having V2
  ready (15 minutes), loading the multilingual model (15 minutes), having it ready (Off), and
  the switch turned off (15 minutes); its old assertion "only Hex loads the multilingual
  model" contradicts the old line with the switch unset.
- Studio simulator (iPhone 17 Pro, iOS 27.2, fresh) on this head, rebased on origin/dev
  bd8da457: xcodebuild test, 861 tests, 0 failures, 3 skipped. scripts/code-shape.py passes.

* iOS: the keyboard's Parakeet budget counts the recording at 91 MB, so it needs a 131 MB limit

What changed
- KeyboardParakeetBudget.recordingFootprint 56 MB -> 91 MB, so requiredLimit 96 MB -> 131 MB
  (V2 and Ultra alike). Both the keyboard's load and transcription guards and Hex's automatic
  download (KeyboardParakeetDownload.hold, keyboardTooSmall) use it.

Why
- 56 MB is a first dictation's recording. With the model loaded, over twenty dictations in one
  keyboard process on an iPhone 17 Pro, recordings started at 74.7-78.0 MB through the ninth
  and at 87.9-91.4 MB after (with Apple's speech recognition they stayed at 55.8-56.3 MB), so a
  96 MB requirement let in a keyboard whose limit is barely above what it records at. The
  iPhone 17 Pro's 177 MB still passes.

Verified
- DictationControllerTests: requiredLimit is 131 MB; a 100 MB keyboard neither loads the model
  nor gets it downloaded by itself; the 177 MB cases still pass.
- Studio simulator suite on the rebased head (see the PR).
2026-11-22 12:18:29 -05:00

65 lines
2.9 KiB
Swift

import FluidAudio
import Foundation
extension SpeechModelStore {
/// Where Parakeet V2 was before it moved to the App Group: FluidAudio's own cache folder.
nonisolated static var legacyParakeetV2: URL {
AsrModels.defaultCacheDirectory(for: .v2)
}
/// Moves Parakeet's files from the app's Application Support into the App Group, once, at
/// launch before anything reads or writes them: a rename on the same volume, so nothing
/// downloads again, `.partial` files included (their downloads resume where they were). A
/// folder already in the App Group wins unless only the old one is complete. Core ML compiles
/// moved files again (18 s for Ultra on an iPhone 17 Pro), so a moved model's compile record
/// goes: V2 then says it's optimizing for this iPhone, and Ultra gets ready beside the engine
/// that dictates, as after its download.
nonisolated static func moveLegacyDownloads(fileManager: FileManager = .default) {
move(
from: legacyParakeetV2,
to: ParakeetTranscriber.cacheDirectory,
isComplete: { AsrModels.modelsExist(at: $0, version: .v2) },
forgetting: ParakeetCompileRecord(key: ParakeetCompileRecord.v2Key),
fileManager: fileManager
)
move(
from: ParakeetUltraModel.legacyDirectory,
to: ParakeetUltraModel.directory,
isComplete: { ParakeetUltraModel.isDownloaded(in: $0) },
forgetting: ParakeetCompileRecord(key: ParakeetUltraTranscriber.compileRecordKey),
fileManager: fileManager
)
// Ultra's files are each checked against their digest as they land, so a complete
// folder from before the marker is complete.
if ParakeetUltraModel.isDownloaded(), !isComplete(ParakeetUltraModel.directory) {
markComplete(ParakeetUltraModel.directory)
}
}
nonisolated static func move(
from old: URL,
to new: URL,
isComplete: (URL) -> Bool,
forgetting record: ParakeetCompileRecord,
fileManager: FileManager
) {
guard old.standardizedFileURL != new.standardizedFileURL,
fileManager.fileExists(atPath: old.path) else { return }
if fileManager.fileExists(atPath: new.path) {
guard !isComplete(new), isComplete(old) else {
try? fileManager.removeItem(at: old)
return
}
try? fileManager.removeItem(at: new)
}
prepareRoot()
try? fileManager.createDirectory(at: new.deletingLastPathComponent(), withIntermediateDirectories: true)
do {
try fileManager.moveItem(at: old, to: new)
record.forget()
print("speech model moved to the App Group: \(new.lastPathComponent)")
} catch {
print("speech model didn't move to the App Group (\(new.lastPathComponent)): \(error)")
}
}
}