mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-22 21:23:02 +01:00
A TestFlight tester (build 120, iPhone 15 Pro Max, iOS 26.6) had Experiments ›
Multilingual model (Parakeet) on: voice input "started demanding Set up", then
sat at "Downloading 632.2 MB · Wi-Fi recommended" with no movement.
Why it blocked: with the experiment on and the dictation language one of
Parakeet Ultra's 25 (English included), Ultra became Hex's engine at once,
before it was downloaded. Hex's phase stayed on preparing (or failed, which
shows Set up) for the whole 632 MB download and the first Core ML compile, and
the keyboard's Dictate and Hex key opened Hex for it whether or not the model
was there, so nothing dictated anywhere until it finished.
Why it sat still: the download ran in prepare() on a foreground URLSession,
one 594 MB encoder file, shown only as a bar at 70% scale under a label that
never changed. Leaving Hex (the keyboard's handoff sends you back to the app you
type in) or Auto-Lock suspends it; on return a dead connection showed nothing
for up to 60-90 s, then failed to Set up.
What changed:
- Until Ultra is ready, Hex and the keyboard dictate with the engine they use
with the experiment off. ParakeetUltraDownload (new) downloads beside it and
ParakeetUltraTranscriber.prepare only loads (throws notDownloaded otherwise).
A first load (Core ML's compile) runs beside the usual engine
(DictationController.loadMultilingualParakeetBeside) and switches to Ultra
when loaded; launches load Ultra directly only once downloaded and compiled
for this iOS (multilingualStep). Where V2 dictates and leaves no room
(English before iOS 26), Ultra takes V2's place for that load.
- The keyboard opens Hex for the experiment only once Hex has loaded it
(MultilingualParakeetSettings.isModelReady in the App Group, set after a
load, cleared while missing, set aside or off).
- The download shows bytes ("Downloading the multilingual model: 123.4 MB of
632.2 MB" with a bar) on home and under the switch; resumes every file from its
.partial (launch, retry, return to the front); a drop or 20 s without bytes
tries again after 2, 4, 8, 16, 30 s, starting over whenever it got further; in
the background it waits and picks up at once on return (fresh connection after
25 s away). After 6 fruitless tries, a refused request or a full iPhone it says
what happened in plain words with Try again and Turn off.
- A failed Ultra load or a load that ended Hex (ParakeetLoadGuard) falls back
to the usual engine and says so with Try again (no longer only by turning the
switch off, which deletes the 632 MB).
- A launch deletes a leftover download only when the switch is stored as off
(isTurnedOff), not when the App Group reads empty (a launch before the first
unlock would have deleted a good model). Turning it off now stops the download
and waits for it before deleting.
- Removed: ParakeetUltraTranscriber.downloadDetail, its .downloadFailed error
and the download/StallWatch inside its prepare (replaced by the above);
strings "Downloading %@ · Wi-Fi recommended" and the old switch detail.
New strings have zh-Hans. README describes the experiment's new behaviour.
- DEBUG builds read HEX_ULTRA_MODEL_BASE to download from a stand-in server.
Verified:
- xcodebuild test, iPhone 17 Pro Max iOS 26.5 simulator on the Mac Studio:
753 tests, 0 failures, 3 skipped (new cases in DictationControllerTests:
resume from .partial with Range, drop + stall retries, a moving download never
running out of tries, 503 x6 / 404 failures with their words, stop before
delete, keyboard flag, stored-off rule, multilingualStep).
- iPhone 15 Pro Max iOS 26.5 simulator, experiment on, a local stand-in server
(Range, throttled, drop/stall switches): Dictate stayed available with the
usual engine (V2 here; the sim has no Apple Speech) while the status counted
MB; a dropped connection said "Trying again in 1 s" and resumed with Range; a
20 s stall said so and resumed; Hex killed mid-download resumed from the
partial at relaunch; backgrounded with the connection cut, it resumed at once
on return; after a kill the resumed download from real Hugging Face finished
from a partial and passed the hashes; Ultra loaded beside and took over (setup
sheet "English · Parakeet Ultra", keyboard flag set); a forced load guard
showed the failure with Try again, which loaded Ultra; deleting the model with
the flag set cleared it at launch and downloaded again; the keyboard logged
"multilingual model on, Dictate opens Hex" only when ready (else the sim's own
"speech model missing" reason) and its Hex key went straight to recording
instead of the download screen; turning it off deleted the folder and cleared
the flag; with it off nothing shows or downloads.
- Not verified: a real iPhone, a real slow or flaky network, iOS's own
suspension timing, the V2-replacement path (sims report no memory limit), and
real audio (the Studio has no microphone).
87 lines
4.1 KiB
Swift
87 lines
4.1 KiB
Swift
import Foundation
|
||
|
||
/// Experiments › Multilingual model (Parakeet), in the shared App Group so the keyboard reads
|
||
/// it too. Off by default. While it's on and Hex has the model ready, Hex dictates its 25
|
||
/// languages with Parakeet Ultra (`ParakeetUltraTranscriber`), which hears them mixed within a
|
||
/// sentence (Dutch with English words), and the Hex key hands its dictations to Hex, as before
|
||
/// iOS 26: the model can't load within a keyboard's memory limit. Until it's ready, both
|
||
/// dictate as they would with the experiment off.
|
||
nonisolated struct MultilingualParakeetSettings {
|
||
static let enabledKey = "experiments.multilingualParakeet"
|
||
/// Set by Hex once Parakeet Ultra loaded; cleared while it's missing, set aside or off.
|
||
static let modelReadyKey = "experiments.multilingualParakeet.ready"
|
||
static let launchArgument = "--experiment-multilingual-parakeet"
|
||
|
||
/// Parakeet Ultra's languages (ISO 639-1), those of NVIDIA Parakeet TDT 0.6B v3.
|
||
static let languages: Set<String> = [
|
||
"bg", "cs", "da", "de", "el", "en", "es", "et", "fi", "fr", "hr", "hu", "it",
|
||
"lt", "lv", "mt", "nl", "pl", "pt", "ro", "ru", "sk", "sl", "sv", "uk",
|
||
]
|
||
|
||
/// As Parakeet V2: 6 GB iPhones report about 5.5 GB, 4 GB ones (iPhone 12, 13) under 4 GB.
|
||
static let minimumPhysicalMemory: UInt64 = 5 << 30
|
||
|
||
private let defaults: UserDefaults?
|
||
|
||
init(defaults: UserDefaults? = UserDefaults(suiteName: KeyboardBridge.appGroup)) {
|
||
self.defaults = defaults
|
||
}
|
||
|
||
var isEnabled: Bool {
|
||
get { defaults?.bool(forKey: Self.enabledKey) ?? false }
|
||
nonmutating set { defaults?.set(newValue, forKey: Self.enabledKey) }
|
||
}
|
||
|
||
/// Turned off by the person, as stored: never set reads false here, and so does an App
|
||
/// Group iOS doesn't let Hex read yet (a launch before the iPhone's first unlock), which
|
||
/// must not delete a model that is on.
|
||
var isTurnedOff: Bool {
|
||
defaults?.object(forKey: Self.enabledKey) as? Bool == false
|
||
}
|
||
|
||
/// Whether Hex has Parakeet Ultra loaded at least once since it last downloaded or was let
|
||
/// back in: until then, the keyboard dictates as it would with the experiment off.
|
||
var isModelReady: Bool {
|
||
get { defaults?.bool(forKey: Self.modelReadyKey) ?? false }
|
||
nonmutating set { defaults?.set(newValue, forKey: Self.modelReadyKey) }
|
||
}
|
||
|
||
/// The keyboard's Dictate and Hex keys open Hex for it: on, ready, and the language its.
|
||
func keyboardOpensHex(language: String?) -> Bool {
|
||
isEnabled && isModelReady && language != nil
|
||
}
|
||
|
||
/// Whether this iPhone has the memory to load it at all.
|
||
static func fits(physicalMemory: UInt64 = ProcessInfo.processInfo.physicalMemory) -> Bool {
|
||
physicalMemory >= minimumPhysicalMemory
|
||
}
|
||
|
||
static func supports(_ language: String?) -> Bool {
|
||
guard let language, let code = Locale(identifier: language).language.languageCode?.identifier else {
|
||
return false
|
||
}
|
||
return languages.contains(code)
|
||
}
|
||
|
||
/// The language (BCP 47) Parakeet Ultra dictates in, or nil while it doesn't dictate: the
|
||
/// experiment off, an iPhone without the memory, or a language it doesn't know, which
|
||
/// stays with the engine Hex uses without it. The one set, else the keyboard's language
|
||
/// Automatic follows, else the iPhone's first language.
|
||
func language(
|
||
set: String? = LanguageSettings().dictationLanguage,
|
||
keyboard: String? = LanguageSettings().keyboardDictationLanguage(),
|
||
preferred: [String] = Locale.preferredLanguages,
|
||
physicalMemory: UInt64 = ProcessInfo.processInfo.physicalMemory
|
||
) -> String? {
|
||
guard isEnabled, Self.fits(physicalMemory: physicalMemory) else { return nil }
|
||
guard let wanted = set ?? keyboard ?? preferred.first else { return nil }
|
||
return Self.supports(wanted) ? wanted : nil
|
||
}
|
||
|
||
/// Simulator scenes pass `--experiment-multilingual-parakeet` to turn the experiment on.
|
||
func applyLaunchArguments(_ arguments: [String] = ProcessInfo.processInfo.arguments) {
|
||
if arguments.contains(Self.launchArgument) {
|
||
isEnabled = true
|
||
}
|
||
}
|
||
}
|