mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-22 21:23:02 +01:00
* iOS: retry '!int' audio activation mixable and show audio errors as sentences
TestFlight testers saw "The operation couldn't be completed. (OSStatus error
560557684.)" in the keyboard strip and the app's setup card. 560557684 is
'!int' (AVAudioSession cannotInterruptOthers): a non-mixable session (the
Bluetooth-microphone .record category) going active from the background while
another app plays or records.
- AudioBackend: when setActive(true) fails with '!int', set .playAndRecord
with .mixWithOthers (keeping HFP when Bluetooth is allowed) and activate
once more, so the warm keepalive and captures start alongside the other
app's audio instead of failing.
- New UserFacingError (HEXShared, app and keyboard): AVAudioSession codes map
to short sentences ("Another app is using audio. Try again."); other
microphone errors get "Hex couldn't start the microphone. Try again.";
transcription and model errors keep their description only when it reads
as a sentence, never Foundation's "couldn't be completed (… error N)" or a
Swift type name.
- DictationController: every error it shows (phase .failed and the
keyboard's message) goes through UserFacingError instead of
localizedDescription.
Verified: xcodebuild test on the iPhone 18 Pro simulator, 179 tests pass,
including new tests for the code-to-sentence mapping and for a warm session
whose keepalive throws '!int' reaching the app and keyboard snapshot as
"Another app is using audio. Try again.". The background '!int' retry itself
needs a physical iPhone (music playing, Bluetooth mic allowed, warm session).
* iOS: end empty dictations quietly and keep error text human in the keyboard
Tapping Done (or ✕) in the keyboard's dictation screen without speaking
showed "Transcription failed: HEXKeyboard.(unknown context at $1…)": the
in-keyboard path printed the error's domain (a private Swift type name),
code and description for Apple Speech's "no words" result.
- New EmptyCapture rule (HEXShared): a capture under 0.3 s (or none at all)
is a tap, and an engine's no-speech error or a cancellation means nothing
was said; both end quietly. Engines say so through NoSpeechError
(Parakeet's and Apple Speech's emptyTranscript, the keyboard's
KeyboardSpeech.Failure.empty).
- KeyboardLocalDictation: Done before the microphone came up, or a capture
too short to hold a word, returns to idle without transcribing (and a late
"started" from WebKit stops the capture instead of opening a file);
no-speech ends quietly; a transcription error from a cancelled or
superseded job is dropped; every other failure shows a sentence from
UserFacingError or the keyboard's own copy, while the probe log keeps the
domain and code. KeyboardSpeech.Failure's descriptions are now sentences.
- DictationController (warm session and app): an engine's no-speech error
returns to Ready with no message instead of failing.
Verified: xcodebuild test on the iPhone 18 Pro simulator after rebasing onto
origin/dev (bba07c64), 210 tests pass (1 skipped as before), including new
tests for the empty-capture rule and for a warm keyboard dictation whose
transcriber hears nothing (Ready, no message) next to one that fails (a
sentence). The in-keyboard WebKit capture path needs a physical iPhone.
* iOS: start Live Activities only for system dictations and end them reliably
Testers saw Hex's activity (dots + ✓, expanded "Copied") stay in the
Dynamic Island, even with Keep mic warm Off. The ✓/Copied look is the
widget's stale rendering of an activity nobody ended: every recording
(keyboard warm dictations and in-app ones included) requested an activity,
and the end ran in a detached task that a suspending app (microphone just
released) could skip, so it went stale and stayed for hours.
- DictationController requests an activity only for dictations that copy
(Action Button, Control Center, Siri/Shortcuts); the keyboard and the app
show their own recording UI. Removed: the activity for keyboard and in-app
dictations.
- LiveActivityCoordinator: ending (cancel, failure, short capture, warm
session cancel) ends every Hex activity there is at that moment with
.immediate; Copied still lingers 2 s via .after. Ends run under a UIKit
background task, and a dictation that is transcribing keeps one until its
activity ends. A current activity the system already ended is replaced
rather than updated.
- Stale activities end at launch (DictationController.shared init), when Hex
becomes active, and when the warm session ends or Keep mic warm is set to
Off, unless a dictation is recording or transcribing.
- README: when Live Activities show and end.
Verified: xcodebuild test on the iPhone 18 Pro simulator, 210 tests pass
(1 skipped as before). A temporary, uncommitted test on the simulator showed
no activity for a warm keyboard dictation, one for a copy dictation that
ended after Copied, one ended at once on cancel, and an orphan activity
ended by Keep mic warm › Off. The Dynamic Island itself, and background
suspension timing, need a physical iPhone.
* iOS: give the dictation bar's keys the full keyboard's light haptic
The full keyboard taps under the finger (Experiments › Haptics, with Full
Access); the dictation bar's keys had no haptic at all.
- KeyboardViewModel gains onKeyFeedback; every control on the dictation bar
calls it: Hex mark, Dictate / Set up, ✕ and Done, Swipe and its return-app
chips, the style and language triggers and chips, the tray's close button,
space, return, and delete (each repeat, as the full keyboard's delete).
- KeyboardViewController plays a light impact at the full keyboard's 0.65
intensity, under the same rule the full keyboard now shares
(allowsKeyHaptics: Full Access and FullKeyboardSettings().haptics).
- Experiments: the Haptics switch moves out of the Full keyboard group so it
shows whether or not the full keyboard is on; same key
(settings.keyboardHaptics), subtitle now says it covers Hex's keys.
Verified: xcodebuild test on the iPhone 18 Pro simulator, 210 tests pass (1
skipped as before); Experiments rendered with --snap-state experiments shows
Haptics after Keyboard layout, outside the Full keyboard rows. The haptic
itself (and its Full Access gate) needs a physical iPhone; the simulator has
no Taptic Engine.
* iOS: lowercase a dictation that continues a sentence
A tester typed a few words, dictated, and the dictation started with a
capital as if it began a sentence: speech engines capitalize every result.
- TranscriptFormatter.startsSentence(after:): the text before the cursor ends
a sentence when it is empty or only spaces, or its last mark is . ! ? … 。
or a line break (closing quotes and brackets after them count too).
- TranscriptFormatter.continuingSentence: mid-sentence, the first letter goes
lowercase unless the first word is "I" or an I' contraction, all capitals
(2+ letters), has capitals inside (McDonald's), or its lowercase form isn't
in the dictation language's dictionary (likely a name).
- insertionText takes that dictionary (isWord; nil keeps the case), never
adds a second space after one already there (spaces at the start of the
dictation are dropped), and adds none after an opening ( [ { " “ ‘ « ¿ ¡.
- KeyboardViewController (the one place both keyboards insert a dictation)
passes a UITextChecker for the dictation language (Apple Speech's last
resolved language, else Settings' or Automatic's, else the first
preferred language), mapped by KeyboardLanguage.spellCheckLanguage(
forDictation:available:). German and Luxembourgish keep the case: on the
simulator de_DE accepts "auto" and "haus", so the name guard can't protect
their nouns. A language with no dictionary keeps the case too.
- README: the rule.
Verified: xcodebuild test on the iPhone 18 Pro simulator, 213 tests pass (1
skipped as before), with new tests for the sentence-start rule, the
exceptions, spacing, and the language-to-dictionary mapping. A temporary,
uncommitted test on the simulator's real UITextChecker found "sarah" and
"london" unknown and "the"/"that's" known in en_US, es and de_DE. How each
host app reports documentContextBeforeInput is unverified on a device.
* iOS: push to talk on the Hex key and the dictation bar's Dictate
A tester asked to hold to talk instead of tapping Dictate and then Done.
Rule: holding still for 0.35 s (HexKeyGesture.holdDuration, the Hex key's
existing hold) starts dictation at once, and lifting the finger finishes it
as Done does. A quick tap keeps tap to start, Done to finish. On the Hex key,
sliding more than 10 pt before the hold fires still opens the style menu;
once dictation has started, moving changes nothing. Push to talk applies
only when dictation runs in the keyboard (warm session or in-keyboard
dictation); a hold that opens Hex stays a tap.
- HexKeyGesture: a fired hold's touch-up is now .release (was ignored).
Changed: before, a hold started dictation and the release did nothing.
- HexKeyControl calls onHoldRelease; FullKeyboardView finishes the dictation
the hold started (model.onStopTap, logged as dictation_stop from
hex_key_release). The recording fade keeps a still-tracking Hex key in
the window at alpha 0 instead of hiding it, so its touch keeps arriving.
- KeyboardSurfaceView: the well tracks the press with a simultaneous
DragGesture on the same state machine (the Dictate button stops taking
touches once recording shows), starts with the light haptic
(onKeyFeedback: Haptics setting and Full Access), and finishes on release;
the button's own touch-up action skips a press the hold took.
- Accessibility hints mention holding; README and docs/typing-logs.md
describe it.
Verified: xcodebuild test on the iPhone 18 Pro simulator, 213 tests pass (1
skipped as before), including the updated HexKeyGesture tests (hold then
release finishes; moving after the hold is ignored). The dictation bar
renders as before (--snap-state keyboard-warm). Real touches (hold timing,
the bar's buttons with the well's gesture, the Hex key's touch surviving the
recording panel) were not driven on the simulator and need a physical
iPhone.
170 lines
5.9 KiB
Swift
170 lines
5.9 KiB
Swift
import ActivityKit
|
|
import Foundation
|
|
import UIKit
|
|
|
|
/// The Dynamic Island and Lock Screen activity of a dictation started by the Action Button,
|
|
/// Control Center, Siri, or Shortcuts. Ending one also ends every other Hex activity, so none
|
|
/// outlives its dictation; each end runs under a background task, so a suspending app can't
|
|
/// leave one behind.
|
|
@MainActor
|
|
final class LiveActivityCoordinator {
|
|
private var currentActivityID: String?
|
|
/// Holds the app awake from transcribing until the activity ends: the microphone, which
|
|
/// kept it running in the background, is already released.
|
|
private var transcribingAssertion: BackgroundAssertion?
|
|
|
|
func startRecording(startedAt: Date) {
|
|
guard ActivityAuthorizationInfo().areActivitiesEnabled else { return }
|
|
let state = HexDictationActivityAttributes.ContentState(
|
|
stage: .recording,
|
|
startedAt: startedAt,
|
|
subtitle: "Recording on this iPhone"
|
|
)
|
|
if let id = currentActivityID, Self.isActive(id) {
|
|
Task.detached {
|
|
await Self.updateActivity(id: id, state: state, staleDate: startedAt.addingTimeInterval(90))
|
|
}
|
|
return
|
|
}
|
|
|
|
let attributes = HexDictationActivityAttributes(sessionID: UUID().uuidString)
|
|
if let created = try? Activity.request(
|
|
attributes: attributes,
|
|
content: ActivityContent(state: state, staleDate: startedAt.addingTimeInterval(90)),
|
|
pushType: nil
|
|
) {
|
|
currentActivityID = created.id
|
|
}
|
|
}
|
|
|
|
func markTranscribing(startedAt: Date) {
|
|
guard let id = currentActivityID else { return }
|
|
if transcribingAssertion == nil {
|
|
transcribingAssertion = BackgroundAssertion(name: "Hex transcription")
|
|
}
|
|
let state = HexDictationActivityAttributes.ContentState(
|
|
stage: .transcribing,
|
|
startedAt: startedAt,
|
|
subtitle: "Transcribing locally…"
|
|
)
|
|
Task.detached {
|
|
await Self.updateActivity(id: id, state: state, staleDate: Date().addingTimeInterval(30))
|
|
}
|
|
}
|
|
|
|
/// Shows Copied for 2 s, then the activity goes; any other Hex activity ends now.
|
|
func finishCopied() {
|
|
guard let id = currentActivityID else {
|
|
endImmediately()
|
|
return
|
|
}
|
|
currentActivityID = nil
|
|
let others = Self.activityIDs().filter { $0 != id }
|
|
let state = HexDictationActivityAttributes.ContentState(
|
|
stage: .copied,
|
|
startedAt: Date(),
|
|
subtitle: "Copied to clipboard"
|
|
)
|
|
runEnding {
|
|
await Self.endActivities(ids: others)
|
|
await Self.endActivity(
|
|
id: id,
|
|
state: state,
|
|
staleDate: Date().addingTimeInterval(4),
|
|
dismissAfterSeconds: 2
|
|
)
|
|
}
|
|
}
|
|
|
|
/// Ends every Hex activity there is now, the current one and any a suspended or
|
|
/// killed process left behind. One a later dictation starts is not touched.
|
|
func endImmediately() {
|
|
currentActivityID = nil
|
|
let ids = Self.activityIDs()
|
|
guard !ids.isEmpty else {
|
|
transcribingAssertion?.end()
|
|
transcribingAssertion = nil
|
|
return
|
|
}
|
|
runEnding {
|
|
await Self.endActivities(ids: ids)
|
|
}
|
|
}
|
|
|
|
private func runEnding(_ work: @escaping @Sendable () async -> Void) {
|
|
let assertion = BackgroundAssertion(name: "Hex Live Activity end")
|
|
let transcribing = transcribingAssertion
|
|
transcribingAssertion = nil
|
|
Task { @MainActor in
|
|
await work()
|
|
assertion.end()
|
|
transcribing?.end()
|
|
}
|
|
}
|
|
|
|
private static func activityIDs() -> [String] {
|
|
Activity<HexDictationActivityAttributes>.activities.map(\.id)
|
|
}
|
|
|
|
private static func isActive(_ id: String) -> Bool {
|
|
Activity<HexDictationActivityAttributes>.activities.contains { $0.id == id && $0.activityState == .active }
|
|
}
|
|
|
|
nonisolated private static func updateActivity(
|
|
id: String,
|
|
state: HexDictationActivityAttributes.ContentState,
|
|
staleDate: Date?
|
|
) async {
|
|
guard let activity = Activity<HexDictationActivityAttributes>.activities.first(where: { $0.id == id }) else {
|
|
return
|
|
}
|
|
await activity.update(ActivityContent(state: state, staleDate: staleDate))
|
|
}
|
|
|
|
nonisolated private static func endActivity(
|
|
id: String,
|
|
state: HexDictationActivityAttributes.ContentState,
|
|
staleDate: Date?,
|
|
dismissAfterSeconds: TimeInterval
|
|
) async {
|
|
guard let activity = Activity<HexDictationActivityAttributes>.activities.first(where: { $0.id == id }) else {
|
|
return
|
|
}
|
|
await activity.end(
|
|
ActivityContent(state: state, staleDate: staleDate),
|
|
dismissalPolicy: .after(Date().addingTimeInterval(dismissAfterSeconds))
|
|
)
|
|
}
|
|
|
|
nonisolated private static func endActivities(ids: [String]) async {
|
|
let state = HexDictationActivityAttributes.ContentState(
|
|
stage: .copied,
|
|
startedAt: Date(),
|
|
subtitle: "Done"
|
|
)
|
|
for activity in Activity<HexDictationActivityAttributes>.activities where ids.contains(activity.id) {
|
|
await activity.end(
|
|
ActivityContent(state: state, staleDate: nil),
|
|
dismissalPolicy: .immediate
|
|
)
|
|
}
|
|
}
|
|
}
|
|
|
|
/// A UIKit background task, ended once (or by the system's expiry).
|
|
@MainActor
|
|
private final class BackgroundAssertion {
|
|
private var id = UIBackgroundTaskIdentifier.invalid
|
|
|
|
init(name: String) {
|
|
id = UIApplication.shared.beginBackgroundTask(withName: name) { [weak self] in
|
|
MainActor.assumeIsolated { self?.end() }
|
|
}
|
|
}
|
|
|
|
func end() {
|
|
guard id != .invalid else { return }
|
|
UIApplication.shared.endBackgroundTask(id)
|
|
id = .invalid
|
|
}
|
|
}
|