hex/ios/HEXShared/FullKeyboard
Repository files (latest commit first)
Filename Latest commit message Latest commit date
Ryan Vogel ef133b8a4f
iOS keyboard: smoother, calmer motion for the live dictation preview (#2424)
* iOS keyboard: the live preview's motion, smooth and calm

What: one spring family (duration and bounce 0, UIKit's springDuration curve) for every
change while dictating, all of it drawn by one view so no memory comes back:
- Keys out, words in as recording starts: the keys fade (.2) as the words fade in rising
  6 pt (.3), the same frame. Words out, keys back as the dictation goes in: the words fade
  drifting 6 pt up toward the field (.15) as the keys fade in (.3); the words stop taking
  touches at once, so the keys work the moment the dictation ends.
- Words arriving (LiveWordsView): each new word fades in rising 4 pt (.3), a burst's words
  40 ms apart and all of them within 240 ms, so the last is in long before Apple's next
  burst (about once a second). "Listening…" fades out a beat (60 ms) ahead of the first words.
- Tentative to settled: the muted tone eases to the keys' ink (.3).
- A new line: the block moves up by a critically damped spring (.3) that carries on from
  where it is when another line comes, and the line leaving the top fades through a 14 pt band
  instead of being cut.
- A changed word ("Okay" to "Hey", "730." to "7.30."): the old word fades out where it was
  (.15) as the new one fades in (.2); words keep their place by position, so the words after
  it never re-fade, and a word that only gains a mark stays as it is.
- The recording bar's morph has no bounce any more (was damping .82) and its fades are the
  same spring (.2), so nothing bounces.
- Reduce Motion: fades only (.2), nothing rises, drifts or slides.
LiveWordsView draws the words with one TextKit 1 layout into one backing store the size of the
box, from a display link that runs only while something moves, and drops whole lines well out
of view so the lines that stay never re-break. It replaces the preview's label.

Why: Ryan asked for smoother motion with better easing after watching the phone recordings
(keys and words swapped in 180 ms with a double exposure; a burst's words popped in at once).

Verified: LiveDictationTests (10) on the Studio simulator; the Studio simulator with the 60 s
replay: footprint +2.7 MB at most while recording (+5.7 with #2410's label), 1,868 frames
drawn while moving, none late; frames of a recording checked for the start, a burst, a new
line and the end. code-shape.py passes.

* iOS keyboard: the preview's words move by Core Animation, not a display link

What: LiveWordsView no longer redraws every frame. It lays the words out and draws them once
per update, in one layer, and Core Animation does every frame of the motion: what changed
cross-fades in place (0.3 s, the cubic closest to a no-bounce spring: new words, a settled
tail turning to ink, a word the recogniser changed), the block moves up by whole lines with a
no-bounce spring (0.3 s) inside a box whose 14 pt band fades the line leaving the top, and once
at rest that line fades out of the band and the lines above the box are dropped from the layout
with nothing on screen changing. The per-word stagger and rise are gone.

Why: on the iPhone 17 Pro the display link drawing the words got only a quarter of its frames
on time while recording (intervals 12-20 ms 26%, 20-30 ms 23%, 30-45 ms 27%, longer 24%; the
drawing itself took 0.3 ms): the main thread is shared with the capture, the recogniser and the
Dot Scope. Core Animation's own animations keep the display's pace regardless.

Verified: LiveDictationTests (9, including whole-line scrolling and dropping the lines above the
box) and recordings on the Studio simulator: a burst, settling, a new line with the band's fade,
the drop at rest, the start and the end; 60 s replay +5.9 MB at most against +7.0 MB for the
same recording without the preview; 1.4 ms of main thread per update.

* iOS README: the live preview's motion, as it now runs

What: the live-words bullet in ios/README.md describes the motion as it ships: one family (a
no-bounce spring by its duration, and its closest cubic for the cross-fade), Core Animation
running every frame, what changes cross-fading in place while the rest stays exactly as it is,
whole-line scrolling through a 14 pt band, and Reduce Motion as cross-fades only. Replaces the
paragraph about per-word staggers and a display link, which the previous commit removed.

Why: the README is where the keyboard's behaviour is written down for the next person.

Verified on the iPhone 17 Pro with 9869899c installed, warm (Parakeet compiled before measuring):
- 60 s dictation, preview on: 59.1 MB at the end, 62.9 MB peak while recording, 76.3 MB at
  transcription (peak 77.3); #2410 was 60.1 / 66.5 / 78.7. Preview off: 58.3 / 60.0 / 75.3.
- The preview's main-thread share: 1.69 ms per update on average, 3.40 ms at most (242 updates).
- In the 30 fps phone recordings, words that didn't change hold the same brightness to the
  decimal through every cross-fade near them; a changing line eases over about 0.2 s with no
  stall; the field's text appears on the frame the preview's words go.

Correction to 9869899c's message: its "+7.0 MB for the same recording without the preview" on
the simulator wasn't measured (the simulator's memory script only runs with the preview); the
phone numbers above are the comparison.
2026-11-22 14:49:09 -05:00
..
Emoji iOS keyboard: stop leaking views, capture web views, and emoji indices across shows (#2352) 2026-11-22 06:45:38 -05:00
AccentPicker.swift ios: quick accents experiment (#2398) 2026-11-22 12:43:34 -05:00
AdaptiveTouch.swift ios: add Dvorak punctuation keys experiment (#2373) 2026-11-22 09:31:25 -05:00
AdaptiveTouchLabels.swift iOS: Adaptive touch, personal key centres learned on device (Experiment, off by default) (#2146) 2026-11-21 06:13:04 +00:00
AutocorrectEngine+Accents.swift iOS: the next word gives the word before its language's accents (#2231) 2026-11-21 14:35:35 -05:00
AutocorrectEngine+Dictionaries.swift iOS keyboard: a word of your own in capitals is offered, never put in for a lowercase one; "i'm" takes its capital (#2391) 2026-11-22 12:05:06 -05:00
AutocorrectEngine+Distance.swift iOS keyboard: a word of your own in capitals is offered, never put in for a lowercase one; "i'm" takes its capital (#2391) 2026-11-22 12:05:06 -05:00
AutocorrectEngine+Vocabulary.swift iOS: words mid-sentence stay lowercase unless you capitalized them (#2218) 2026-11-21 14:35:10 -05:00
AutocorrectEngine.swift iOS keyboard: French reads Hex's own word list and capitals, not the phone's dictionary (#2404) 2026-11-22 13:34:40 -05:00
CzechTyping.swift iOS keyboard: Canadian and Swiss French get Apple's apostrophe key ending row 2 (#2356) 2026-11-22 05:36:52 -05:00
DeleteWordSwipe.swift iOS keyboard: Korean two-set (두벌식) layout with Hangul composition (experiment, off by default) (#2339) 2026-11-22 04:39:53 -05:00
DictationUndo.swift iOS: Undo dictation works after the cursor moves, and goes when it can't (#2240) 2026-11-21 15:15:23 -05:00
FullKeyboardEngine+Field.swift iOS keyboard: a swipe right after its word was deleted types its next reading (#2422) 2026-11-22 14:40:55 -05:00
FullKeyboardEngine+Hangul.swift iOS keyboard: Korean two-set (두벌식) layout with Hangul composition (experiment, off by default) (#2339) 2026-11-22 04:39:53 -05:00
FullKeyboardEngine+HostText.swift iOS keyboard: Korean two-set (두벌식) layout with Hangul composition (experiment, off by default) (#2339) 2026-11-22 04:39:53 -05:00
FullKeyboardEngine+Keys.swift ios: add Dvorak punctuation keys experiment (#2373) 2026-11-22 09:31:25 -05:00
FullKeyboardEngine+Revert.swift iOS: avoid double spaces when editing a word mid-sentence (#2214) 2026-11-21 12:43:26 -05:00
FullKeyboardEngine+Selection.swift iOS keyboard: Korean two-set (두벌식) layout with Hangul composition (experiment, off by default) (#2339) 2026-11-22 04:39:53 -05:00
FullKeyboardEngine+SmartSpace.swift iOS keyboard: a late host echo keeps the swipe's pending space (#2273) 2026-11-21 19:14:42 -05:00
FullKeyboardEngine+SwipeTyping.swift iOS keyboard: a swipe right after its word was deleted types its next reading (#2422) 2026-11-22 14:40:55 -05:00
FullKeyboardEngine+Trackpad.swift iOS keyboard: Korean two-set (두벌식) layout with Hangul composition (experiment, off by default) (#2339) 2026-11-22 04:39:53 -05:00
FullKeyboardEngine.swift iOS keyboard: a swipe right after its word was deleted types its next reading (#2422) 2026-11-22 14:40:55 -05:00
FullKeyboardPalette.swift iOS keyboard: add a keyboard themes experiment (#2365) 2026-11-22 08:24:11 -05:00
FullKeyboardSettings.swift iOS keyboard: French reads Hex's own word list and capitals, not the phone's dictionary (#2404) 2026-11-22 13:34:40 -05:00
FullKeyboardTouch.swift iOS: a quick space tap that drifts still types a space, and space keeps its place in a rollover (#2233) 2026-11-21 14:36:17 -05:00
FullKeyboardView+Debug.swift iOS keyboard: Korean two-set (두벌식) layout with Hangul composition (experiment, off by default) (#2339) 2026-11-22 04:39:53 -05:00
FullKeyboardView+Emoji.swift iOS keyboard: stop leaking views, capture web views, and emoji indices across shows (#2352) 2026-11-22 06:45:38 -05:00
FullKeyboardView+FullAccess.swift iOS: tappable Paste from Other Apps notice and app Settings card (#2394) 2026-11-22 11:13:25 -08:00
FullKeyboardView+Selection.swift iOS: tappable Paste from Other Apps notice and app Settings card (#2394) 2026-11-22 11:13:25 -08:00
FullKeyboardView+Settings.swift iOS keyboard: Experiments › Comma and period keys beside the space bar (#2392) 2026-11-22 12:11:21 -05:00
FullKeyboardView+Vocabulary.swift iOS: Czech keyboard, and custom words for typing (#2205) 2026-11-21 10:37:55 -05:00
FullKeyboardView.swift iOS keyboard: smoother, calmer motion for the live dictation preview (#2424) 2026-11-22 14:49:09 -05:00
GermanCapitals.swift iOS keyboard: French reads Hex's own word list and capitals, not the phone's dictionary (#2404) 2026-11-22 13:34:40 -05:00
GreekTyping.swift iOS keyboard: a late host echo keeps the swipe's pending space (#2273) 2026-11-21 19:14:42 -05:00
HangulComposer.swift iOS keyboard: Korean two-set (두벌식) layout with Hangul composition (experiment, off by default) (#2339) 2026-11-22 04:39:53 -05:00
HexKeyGesture.swift iOS: the keyboard starts listening on touch-down, with its capture page ready when it opens (#2089) 2026-11-20 20:04:35 +00:00
HexKeyViews.swift iOS keyboard: remove the "Logging" mark; Record typing logs is only the switch (#2382) 2026-11-22 10:50:35 -05:00
KeyboardBackdropView.swift iOS keyboard: add a keyboard themes experiment (#2365) 2026-11-22 08:24:11 -05:00
KeyboardClipboard.swift iOS: tappable Paste from Other Apps notice and app Settings card (#2394) 2026-11-22 11:13:25 -08:00
KeyboardKeyPlaneView.swift ios: add Dvorak punctuation keys experiment (#2373) 2026-11-22 09:31:25 -05:00
KeyboardLanguage.swift iOS keyboard: English and German together type on the first one's keys, as Apple's bilingual keyboard (#2402) 2026-11-22 12:55:14 -05:00
KeyboardLaunchWork.swift iOS: the dictation bar can start dictating by itself (Auto-start: never, on switch, always) (#2267) 2026-11-21 17:08:50 -05:00
KeyboardLayout+CommaPeriod.swift iOS keyboard: Experiments › Comma and period keys beside the space bar (#2392) 2026-11-22 12:11:21 -05:00
KeyboardLayout.swift iOS keyboard: the gap above row 4 is the space bar's only over the space bar (#2395) 2026-11-22 12:24:54 -05:00
KeyboardPadRows.swift iOS keyboard: Canadian and Swiss French get Apple's apostrophe key ending row 2 (#2356) 2026-11-22 05:36:52 -05:00
KeyboardStripView.swift iOS keyboard: Code from Messages opens the keyboard list where the switch could land on Emoji, and needs no Full Access (#2384) 2026-11-22 12:02:17 -05:00
KeyboardTheme.swift iOS keyboard: add a keyboard themes experiment (#2365) 2026-11-22 08:24:11 -05:00
LanguageLearning.swift ios: learn each language separately (experiment) (#2188) 2026-11-21 09:00:58 -05:00
never-suggest.tsv iOS: never suggest slurs, and swear words only when asked (#2093) 2026-11-20 20:17:31 +00:00
OffensiveWords.swift iOS: never suggest slurs, and swear words only when asked (#2093) 2026-11-20 20:17:31 +00:00
ReturnKeyStyle.swift iOS: Return follows the app's return key type, and the dictation bar's does too (#2200) 2026-11-21 09:54:57 -05:00
ShiftStateMachine.swift iOS keyboard: the dictation bar's globe opens the keyboard list where Apple's keyboard is split, so the host doesn't crash (#2417) 2026-11-22 14:13:11 -05:00
SmartPunctuation.swift iOS keyboard: a late host echo keeps the swipe's pending space (#2273) 2026-11-21 19:14:42 -05:00
SwipeDecoder.swift iOS: swipe reads the letters under the path, and the emoji bar scrubs (#2189) 2026-11-21 08:47:45 -05:00
SwipeTyping.swift iOS keyboard: a word of your own in capitals is offered, never put in for a lowercase one; "i'm" takes its capital (#2391) 2026-11-22 12:05:06 -05:00
TextChecking.swift iOS keyboard: French reads Hex's own word list and capitals, not the phone's dictionary (#2404) 2026-11-22 13:34:40 -05:00
TrackpadCursor.swift iOS: the space-bar trackpad moves like Apple's (#2219) 2026-11-21 13:23:46 -05:00
TrackpadSelection.swift iOS: two-thumb selection on the space bar (experiment, off by default) (#2265) 2026-11-21 20:44:15 -05:00
TrackpadSelectionStrip.swift iOS: two-thumb selection on the space bar (experiment, off by default) (#2265) 2026-11-21 20:44:15 -05:00
TypingDecoder.swift ios keyboard: read a missed space with a slip, and run-together words (#2244) 2026-11-21 16:15:02 -05:00
TypingJournal.swift iOS: hold the bar to dictate, Learn how I type on, a plain centred Paste, and typing logs off (#2210) 2026-11-21 12:11:58 -05:00
TypingJournalLearning.swift ios: learn each language separately (experiment) (#2188) 2026-11-21 09:00:58 -05:00