hex/ios/README.md
Ryan Vogel ef133b8a4f
iOS keyboard: smoother, calmer motion for the live dictation preview (#2424)
* iOS keyboard: the live preview's motion, smooth and calm

What: one spring family (duration and bounce 0, UIKit's springDuration curve) for every
change while dictating, all of it drawn by one view so no memory comes back:
- Keys out, words in as recording starts: the keys fade (.2) as the words fade in rising
  6 pt (.3), the same frame. Words out, keys back as the dictation goes in: the words fade
  drifting 6 pt up toward the field (.15) as the keys fade in (.3); the words stop taking
  touches at once, so the keys work the moment the dictation ends.
- Words arriving (LiveWordsView): each new word fades in rising 4 pt (.3), a burst's words
  40 ms apart and all of them within 240 ms, so the last is in long before Apple's next
  burst (about once a second). "Listening…" fades out a beat (60 ms) ahead of the first words.
- Tentative to settled: the muted tone eases to the keys' ink (.3).
- A new line: the block moves up by a critically damped spring (.3) that carries on from
  where it is when another line comes, and the line leaving the top fades through a 14 pt band
  instead of being cut.
- A changed word ("Okay" to "Hey", "730." to "7.30."): the old word fades out where it was
  (.15) as the new one fades in (.2); words keep their place by position, so the words after
  it never re-fade, and a word that only gains a mark stays as it is.
- The recording bar's morph has no bounce any more (was damping .82) and its fades are the
  same spring (.2), so nothing bounces.
- Reduce Motion: fades only (.2), nothing rises, drifts or slides.
LiveWordsView draws the words with one TextKit 1 layout into one backing store the size of the
box, from a display link that runs only while something moves, and drops whole lines well out
of view so the lines that stay never re-break. It replaces the preview's label.

Why: Ryan asked for smoother motion with better easing after watching the phone recordings
(keys and words swapped in 180 ms with a double exposure; a burst's words popped in at once).

Verified: LiveDictationTests (10) on the Studio simulator; the Studio simulator with the 60 s
replay: footprint +2.7 MB at most while recording (+5.7 with #2410's label), 1,868 frames
drawn while moving, none late; frames of a recording checked for the start, a burst, a new
line and the end. code-shape.py passes.

* iOS keyboard: the preview's words move by Core Animation, not a display link

What: LiveWordsView no longer redraws every frame. It lays the words out and draws them once
per update, in one layer, and Core Animation does every frame of the motion: what changed
cross-fades in place (0.3 s, the cubic closest to a no-bounce spring: new words, a settled
tail turning to ink, a word the recogniser changed), the block moves up by whole lines with a
no-bounce spring (0.3 s) inside a box whose 14 pt band fades the line leaving the top, and once
at rest that line fades out of the band and the lines above the box are dropped from the layout
with nothing on screen changing. The per-word stagger and rise are gone.

Why: on the iPhone 17 Pro the display link drawing the words got only a quarter of its frames
on time while recording (intervals 12-20 ms 26%, 20-30 ms 23%, 30-45 ms 27%, longer 24%; the
drawing itself took 0.3 ms): the main thread is shared with the capture, the recogniser and the
Dot Scope. Core Animation's own animations keep the display's pace regardless.

Verified: LiveDictationTests (9, including whole-line scrolling and dropping the lines above the
box) and recordings on the Studio simulator: a burst, settling, a new line with the band's fade,
the drop at rest, the start and the end; 60 s replay +5.9 MB at most against +7.0 MB for the
same recording without the preview; 1.4 ms of main thread per update.

* iOS README: the live preview's motion, as it now runs

What: the live-words bullet in ios/README.md describes the motion as it ships: one family (a
no-bounce spring by its duration, and its closest cubic for the cross-fade), Core Animation
running every frame, what changes cross-fading in place while the rest stays exactly as it is,
whole-line scrolling through a 14 pt band, and Reduce Motion as cross-fades only. Replaces the
paragraph about per-word staggers and a display link, which the previous commit removed.

Why: the README is where the keyboard's behaviour is written down for the next person.

Verified on the iPhone 17 Pro with 9869899c installed, warm (Parakeet compiled before measuring):
- 60 s dictation, preview on: 59.1 MB at the end, 62.9 MB peak while recording, 76.3 MB at
  transcription (peak 77.3); #2410 was 60.1 / 66.5 / 78.7. Preview off: 58.3 / 60.0 / 75.3.
- The preview's main-thread share: 1.69 ms per update on average, 3.40 ms at most (242 updates).
- In the 30 fps phone recordings, words that didn't change hold the same brightness to the
  decimal through every cross-fade near them; a changing line eases over about 0.2 s with no
  stall; the field's text appears on the frame the preview's words go.

Correction to 9869899c's message: its "+7.0 MB for the same recording without the preview" on
the simulator wasn't measured (the simulator's memory script only runs with the preview); the
phone numbers above are the comparison.
2026-11-22 14:49:09 -05:00

183 KiB
Raw Permalink Blame History

Hex for iPhone

Status: Internal TestFlight app (HEXMobile), custom keyboard extension (HEXKeyboard), and Live Activity + Control Center widget extension (HEXWidgets). The containing app records a generation-scoped temporary WAV, transcribes on-device with Apple Speech on iOS 26+ (SpeechTranscriber, or DictationTranscriber for the languages it lacks, such as Norwegian, Danish, Swedish and Hebrew) or FluidAudio Parakeet V2 (iOS 18+, English). Before iOS 26, every other language, and English on 3 GB iPhones (iPhone XR, SE 2nd generation) or after Parakeet couldn't load, goes to Apple's SFSpeechRecognizer on device only (OnDeviceSpeechTranscriber, chosen by SpeechEngineChoice). Hex removes each capture's file after inference, cancellation, or failure. During a warm session the keyboard extension sends commands and inserts completed text over the shared App Group (group.anomaly.hex.mobile). Otherwise it dictates by itself (in-keyboard dictation, iOS 26+): iOS refuses to record from a keyboard extension directly, so it captures through getUserMedia in a hidden WKWebView under Hex's microphone permission, writes a temporary WAV, and transcribes it on-device with Apple Speech, downloading the speech assets on first use (HEXKeyboard/KeyboardLocalDictation.swift), or, with Experiments › Speech model in keyboard on, with Parakeet on the Neural Engine in the keyboard's own process (HEXKeyboard/KeyboardParakeet.swift, below). With Full Access, the web view and its page load as the keyboard shows (unless memory is already short), without the microphone, so no orange dot shows until you dictate; the full keyboard's Hex key then starts capture as the finger lands, before the touch is a tap or a hold, and a slide to the style menu or a cancelled touch drops it with nothing transcribed (its orange dot shows for that moment). iOS ends a keyboard that passes its memory limit (about 60–90 MB depending on the model), so while less is left than starting needs (os_proc_available_memory: 32 MB for the first dictation, 24 MB once the capture web view is loaded) Dictate and the Hex key open Hex instead, as before iOS 26. Audio never leaves the iPhone. From iOS 26 Hex never asks for Speech Recognition access: that permission covers SFSpeechRecognizer, which Hex uses only before iOS 26, and SpeechAnalyzer runs without it in the app and the keyboard alike (a TestFlight tester who turned it off dictated, iOS 27.0.1, build 51). Before iOS 26 Hex asks for it as it gets SFSpeechRecognizer ready (setup, or Set up), never at a launch by itself.

Capabilities

  • One-tap keyboard dictation (HEXKeyboard):

    • First run (HEXMobile/Onboarding.swift): The desktop's pages and order. Setting up Hex starts the speech model downloading (the engine transcriberFactory picks; a launch that found Parakeet ended mid-load stays failed until Try again) and moves on after a beat. With Hex, a short demo, moves on by itself. Let Hex hear you has two rows, the microphone (its pill and the page's Continue say Continue before iOS's prompt, never Allow, per App Review 5.1.1(iv)) and the Hex keyboard with Full Access (Open Settings, Hex's page); a done row shows only its check, and the page moves on once both are done. Continue never needs the microphone. Almost ready shows only while the model downloads, with Continue without a model after a failure or 30 s without progress. Your turn keeps the keyboard up and counts once a dictation arrives or three words are typed with Full Access; an empty dictation says Hex heard nothing (the keyboard notes it in the App Group). The keyboard also notes when it comes on screen and leaves (KeyboardPresence): when Hex's keyboard isn't up 2 s after a keyboard switch, the page says to tap the globe until it is, and only words that arrive while it's up count as typed. iOS's dictation and other keyboards take it off screen, so their text never finishes the practice; the page says to tap the Hex key instead. Hex is ready shows only once every step is done, else Almost ready with what's left. The model step counts the keyboard's own model too (KeyboardSpeechPreparation): on iOS 26 the keyboard dictates with Apple Speech even when Hex uses Parakeet, so Hex installs that language's model as the keyboard would; where Apple has none on the iPhone, it marks keyboardNeedsHex so the Hex key opens Hex from the first tap. Home's Ready counts it the same way. The model's progress pill sits in the top bar until it's ready; Skip shows on the demo and the practice only. It shows once: onboarding.completed in the app's UserDefaults records Start using Hex or Continue without a model, onboarding.page resumes a quit part-way on the same page, and a launch that finds the keyboard ready and the microphone allowed (an existing user) records completion without showing it. Run setup again replays it from page 1. Its --snap-state scenes are onboarding-setup, -demo, -permissions, -permissions-done, -waiting, -failed, -try, -try-done, -try-empty, -try-off, -ready, and onboarding-tour, which walks every page by itself for a recording.
    • Without Full Access (App Review 4.4.1): Hex is a keyboard first. The full keyboard types without Full Access or network: letters, shift and caps lock, numbers and symbols, accents, delete, return, the globe key, and suggestions and autocorrect (UITextChecker, the supplementary lexicon, and, where a typing model is bundled for the language, Spatial autocorrect's model: English, Spanish and French, and German with its experiment on). Words it learns may last only while the keyboard runs, as an iPhone may not let it write the App Group; haptics, key clicks (Apple's keyboard guide), Paste and dictation need Full Access. Without Full Access the keyboard always mounts the full keyboard, whichever keyboard is chosen (the dictation bar types only spaces and line breaks). Dictation needs Full Access (Hex's App Group and the microphone): tapping the Hex key, or sliding for its menu, shows for 8 s in the suggestion strip "Allow Full Access to dictate" and "Settings › Apps › Hex › Keyboards", with an Open Hex chip (hex-dictation://keyboard/setup) that opens the app's setup sheet. Nothing else opens Hex without Full Access, and the strip shows no notice until the Hex key is tapped.
    • Setup: Home's status line counts the setup steps left ("Finish setup · 1 step ›", else "Ready") and opens the same sheet: live status for the keyboard, Full Access, the microphone ("For dictation" with Continue, which iOS's prompt follows, until it's asked; never a button that says Allow, per App Review 5.1.1(iv)), and the speech model, a field to switch to Hex in while Full Access is unconfirmed, and Open Settings (Hex's page in Settings) while the keyboard isn't ready. iOS reports Full Access only inside the keyboard, so the keyboard reports it to the App Group as it loads, each time it appears and every 5 s while shown (keyboard.fullAccessOn/Off, with when its process started, and keyboard.fullAccessSeenAt for an on), and posts a Darwin notification the app refreshes on. Full Access shows as on once a keyboard has run with it, and as off only when a keyboard said so since Hex last came to the foreground (it changes only in Settings), from a keyboard process no older than the one that last saw it on. Otherwise the sheet says "Not confirmed yet" with "Switch to Hex below to confirm." under it, and focuses the field.
    • Opening Hex: SwiftUI Link, openURL, and extensionContext.open do nothing in a keyboard extension, so the keyboard opens Hex through the UIApplication at the end of its responder chain (ContainingAppOpener).
    • Cold start (before iOS 26): Tapping Dictate opens the containing app (hex-dictation://keyboard/record?...) with a 10-second handoff window and starts recording. Hex never opens another app: the handoff screen shows a quiet "Swipe bar to return" cue over the home indicator, and you swipe back to the app you were typing in. The text goes in once the keyboard is on screen there again: iOS often starts a new keyboard on the way back, and one that inserted while it loaded, before its field was connected, lost the text (iPad mini, iPadOS 18, TestFlight build 115).
    • GPU Dot Scope (HEXShared/DotScope.swift, DotScopeShader.metal): Both HEXMobile and HEXKeyboard render Hex's desktop menu-bar braided oscilloscope (src/dot_scope.rs, src/rune_hud.wgsl) at native display refresh via CAMetalLayer + CADisplayLink, pausing at 0% CPU/GPU once settled in idle. The keyboard extension mounts it only while a capture or transcription is active and shows a static Canvas dot row at rest, so a fresh keyboard process builds no Metal device, pipeline, layer, display link, or idle pre-roll before its first frame; the app's docked keyboard preview keeps the live scope. The home-indicator swipe cue's pipeline is compiled on first use, which only the app shows. Both layers turn off the Metal Performance HUD (Settings › Developer › Graphics HUD), which would draw its frame rate under the home well's Dictate pill, unless launched with MTL_HUD_ENABLED=1. Lock Screen / Dynamic Island Live Activities render a matching Canvas dot-matrix badge (HexDotMatrixBadge) because WidgetKit archives static view snapshots rather than hosting live CAMetalLayers. Meter polling stays alive through a temporary non-recording state of the same capture; it resumes app levels and keyboard snapshots when the recorder is ready again. Capture completion, interruption, and Stop end polling.
    • Warm session (Off, 5m, 15m, 1h): Picking a duration on Keep mic warm starts an AVAudioEngine input session immediately so subsequent taps on Dictate record directly inside the keyboard with zero app switches; picking Off or tapping the header timer ends it. Unless chosen, it is Off, with the row hidden, where the keyboard dictates by itself (KeyboardSelfDictation: iOS 26, and the keyboard hasn't opened Hex for a dictation in the last 7 days, for any reason: the keyboard records its own (memory, a missing speech model, the Multilingual model) and Hex records every keyboard launch, since TestFlight build 129 showed keyboards that opened Hex for reasons they left no record of; a dictation the keyboard finishes by itself clears it), and 15m elsewhere; worked out on each read, never stored. The row shows whenever a duration is in effect, and stays on the page once picked. Once on iOS 26, a stored 15m (which tapping the old default to warm the mic also stored) was taken for not chosen where the keyboard dictates by itself. A Dictate that opens Hex there brings the 15m default back from that launch, so it warms as before 129.
    • Styles (TranscriptFormatter, run by DictationPipeline wherever Hex dictates: the app, the keyboard's warm session and cold start, the Action Button, Control Center and in-keyboard dictation): Polished (full sentences: fillers trimmed, each sentence capitalized, "I" too, and ended, in Chinese and Japanese with "。"), Casual (texting: sentences may start lowercase, though never an acronym or a word with inner capitals; "i" for "I"; no comma after an opening "yeah" or "ok"; no final full stop on a message of one line and up to three sentences), Notes (spoken "bullet" / "new line" markers, else several sentences become a bulleted list and "Shopping list: eggs, milk and bread" bullets its items; no opening "so", "yeah" or "and then", no final full stops), and Verbatim (as the speech model wrote it), with cursor-aware insertion spacing and repeating backspace. On an already punctuated sentence ("Sounds good, see you at 5.") Polished and Verbatim read the same; they differ on fillers, missing capitals or endings, and with Clean up what I say on.
    • Lowercase (Settings › Dictation › Lowercase): Off by default. After the style and custom vocabulary's spellings, everything is lowercased, custom vocabulary included, as desktop's Lowercase step (str::to_lowercase on the final text) does, on every path (DictationLowercaseSettings, App Group key settings.dictationLowercase).
    • French spacing (every style but Verbatim): dictation in French, or translated into it, gets the spaces French puts around punctuation, last in the pipeline (FrenchSpacing): a narrow no-break space (U+202F) before ? ! ; and a no-break space (U+00A0) before : and inside « »; Canadian French none before ? ! ; (OQLF), Swiss French narrow ones inside « » too. Whatever was there becomes exactly that one space, and only a mark that ends a word is spaced, so 10:30, URLs, addresses, emoticons (":)", ";-)"), code in backticks and a mark at a line's start stay as they are. Apple Speech writes a plain space before "?" in fr-FR and fr-CH and none in fr-CA's DictationTranscriber (macOS 27.2.1, say voices), and every style but Verbatim took the space out ("ça va?"). Apple's French keyboards type no space before a mark (iOS 27.2 simulator, black box), so the full keyboard doesn't either.
    • Format lists (Settings › Dictation › Format lists): On by default (DictationListSettings, App Group key settings.dictationFormatLists), in English and Spanish, every style but Verbatim (SpokenLists, inside the style after its fillers, so each style ends the lines its own way). "first … second …" ("1st … 2nd …", "primero … segundo …", "en primer lugar …"), "number one … number two …" (also "step", "item", "point", "número", "paso") and "one, … two, …" become 1. … lines; "bullet", "bullet point", "next item", "dash" ("viñeta", "guion") said twice or more become - … lines; "new line" and "new paragraph" ("nueva línea", "nuevo párrafo", "salto de línea") break lines. Text before the list gets its own line (with a colon unless it ends a sentence), and what follows the last item's sentence its own line. Guards keep prose: markers count up from one, at least two non-empty items, each marker at a clause start (the start, after a mark or line break, or after "and"/"then"/"y"/"luego" that are), a bare number needs its comma, and words like "the first time", "second guess", "a new line of", "bullet points" stay as said. TranscriptCleanupTests holds the list cases and 62 prose sentences that must not change.
    • Spoken punctuation (Settings › Dictation › Spoken punctuation): On by default (DictationPunctuationSettings, App Group key settings.dictationSpokenPunctuation), in English, Spanish, French, German, Italian and Portuguese, every style but Verbatim (SpokenPunctuation, inside the style after Format lists, so "dash … dash …" stays a list). Punctuation said by name becomes the mark: "comma", "period"/"full stop", "question mark", "exclamation point", "colon", "semicolon", "ellipsis", "dash", "hyphen", "apostrophe", "hashtag", "dollar sign", "open quote … close quote" (or "quote … end quote"/"unquote"), "open paren … close paren" and "smiley" (":-)"), and each language's own words ("coma", "punto", "signo de interrogación", "abrir comillas", "punto y aparte"; "virgule", "point d'interrogation", "ouvrez les guillemets"; "Komma", "Fragezeichen", "Anführungszeichen unten … oben", "Klammer auf … zu"; "virgola", "punto interrogativo", "apri virgolette"; "vírgula", "ponto final", "abre aspas"). Quotes are “ ” (« » in French, „ “ in German), French spacing follows (FrenchSpacing), and a word after "." "?" "!" gets a capital. Black box (macOS 27.2.1, say voices): DictationTranscriber already writes every mark, so its text has no names left; SpeechTranscriber writes "comma", "question mark" and some French, German and Italian names, leaving the rest as words, often between commas; Parakeet writes every name as words ("noon, comma, see you soon, period", "tonight question mark?"). A mark the model already put beside a name is replaced, never doubled. Guards keep prose: never after an article, possessive, preposition or a word that makes the name a noun ("a period of time", "the comma is", "Oxford comma", "trial period", "ein wichtiger Punkt"), before one that continues it ("question mark tattoo", "colon cancer", "punto de vista"), in a compound ("comma-separated"), or between numbers ("two comma five"); the names that are everyday nouns ("period", "point"/"Punkt"/"punto"/"ponto", "question mark", "dash", "smiley") also need the line's end, a mark after them or a clause start after them; a colon needs words after it; a closing quote or bracket closes only an open one. TranscriptCleanupTests holds the models' transcripts and 141 prose sentences (84 English) that must not change. Polished no longer ends a line that ends in "," or ";" ("Hi Sarah," before a line break) with a period, and keeps the space before an emoticon (":-)").
    • Spoken "@" (English, every style but Verbatim): "at the rate" (India's name for @) and "at sign" become "@", and an address said around it joins up: "John at the rate Gmail dot com" becomes "john@gmail.com", "dot" turning into "." only inside the address (SpokenCleanupRules.spokenSymbols, run by CleanupTagger.cleanIfEnabled with Clean up what I say on or off, in the app and in-keyboard dictation). "at the rate" needs a domain after it, so "at the rate of 5%" and "at the rate we planned" stay; "at" alone is never touched. Apple's SpeechTranscriber glues the domain to the word before ("at the rategmail.com"), which is split again; DictationTranscriber already writes "@" for "at sign" and for "john at gmail dot com". Polished leaves an address's case alone, starts no sentence at a full stop inside a word ("gmail.com"), and ends no lone address with a full stop.
    • Spoken counts (English, every style but Verbatim): a count of three numbers or more from one, in words or digits with only commas or spaces between them, is written in digits: Apple's SpeechTranscriber writes "Test one, 2, 3." where Parakeet writes "Test 1, 2, 3", and both become "Test 1, 2, 3." "123" right after "test" or "testing" and ending its clause, which Parakeet and DictationTranscriber write for a quick "testing one two three", becomes "1, 2, 3" too (SpokenCleanupRules.spokenCounts, run by CleanupTagger.cleanIfEnabled before the tagger, with Clean up what I say on or off). Other numbers stay ("Chapter one, two and three", "room 123", "test 123 passed"). The tagger alone took a count in words for a self-correction ("Test one two three" became "Test three."); CleanupGuard keeps it.
    • Numbers as digits (Settings › Dictation, on by default; English, every style but Verbatim): numbers said in words are written in digits, as Apple's SpeechTranscriber writes them, so every engine reads the same (SpokenNumbers, run by CleanupTagger.cleanIfEnabled after Spoken counts and before the tagger, so the tagger never removes a number in digits outside a self-correction). Parakeet V2 now and then writes a whole sentence in words ("The ratio is five point five.", "november twenty second"), mostly in a real voice: 53 of 80 test sentences in Ryan's cloned voice against 11 of 80 in macOS voices (macOS 27.2.1, experiments/2026-11-22-hex-numbers). It writes decimals ("5.5", "3.1.4", "1.2 million"), percentages ("20%"), numbers from ten up and compounds ("25", "312", "15,000"; commas from 10,000, as Apple writes "8000"), years ("1999", "2026"), money with "dollars" ("$5", "$15.40"), two to nine before a plural noun or unit ("2 dogs", "6 foot"), ranges ending in one ("5 to 10 minutes"), times with am, pm or o'clock ("6 am", "7:15 pm"), ordinals in dates ("November 22nd", "the 4th of July"; "May" and "March" only with a capital), three or more spoken digits ("4729", "555-123-4567"), and minus or negative before a number ("-5", "-$40"). Text already in digits keeps its form ("2,400", "$4.99"). All of a number converts or none of it ("a hundred and fifty" 150, "one point twenty one" 1.21, "one in ten people" 1 in 10, "five to ten percent" 5 to 10%, "five foot six" 5 foot 6, "oh four seven two" 0472 mid-sentence); arithmetic stays as said ("ten minus five is five"), as do "two to three hundred" and "one or two hundred". A number the engine capitalised as a name stays: mid-sentence or with a capital inside, before another capitalised word ("I love Twenty One Pilots", "Ten Thousand Villages", "Fifty Shades"), but "Area fifty one", "Forever Twenty One" and "Twenty people came" convert. Custom vocabulary and learned words are never converted (protectedTerms, matched word by word without case): adding "Five Guys" or "Twenty One Pilots" keeps them, the fix for brands the engine writes in lower case, which no rule can tell from a count. Not yet: "one" alone ("one of them", "no one", "one more"), times and prices without a marker ("three thirty", "four ninety nine", "twelve fifty"), ranges and scores without a unit ("eight to twelve", "three to one"), fractions, ordinals outside dates ("finished first", "twenty first floor", "a second"), small numbers before anything else ("at eight", "GPT five"), and any run of number words that reads two ways ("one thirty"). Other languages keep their words: each needs its own number words, compounds (German "fünfundzwanzig", French "quatre-vingt-dix") and separators ("5,5" and "15.000" in German, French, Spanish, Italian and Dutch). SpokenCleanupRulesTests holds the 80 sentences as both engines wrote them in both voices.
    • Mid-sentence dictation: when the text before the cursor doesn't end a sentence (. ! ? … or a line break), the keyboard lowercases the dictation's first word, except "I" and its contractions, acronyms, words with inner capitals, and words the dictation language's UITextChecker dictionary doesn't know in lowercase (likely names). German keeps the case, since its dictionary takes nouns in lowercase; so does a language with no dictionary on the iPhone (TranscriptFormatter.insertionText).
    • Full keyboard Hex key (Settings › Keyboard › Full, the default): A tap, or a still hold of 0.35 s, starts dictation: in a warm session, else inside the keyboard (in-keyboard dictation, iOS 26+), else by opening Hex. In-keyboard dictation listens from touch-down, so a tap's or hold's first words are kept; moving more than 10 pt first opens the style menu, dropping that capture; releasing over a row picks it, and releasing elsewhere leaves the menu open for taps (HexKeyGesture). VoiceOver activation dictates, and the Dictation styles action opens the menu.
    • While the keyboard records: Show what I say and Type as I speak (Settings › Dictation, iOS 26+): While the keyboard records by itself (in-keyboard dictation), the capture's chunks also go to Apple's SpeechAnalyzer as they arrive (AppleLiveTranscriber: SpeechTranscriber with volatile and fast results, or DictationTranscriber's progressive long dictation for its languages), only when the language's model is already on the iPhone: it never downloads one. Show what I say (on by default): the full keyboard's keys go and the words so far take their place under the recording bar (LiveDictationPreviewView), the settled words in the keys' ink and the tail the recogniser may still change muted, the newest line in view with whole lines above it. It takes every touch where the keys were, so nothing types while it shows; the recording bar's ✕ and ✓ work as before, and an iPhone with a Home button keeps its globe key where it was. VoiceOver reads it as one element, never announced as words change. Its motion is one family, a spring with no bounce by its duration (LiveWordsMotion, LiveDictationPreviewHost.Motion, and the recording bar's morph and fades), and Core Animation runs every frame of it, so it keeps the display's pace however busy the main thread is with the capture, the recogniser and the Dot Scope (a display link drawing the words got a quarter of its frames on time on an iPhone 17 Pro): the keys fade (0.2 s) as the words fade in rising 6 pt (0.3 s); whatever changes in the words (new words, a settled tail turning to ink, a word the recogniser changed) cross-fades in place (0.3 s, the cubic closest to the spring) while what didn't change stays exactly as it is; a new line moves the block up (0.3 s) and the line leaving the top fades through a 14 pt band, then out of it once the words are at rest. With Reduce Motion, the cross-fade only (0.2 s) and lines jump. LiveWordsView draws the words once per update into one layer (TextKit 1) and drops the lines above the box, whole, at rest. When the dictation goes in, the preview's words fade drifting 6 pt up (0.15 s) as the keys come back (0.3 s), and they stop taking touches at once; the field gets the dictation's own text, with its style, clean-up, punctuation, held results and Undo dictation as before, so it can differ from the preview: with the speech model in keyboard (Parakeet, English's default) the preview's words are Apple's and the text Parakeet's ("10" can go in as "ten"). Type as I speak (off by default): the words are typed into the field as they come (LiveFieldTyping), only the changed tail retyped. It keeps exactly what it typed and edits only while the text before the cursor still ends with it; when the dictation's text goes in, its words come out first, so the text goes in where and as it would without them, in one step. ✕, a failure or nothing heard take them out. The keyboard hiding mid-dictation takes them out where the app still takes edits; one gone to the background doesn't, so they are noted (LiveDictationSettings.leftover) and the held text, inserted by itself or with Paste dictation, replaces them where the field still ends with them. Once the field changes under it (a key, a cursor move, a selection, or an app that reports no text), it stops and leaves its words, and the text goes in at the cursor as without it. Never in secure, password, one-time code or card number fields, over a selection or a two-thumb Dictate's range, or with Translate what I say on. Neither runs before iOS 26, for dictations recorded through Hex (Keep mic warm), when less than 32 MB is left once the capture runs (KeyboardLiveDictation: the recogniser's share in the keyboard with margin, and the speech model's transcription after it), on an error, or when the recogniser isn't running 4–6 s into the audio; under 24 MB, or on a memory warning, the recogniser goes and the dictation goes on. The dictation bar shows no preview (Type as I speak works there, unverified on an iPhone). On an iPhone 17 Pro (iOS 27.2, keyboard limit 177 MB, Parakeet loaded, a clip played from a Mac's speaker): the first words show about 1.2–1.3 s after speech starts (2 s after the capture's first audio; 2.1 s and 3 s on the first dictation after the keyboard starts), then about a second behind; over a 60 s dictation the keyboard ended at 60.1 MB with the preview against 58.1 MB without (peak while recording 66.5 against 60.1, at Parakeet's transcription 78.7 against 76.6), and two-sentence dictations 0.4–0.8 MB higher. A scrolling text view holding the whole text had peaked 27 MB higher there, so the preview draws only about its box's worth. Measured on the Mac Studio (macOS 27.2.1) with the keyboard's own code fed a clip in the page's 4,096-sample chunks in real time: first words 0.5–1.0 s after the first word ends, then 0.6–0.8 s behind (90% within 1.2 s), in bursts about once a second; 2% of words shown were rewritten (7 of 336, "730." to "7.30."); words settle only at a pause. Without fast results words came 2.7–3.5 s late, and DictationTranscriber rewrote 15% of them. The simulator has no speech models: in simulator builds only, LiveTranscriptReplay plays a recorded run (live-replay.json in the App Group, or the keyboard's own Library for an unsigned build) from the first audio. ParakeetLiveTranscriber (the audio so far again about once a second, a 20 s window) is written for when the speech model's own words should show, and not used yet.
    • Auto-start: Settings › Keyboard › Auto-start (FullKeyboardSettings.dictationBarAutoStart) lets the dictation bar start dictating by itself as it shows, as a tap on Dictate would, 0.8 s after it appears (after the first frame and the launch's optional work), only while idle, outside number pads and the emoji panel, and only where it dictates without leaving the field (a warm session or in-keyboard dictation, never by opening Hex). Never, On switch (the default) or Always. iOS doesn't tell a keyboard whether you switched to it or a field opened with it, so On switch guesses (KeyboardSwitchGuess): the keyboard comes back in the field it last hid in (documentIdentifier), and the field's app didn't go to the background (NSExtensionHostDidEnterBackground) from 2 s before that hide on. Switching to another keyboard and back in one field starts; a field that opens with Hex already chosen doesn't. Known gap: the first switch to Hex in a field another keyboard opened is another document, so it doesn't start. Unproven on a device: whether documentIdentifier stays the same across a keyboard switch and changes when a field is focused again, which the guess relies on.
    • Push to talk: Holding still for 0.35 s on the full keyboard's Hex key or the dictation bar's Dictate starts dictation at once with a light haptic (Settings › Keyboard › Haptics), and lifting the finger finishes it as Done does; a quick tap still starts it and Done finishes it. On the Hex key, sliding more than 10 pt before the hold fires opens the menu instead, and once dictation has started, moving the finger changes nothing. Push to talk needs dictation in the keyboard (a warm session or in-keyboard dictation); a hold that opens Hex is only a tap. The full keyboard's bar takes the same hold (0.4 s, Apple's long press) anywhere no word is: an empty slot, beside the Hex key, or a bar with nothing in it, through the Hex key's own path (HexKeyGesture, hexKeyHeld), but not while the bar shows the Full Access hint or a notice. A tap there does nothing, a slide cancels it, and a word's slot keeps its tap and its hold (Add to vocabulary). The bar's Paste is plain text on the words' line, and Paste and Next keyboard centre on the keyboard's middle, not the strip's. Paste shows only while the field is empty, as Apple's keyboard offers a copy, and Code from Messages only in an empty one-time-code field with a number pad or with autocorrection set either way (as WebKit sets it on Safari's autocomplete="one-time-code"): a chat box that marks itself one-time-code keeps its suggestions. On a number pad, or with only Apple's keyboards beside Hex, it switches to the next keyboard; elsewhere (Emoji or another third-party keyboard on, where that switch lands on them about half the time) a tap opens iOS's keyboard list, as the globe's hold does, to pick Apple's. It needs no Full Access, which only Paste does. When iOS prompts Allow Paste (read ≥ 0.3 s) or refuses a paste read, the strip shows a tappable two-line notice (Paste without asking / Paste from Other Apps › Allow with a Settings chip that routes through hex-dictation://keyboard/paste-settings straight into Settings › Apps › Hex), and Hex records keyboard.clipboard.allowPastePrompted in the App Group so Home, Settings › Keyboard, and Setup surface the same one-tap Open Settings banner once iOS has created that row. A later fast cross-app paste (ignoring Hex's own copies via markOwnedCopy) or tapping dismiss clears it.
    • Apple Intelligence cleanup (Settings › Experiments › Clean up with Apple Intelligence, iOS 26+): Off by default. For every style but Verbatim, one on-device FoundationModels pass before formatting fixes clearly misheard words and spoken self-corrections, within a 2 s budget. A timeout, error, empty or implausible output keeps the raw transcript, and the toggle is disabled with a reason when Apple Intelligence is unusable.
    • Multilingual model (Settings › Experiments › Multilingual model (Parakeet), iOS 18+): Off by default, and disabled on iPhones under 6 GB (MultilingualParakeetSettings, in the App Group). While on, Hex dictates Parakeet Ultra's 25 languages (Moondream's post-training of NVIDIA Parakeet TDT 0.6B v3) with it, which hears them mixed within a sentence (Dutch with English words, a TestFlight request); any other dictation language keeps the engine Hex picks without it. The language Ultra takes is the one set, else the one Automatic follows on the keyboard, else the iPhone's first. The experiment never takes dictation away: until Ultra is ready, Hex and the keyboard dictate as they would with it off, and Hex switches to Ultra once it has loaded (a dictation in flight isn't interrupted). Turning it on downloads FluidInference's Core ML build (632 MB, pinned to revision 95eaa59, each file checked against its SHA-256 and resumed from its .partial) into Application Support, out of backups, beside that engine (ParakeetUltraDownload): home and the switch say "Downloading the multilingual model: 123.4 MB of 632.2 MB" with a bar. A dropped connection, or bytes that stop for 20 s, tries again by itself after 2, 4, 8, 16, then 30 s, starting over whenever it got further; in the background it waits for Hex to come back to the front, which picks it up at once (and starts a fresh connection after 25 s away). After six tries in a row that got nothing further, or a refused request or a full iPhone, it says what happened in plain words with Try again and Turn off. Once downloaded, Ultra's first load (Core ML's compile, a minute or more) runs beside the usual engine too, except where Parakeet V2 dictates and leaves no room for it (English before iOS 26), where Ultra takes V2's place for that load; later launches load it directly when Core ML has compiled it for this iOS. Turning it off stops the download, then deletes it; a launch deletes a leftover only when the switch reads off as stored, never when the App Group can't be read (a launch before the first unlock). FluidAudio 0.15.5 has no .ultra, so ParakeetUltraModel loads the four models itself and hands FluidAudio v3 AsrModels (its own loader would replace a folder it can't load with v3's download). It is chosen only with 1.5 GB left to load it (os_proc_available_memory), and never again after a load that ended Hex (ParakeetLoadGuard, its own keys) until Try again beside the status, or the switch turned off and on. Unless Experiments › Speech model in keyboard has it loaded in the keyboard (below), the keyboard doesn't use it, so once Hex has it ready (MultilingualParakeetSettings.isModelReady, set after a load, cleared while it's missing, set aside or off) and it takes the dictation language, Dictate and the Hex key open Hex, as before iOS 26 (a warm session still dictates without leaving the app); until then the keyboard dictates itself. Measured on a Mac (~/dev/experiments/2026-11-21-hex-multilingual-parakeet): mixed Dutch-English test sentences 7.5% WER against Apple Speech nl-NL's 26.9%, pure Dutch 0.9% (0.4%), pure English 0% (0%). Load memory, compile time and speed on an iPhone are unmeasured.
    • Speech model in keyboard (Settings › Experiments › Speech model in keyboard, iOS 26+): On unless the person turns it off, and disabled (shown off) on iPhones under 6 GB (KeyboardParakeetSettings, in the App Group; a value stored by hand wins, so a switch turned off stays off). Off, nothing of it loads and the keyboard dictates exactly as before. On, Hex downloads Parakeet V2 (464 MB, about 700 MB with the keyboard's compiled copy) into the App Group (KeyboardParakeetDownload; turning it off stops and deletes it unless Hex itself dictates with V2). By itself it downloads only where the keyboard would use it (KeyboardParakeetDownload.hold): English dictation without Two-language dictation or the Multilingual model, the keyboard's own memory limit measured on this iPhone (the keyboard writes footprint plus os_proc_available_memory to the App Group at each launch and tells Hex with a Darwin notification) and at least KeyboardParakeetBudget.requiredLimit, a connection that isn't expensive or constrained (NWPathMonitor: not cellular, a personal hotspot or Low Data Mode), and 2 GB free. It checks at launch, whenever Hex comes to the front, when such a connection returns and when the keyboard reports its limit. Turned on by hand (a stored true, including from the builds where it was off by default) it downloads at once on any connection. Until then the row says what it waits for: "Downloads after you first use the Hex keyboard", "Downloads on Wi-Fi", "Downloads once the iPhone has 2 GB free", "Only for English dictation". The keyboard's own dictation transcribes with Parakeet in the keyboard's process (HEXKeyboard/KeyboardParakeet.swift), with the app closed: V2 for English, and Parakeet Ultra for a language Experiments › Multilingual model dictates once its files are there, in which case Dictate and the Hex key stop opening Hex for it. Parakeet's files moved from the app's Application Support into the App Group (SpeechModelStore), renamed in place at launch (moveLegacyDownloads) rather than downloaded again; each folder is marked complete (.hex-complete) once every file is there, and the keyboard loads only a marked one. The keyboard loads it after its first frame and launch work, off the main thread (the preprocessor on the CPU, the rest on the CPU and Neural Engine, one chunk at a time, never the GPU, which iOS gives a keyboard only in the background role), and keeps it while its process lives, until a memory warning. Its first load compiles it for the Neural Engine in the keyboard's own Caches (17 s for V2, 12 s for Ultra on an iPhone 17 Pro); a dictation never waits for that, and uses Apple's speech recognition exactly as with the experiment off whenever the model isn't loaded, isn't on the Neural Engine (the keyboard's Neural Engine memory, else Core ML's compute plan before iOS 27), another transcription runs, the keyboard's memory limit (footprint plus os_proc_available_memory) is under 131 MB (V2 and Ultra alike: with the model loaded the keyboard records at up to 91 MB, and Ultra's transcription rose 9 MB in Hex's keyboard, V2's 16 MB), too little is left, the recording is longer than 90 s or there isn't room for it (FluidAudio holds the whole recording in memory, about 0.42 MB a second measured, counted at 0.5 MB on top of the 24 MB transcription headroom; Apple's speech recognition streams the file, so a long dictation goes there and arrives a little later), it hears nothing, or it fails (KeyboardParakeetBudget has the measured numbers). Core ML keys what it compiled to the files themselves: a model downloaded again, or moved, compiles again, so the log calls a load "cold" by its time (5 s or more), and moving a legacy folder forgets the app's compile record for it (ParakeetCompileRecord.forget), so V2 says it's optimizing for this iPhone and Ultra gets ready beside the engine that dictates. The Experiments row says what it's doing: downloading, downloaded, getting ready in the keyboard, ready, or that the keyboard can't use it now (KeyboardParakeetSettings.keyboardState, written by the keyboard). Each dictation logs its engine, the keyboard's limit, footprint before and at peak, load and transcription time to keyboard-capture-probe.log. Measured in Hex's keyboard on an iPhone 17 Pro (iOS 27.2, keyboard limit 177 MB) with the app force-quit: V2's first load 15.4 s, the keyboard recording at 56 MB and peaking at 72 MB while V2 transcribed 9.4 s of audio in 0.28 s (Apple's speech recognition, same clip: 0.34 s), with 468 MB on the Neural Engine outside the footprint. Twenty dictations in one keyboard process stayed on V2 with a peak of 96 MB, ten hide-and-show cycles and a five-minute hide kept the process and the model, a new process reloaded it in 0.3 s, and Ultra peaked at 66 MB (607 MB on the Neural Engine). A probe keyboard (~/dev/experiments/2026-11-22-hex-keyboard-asr/native) put 60 s of audio at 0.25–0.5 s; on the CPU, four chunks at once passed the limit there, so the keyboard never runs that. Linking FluidAudio into the keyboard grows its binary from 5.9 to 12.6 MB and, with the experiment off, its footprint at launch by about 0.2 MB (writable data dyld fixes up), with no change to its first frame.
    • Clean up what I say (Settings › Dictation › Clean up what I say): Off by default. For every style but Verbatim, Hex's dictation cleanup tagger labels each spoken word KEEP or DELETE on the iPhone and drops ums, repeats, and the words a spoken correction replaces ("3pm, actually 4pm" becomes "4pm"); a word goes only above P(DELETE) 0.98. CleanupReconstruction, a port of the tagger's reconstruct.js, then repairs the commas and capitals next to a deletion. It never adds, reorders, or changes words. The tagger only knows English (run on Spanish it dropped negations: "Te digo que no puedo ir" became "Te puedo ir"), so Spanish, German and Norwegian get word rules instead (SpokenCleanupRules): the language's hesitation sounds go (the styles' list below), a filler between commas goes ("este", "o sea"), and a correction marker set off by commas ("digo", "perdón", "no", "mejor dicho"; "nein", "ich meine"; "nei", "nei vent", "jeg mener") drops the words it replaces, counted back to the nearest earlier word that matches the repair's first word or is of its kind (a number, weekday, month, or today and tomorrow): "a las 3, digo, a las 4" becomes "a las 4". With no such word the sentence stays as said, and other languages stay as heard. It runs after transcription and learned fixes and custom vocabulary, before Apple Intelligence cleanup, translation, and formatting, in in-keyboard dictation and in the app. The tagger is a BERT L-4/H-512 token classifier fine-tuned from Google's Apache-2.0 bert_uncased_L-4_H-512_A-8 on generated examples, shipped as an int8 Core ML model (28.5 MB) with its WordPiece vocabulary once, in HEXKeyboard.appex (HEXKeyboard/CleanupTagger/). The app loads it from there. CleanupTokenizer matches Hugging Face's BertTokenizer, including its Unicode 8.0 character classes. The model loads on the Neural Engine or CPU when a dictation starts recording with the experiment on, stays loaded, and is freed on a memory warning. Cleanup gets 150 ms; a timeout, an error, or a result with no words left keeps the transcript as heard.
    • Custom vocabulary (Settings › Dictation › Custom vocabulary): Off by default. A local list of up to 100 names and phrases, applied to every engine, in the app and in-keyboard dictation. The full keyboard uses the same list whether or not the switch is on (Settings › Keyboard › Vocabulary edits it too; TestFlight build 56: "custom dictionary for uncommon programmer slang"): its words, and each word of a phrase, are never autocorrected away, a different case is put right on space ("github" → "GitHub"), the strip completes them before the dictionary's words ("kub" → "kubectl") with no spelling guesses for them, Swipe typing reads them as your own words, and a bilingual partner shares them. Holding the word you typed in the strip's left slot offers Add to vocabulary there (FullKeyboardView+Vocabulary; a capital only a sentence's start gave the word is left out); without Full Access the keyboard can't write the App Group, so the word is kept for that session only. Before this the keys knew only iOS's text replacements and Contacts names (requestSupplementaryLexicon), never Hex's list. After transcription, matching words are rewritten to the listed spelling across case, spaces, tabs, a single hyphen, and possessive apostrophes, before cleanup and again after formatting, so a style cannot undo a spelling. In English, a name heard with another spelling then takes the listed one (SoundAlikeSpelling; every engine writes a name's commonest spelling, so TestFlight build 130's "Stacey" kept coming back "Stacy"): a word the engine capitalized that reads the same as a listed term of three or more letters starting with a capital, once interchangeable spellings are evened out (ph and f; a hard c, k and ck; chr and kr; a soft c and s; ae and e; a leading ai and a; a final y, ey, ie, ee and i; any other y and i; a silent h; doubled letters), takes the term ("Stacy" → "Stacey", "John" → "Jon", "Catherine's" → "Katherine's", "Amy" → "Aimee"). Inside a sentence, and not right after another capitalized word, it may instead sound the same by a Metaphone-style key (its consonants with voicing kept, first vowel sound, vowel ending and vowel groups) and be at most two edits away ("Jeff" → "Geoff", "Versal" → "Vercel"). Lowercase words, words in capitals, listed words and two equally close terms stay as heard. The listed spelling is kept exactly, even at a sentence start (an all-lowercase term stays lowercase), and a term that spells its own boundaries ("iPhone 7", "X-ray") matches those boundaries exactly. A word that carries its own apostrophe only matches a term that has one, several transcript words are joined only when the term reads as a compound, and compound terms like "WeChat" can also match the plain phrase ("we chat"). On iOS 26 Apple Speech the list is also sent as a best-effort recognition hint, in the app and in in-keyboard dictation (through build 130 the keyboard sent only Learn how I type's words: #2103 added the list's rewrite there but not its hint); Parakeet's batch path gets no hint.
    • Learn how I type (Settings › Privacy › Learn how I type): On by default (an unset switch reads on; turning it off is saved and stays off), and shown whether or not the full keyboard is on. It learns only from typing and dictating with Hex, on this iPhone, never from what you write elsewhere. A private journal in the App Group (typing-journal.json) that never stores what you write: per-key tap offsets, and word-level pairs with counts and at most one word of context on each side. Only plain text fields that allow autocorrect count (TypingJournal.allows). Nothing leaves the iPhone.
      • Typing: words you erase and retype, autocorrect fixes you undo from the strip, suggestions you pick, and autocorrections you turn down by tapping what you typed. A typo you fixed the same way twice (never a completion, never fixed two ways) becomes a fix on space, only a suggestion when what you typed is a real word; an autocorrection you turned down twice never applies again (TypingLearning, read by AutocorrectEngine).
      • Dictation: from the full keyboard and the dictation bar alike, the keyboard remembers each dictation by the text around it (DictationEditTracker), finds it again after edits through Hex's keys and on textDidChange, and diffs it word by word against what Hex inserted and the raw transcript the app or in-keyboard dictation sends with it (DictationEditInference.swift). It settles after 60 s, when the keyboard goes away or the field changes, on a Send return key, or when the host clears the field (a send). Each dictation gets an outcome (kept, edited, rewritten when more than half changed, undone, deleted, or redictated when deleted and dictated again within 20 s) and its pairs: substitutions, insertions, deletions, words cleanup dropped that you added back, and fillers Hex left that you deleted, each as heard, inserted, and final words. New writing before or after a dictation, changes that aren't plausibly the same words, edges the host's cut-short context can't show, and translated dictations teach nothing.
      • Applied: a substitution seen twice and never contradicted (you kept the heard words in a dictation you corrected, changed the fix back, or fixed the same words another way) is applied right after transcription, in in-keyboard dictation and in the app before custom vocabulary and cleanup, whole words only (never inside a longer, hyphenated, dotted, or slashed word) and following the original's case unless the fix has capitals. Its meant words join the Apple Speech hints after custom vocabulary's. Typing insights lists these fixes with a switch each and the share of dictations kept and edited; Delete what Hex learned erases everything, including what a running keyboard still holds.
      • Per language (Settings › Experiments › Learn each language separately): Off by default (experiments.learnPerLanguage, App Group). Every correction and dictation fix is tagged with the language it was typed or dictated in either way, so turning it on needs nothing learned again; a fix made in two languages is tagged with neither and counts for every one. On, the keys learn a language's typos and rejections from that language's corrections and the untagged ones, and a dictation applies only the fixes made in its language and the untagged ones (a fix learned in en-US applies in en-GB). Learned words are one list per keyboard language (keyboard.learnedWords.<code>), started from the shared keyboard.learnedWords list the first time that language is used; bilingual autocorrect's partner keeps its own list and its own fixes. Where fingers land and Adaptive touch stay shared. Delete what Hex learned erases the shared list and every language's. Two limits: a dictation fix you kept as heard in one language still counts against that fix in every language, and retyped words and picked suggestions are tagged with the keyboard's main language, never the bilingual partner's. Off, everything is the one shared list and learning it was before.
    • Auto-capitalization (Settings › Keyboard › Auto-capitalization): On by default, as Apple's (FullKeyboardSettings.autoCapitalization, App Group key settings.keyboardAutoCapitalization). Off, shift never turns itself on, whatever the field's autocapitalizationType asks (a field's or sentence's start, after "." "!" "?" or the double-space period, which still types ". "), and autocorrect and swipe typing add no capital you didn't type: "i" stays "i", "im" becomes "i'm", "mary" stays "mary" and "mondya" becomes "monday", as Apple's keyboard does with Auto-Capitalization off (seen on the iOS 27.2 simulator). Words you typed with a capital, and your own spellings (text replacements, Contacts names, words you kept, fixes Hex learned, your vocabulary), keep theirs; shift and caps lock work as before. The strip offers no capital you didn't type either (TestFlight build 56: "even with auto-capitalization off, many suggested words are capitalized"): a spelling guess or completion iOS knows capitalised ("Mandy", "Monday") and the contraction "I'd" for "id" come lowercase there; sentence and clause starters keep "I" and "I'm" (#2135), and next words and swipe readings already had no capital. Dictation has its own switch (Settings › Dictation › Lowercase).
    • Spatial autocorrect (Settings › Keyboard › Spatial autocorrect): On by default, shown under Autocorrect while it's on (it needs the full keyboard with Autocorrect on). Stored as experiments.spatialAutocorrect from when it was an experiment off by default: unset or true is on, false off, so testers who turned it off keep it off. A noisy-channel decoder (TypingDecoder) replaces UITextChecker's one-edit guesses on space and in the strip for English, Spanish and French (France): each tap is scored against every letter with a 2D Gaussian around the key (shifted by Learn how I type's tap offsets when it has 20 or more taps on a letter), words are found with a beam search over a memory-mapped word list and over your own words (Contacts names and other UILexicon terms, one-word text replacements, words you kept, fixes Hex learned), a word-pair language model weighs them by the word before, a missed space is tried by splitting the taps, accented letters read from their base key (é from e), and what you typed is replaced only when the best reading beats it by a margin (more when what you typed is a word, including a name, day or place typed in lowercase that the dictionary knows only capitalised, which is capitalised as the dictionary path does: "mary" -> "Mary"). A run that is no word and no dictionary's at all is also read as a word plus one slip in the next ("specialchsracter" -> "special character") or as three to six words typed together ("seeyoutomorrow" -> "see you tomorrow"), when the words without their missed spaces are no less likely than the run; those can replace a run the whole-word margin would keep. Deutsch keeps the ordinary readings: its lowercase model cannot give a split's nouns their capitals. Shortcuts, learned fixes, contractions, and words you kept still come first. Other layouts (Norsk, Deutsch, Italiano, Français (Canada), Français (Suisse), Português, Polski, Română, Русский, Українська, Ελληνικά) keep the dictionary engine in their own language, and Srpski (no iOS dictionary) none, never the English model, except as bilingual autocorrect's second language (below). The models (HEXKeyboard/TypingModel/en.hextm and es.hextm, about 3.6 MB each) are built from Google Books Ngram v3 by scripts/build-typing-model.py (see Acknowledgements below); English's contractions count as often as the words they stand for ("do not"), and it adds Hex's own chat and tech words (scripts/en-extra-words.tsv), byte-identical to Android's en.hextm; de.hextm is the German model Android builds and ships (android/benchmark/typing/build-layout-model.py), the same v3 format and, on iOS, off until the experiment below is turned on. fr.hextm is Android's French model, built the same way from Google Books v3's French shards (corpus spellings, with the books' elided forms such as "c'est") and byte-identical to android/app/src/main/assets/typing/fr.hextm (its SHA256SUMS hash). On the held-out French replay (2,000 Tatoeba sentences typed on AZERTY, scored as android/benchmark/typing/quality-gate.py does) it makes 76.4% fewer word errors than the raw taps (35.3% with UITextChecker alone), changes 0.10% of correctly typed words (0.56%) and restores 81.1% of the accents left out where the plain spelling is in neither the model nor fr.valid (78.7%), short of the gate's 85%. Canada's and Switzerland's French don't read it yet (fr-CA, fr-CH). Beside a model, a language with its own word list (HEXKeyboard/TypingModel/<code>.valid, Android's android/benchmark/typing/build-valid-words.py output: today French's) takes what is a word from the model and that list alone, not from UITextChecker (HexWordList, memory-mapped and searched in place; SystemTextChecker.ownWords), and a missing system dictionary no longer turns its model off (AutocorrectEngine.hasWords). UITextChecker still gives guesses and completions while it is in that language. Beside the list, the language's <code>.capitals (Android's build-capitals.py output, read in place the same way) says which words are names: one the books write with a capital at least 95% of the time is no word typed all lowercase, so it takes its capital as UITextChecker gives one ("paris" -> "Paris", "jean" -> "Jean"; "marie", 94%, stays), and a listed name outside the model is a word. German's de.capitals is read in place through the same reader (GermanCapitals). With fr.valid French restores 86.1% of the accents the gate counts (81.1% with UITextChecker). English, Spanish and German keep UITextChecker: their own lists moved a gate number the wrong way. Polish reads pl.hextm (150,000 words, 184,327 pairs, 3.60 MB) and pl.valid (525,084 words, 8.29 MB), both built by scripts/build-typing-model.py pl from NKJP's n-grams, ParlaMint and PoliMorf (Settings › Acknowledgements). Its letters are held on their base keys, so the model puts them back ("zolw" -> "żółw", "wiem ze" -> "wiem że"). iOS's own Polish dictionary is no use without Apple's word data (it takes any letters for a word), and the list replaces it. On the held-out Polish replay (scripts/typing-gate.py run … pl): 85.98% fewer word errors than the raw taps, 0.22% of correctly typed words changed, 95.46% of the marks restored where the plain spelling is no word, 5/5 critical cases. pl.capitals (android/benchmark/typing/build-capitals.py pl, case counted in ParlaMint) gives names their capital: of the names typed all lowercase, 10.65% come out capitalised (0% with the phone's dictionary on the simulator, which has no Polish word data, and 0% without the list). With English as the second language, Polish makes 85.0% fewer errors (86.4% alone: English takes "cos", "troche", "beda" as its words) and English is unchanged.
    • German spatial autocorrect (Settings › Experiments › German spatial autocorrect): Off by default (experiments.germanSpatialAutocorrect, App Group), and only while Spatial autocorrect is on too. With both on, the Deutsch layout reads taps with HEXKeyboard/TypingModel/de.hextm, byte-identical to android/app/src/main/assets/typing/de.hextm and the hash its SHA256SUMS lists: the word list feeds the decoder, the letter odds and the strip, and a bilingual partner may read German taps too. German's model is lowercase throughout, so a noun takes its capital from the list beside it, whether the decoder corrected it or it was typed right ("hsus" reads "haus" and comes out "Haus"; "das haus" becomes "das Haus"): 95% and up anywhere, and 50-94% after an article or determiner, while Auto-capitalization is on. A 50-94% word with no determiner before it, and any word the list lacks, keeps the dictionary's rule: a verb stays lowercase ("zu leben", "ich frage", "wollt ihr essen") and a word the dictionary knows only capitalised still takes its capital ("heute abend" becomes "heute Abend"). The builder caps at 94, so a capital arrives only after a determiner: a word that is also a common first-person or imperative verb form ("frage", "reise", "drucke") or the stem people type alone ("kauf", "schlaf", "sitz": "ich kauf noch brot", "schlaf gut"); the separable prefixes "teil" and "preis", the adjective "klasse", and the verb forms the counts miss ("filme", "rolle", "presse", "pumpe", "campe", "pflanze", "beichte", "fische", "taufe"); and every two-letter "word", a chat abbreviation ("gl", "rn") rather than a noun. The determiner list includes "in" ("in Ruhe", "in Frage") but not "ohne" ("ohne essen" is a verb). The list is HEXKeyboard/TypingModel/de.capitals, built by scripts/build-german-capitals.py from the same Google Books shards: for each model word it compares the 1980–2019 counts of the all-lowercase and first-letter-capitalised spellings and lists word<TAB>percent for the words at least 50% capitalised and seen at least 40 times, leaving out pronouns and possessives capitalised for other reasons (FUNCTION_WORDS: "sie", "ihr", "mein", and the rest). Rebuild it from ios/ with python3 scripts/build-german-capitals.py HEXKeyboard/TypingModel/de.hextm HEXKeyboard/TypingModel/de.capitals; it streams the eight shards through curl and gzip four at a time, caches each shard's counts under scripts/.build-german-capitals, keeps their merged per-token counts there so a rule change renders without streaming again, and applies the caps after counting. Off, Deutsch keeps the dictionary engine exactly as before: no decoder, no word list for the letter odds or model completions, and no model for a bilingual partner. Swipe typing stays English and Spanish only.
    • Adaptive touch (Settings › Experiments › Adaptive touch): Off by default (experiments.adaptiveTouch). The keyboard learns where you land on each physical letter key, on this device, and Spatial autocorrect's decoder and the strip read your taps with those personal centres (AdaptiveTouch.swift). Keycaps, hit rectangles, the next-letter claims and Swipe typing are unchanged, and it never makes an automatic correction the keyboard wouldn't make with it off: space puts in a personal reading only when the centres read without it also clear it by the margin (AutocorrectEngine.decoderBaselineKeys, TypingDecoder.replaces), else their own decision, and joining two words at the space bar's edge is theirs alone to decide; what only the personal centres read better is offered in the strip. While on, Learn how I type's per-letter offsets stop moving the decoder's centres (they stay that baseline), and its word-level learning carries on.
      • What teaches it (AdaptiveTouchLabels.swift): only taps with evidence of the letter meant, and only once the next word shows the word stood: a word left as typed (letters the key under the finger typed, not ones the next-letter odds claimed), a picked suggestion of the same length (at most two letters changed), a word kept over an autocorrection by reverting it from the strip or tapping it, and a letter erased and retyped as a neighbour (the erased tap is labelled with the letter that replaced it). Never an autocorrection left in place (or a word before it revised, or two joined), a swipe, a slide, a strip touch, an accent pick, or a password, code, email, URL, secure or no-autocorrect field. Taps and words wait in memory for at most two words and 30 s, and are dropped when the cursor moves, the host changes the text, the field or the keyboard's size changes.
      • What it keeps: sums per physical key in four rotating bins of 200 labelled taps, for each context (letters, posture, orientation, split, Hex key side, width) and one of two grip profiles: offsets in keycaps (so a new key plane of the same size keeps them), shrunk toward the row and the keyboard, at most 0.3 keycap and 0.05 per update, fixed spread. No letters, words, times or sequences: 23,588 bytes, in a file in the keyboard's own container with Full Access (protected, excluded from backup, never in the App Group or its exports, written in batches off the main thread), in memory only without it. Settings shows that it resets without Full Access, Learning (n of 400 taps) or Active from counts the keyboard reports; Reset touch learning, and Delete what Hex learned, bump a reset generation that drops it from memory and disk (the file is the keyboard's, so it goes the next time the keyboard runs) and that a write queued before can't undo.
      • When the decoder uses it: a profile is used only after 400 labelled taps over three sessions, while it fits your recent taps better than the centres read without it (log-likelihood per tap, each axis, over 32 labels). Once it fits worse (a new grip, another hand), the keyboard goes back to them at once and labels teach the second profile; a grip that comes back gets its profile back. Before the first 400 taps nothing is in use, and a new grip just mixes into what is learned.
    • Offensive words (Settings › Keyboard › Suggest swear words): like Apple's keyboard, the strip and autocorrect never offer slurs, and offer swear words only with Suggest swear words on (off by default; App Store rating 4+). The list (HEXShared/FullKeyboard/never-suggest.tsv: layout, slur or swear, spelling or two-word expression; * lines apply to every layout) is LDNOOBW as curated for Hex for Android, entry for entry the same as android/app/src/main/assets/typing/never-suggest.tsv (see Acknowledgements below); change both together. OffensiveWords is checked when a suggestion is looked up: UITextChecker's guesses and completions, learned fixes, and Spatial autocorrect's readings (TypingDecoder.NeverSuggest, so the models still score them as words). An expression is blocked only as the pair ("blow job"), never its words alone. A word it may not offer, typed in full, is never corrected (as on Android); your own UILexicon words (Contacts names, text replacements) are always offered.
    • Swipe typing (Settings › Keyboard › Swipe typing): On by default (stored as experiments.swipeTyping from when it was an experiment off by default: unset or true is on, false off, so testers who turned it off keep it off), for the English and Spanish layouts only (every other layout keeps a drag as a slide to another key) and off under VoiceOver. A touch that goes down on a letter and travels more than a key's pitch (width plus gap) is a swipe (SwipeTyping): its key no longer types, a light trail (a shape layer, the last 0.45 s of the path) follows the finger, and the strip shows what the path reads as so far, decoded off the main thread at most every 70 ms. Lifting types the best reading after a space, unless the text before ends in a space, a line break, an attachment (U+FFFC: a shared link's card in Messages, a picture in the text) or opening punctuation (( [ { « “ ‘ ¿ ¡ @ # /, or a straight quote after a space), as Apple's QuickPath does; shift capitalises its first letter (caps lock all of it), the space it adds can start a sentence ("Hi." then a swipe types " There"), and a name the dictionary only knows capitalised is capitalised. The strip then offers the next three readings, the likeliest in the centre; tapping one puts it in the word's place. Until anything else changes the text, delete takes the whole word back (with the space it added), unless Settings › Experiments › Delete swiped words letter by letter is on (experiments.swipeDeleteLetterByLetter, off by default), which makes delete act as anywhere else and take one letter at a time; space never autocorrects it. As on Apple's keyboard (watched on iOS 27.2), a space stays pending after the word: a tapped letter, digit, opening mark or symbol goes in after a space ("hello" then w-o-r-l-d types "hello world", "(" types "hello ("), while , . ! ? ; : ) ] } ' - / go against the word ("hello,") and take that space; shift you set stays, and delete then takes the letters and the space one at a time. A second finger going down ends the swipe first, so its word goes a space before that key's letter. On the iPhone Duo's split keyboard a swipe reads across the middle between the halves: its points are read with the right half pushed against the left and the middle shrunk to one key's gap (KeyboardLayout.joinedForSwipe), so either half's G and V is the one letter, as for Spatial autocorrect's taps. Taps, holds for accents, slides shorter than a key, delete's word slide, the space bar's trackpad, 123 and the Hex key work as before: no swipe starts on them, and none once an accent menu is open. A swipe that lifts on the key it went down on, never more than 1.6 key pitches from it, is a tap that slid: its letter goes in as a tap's would ("a" dragged a little too far and back), unless it read as a word that starts and ends on that key ("lol"). SwipeDecoder is a shape-matching decoder after SHARK2: the path is resampled to 32 points in key pitches; words come from the language's typing model by the keys the path starts and ends near (each start letter is one range of the sorted word list, read in place from the memory-mapped model, so nothing is built or kept between swipes), kept when every key lies near the path and their ideal path (the polyline through their keys' centres) is about as long; each is scored on location (point by point, then dynamic time warping for the best 80), shape, and how near its first and last keys the finger went down and lifted, the best 80 also on how their keys account for where the finger turned and, from the touches' times, where it slowed (an edit distance: a turn or pause no key of the word explains costs, and so does a key the word goes straight through without a pause on it), and weighed with the model's word-pair frequency after the word before. A straight stroke runs over the letters between its ends, so the path alone can't tell "tired" from "tied" or "wire" from "write" (X tester: guesses didn't account for the letters under the swipe); a pause on r can. English contractions, which the Google Books model barely counts, and your own words (Contacts names and other UILexicon terms, one-word text replacements, words you kept) are read beside it. It never reads a slur, and reads swear words only with Suggest swear words on (TypingDecoder.NeverSuggest and mayOffer, as the strip). Its weights are tuned on synthetic swipes of the autocorrect benchmark's held-out dev sentences and on mined sets of common words that share a path; on the held-out test sentences (typical fingers, 390 pt) top-1 is 97.1% in English and 94.2% in Spanish (94.9 and 90.9 without the turns and pauses), and on held-out words that share a path 96.0% and 94.5% (85.1 and 81.2); a decode takes 0.91 ms at the median and 2.85 ms at p99 on an M5 Max (experiment folder 2026-11-20-hex-swipe-typing, results/2026-11-21). Record typing logs keeps a swipe's touch moves and a swipe_end line (points, decode time, readings) for fitting it to real fingers.
    • Record typing logs (Settings › Privacy › Advanced › Record typing logs): Off by default, and apart from Learn how I type. A one-time migration (migrations.typingLogsOff.v1 in the App Group, TypingLogSettings.turnOffOnce, run by the app and the keyboard at launch) turned it off for everyone who had it on, keeping the logs already recorded. Its description says what it records, and turning it on asks first in a dialog (App Review 2.5.14): each tap and the text you type or dictate with Hex, including the text around the cursor, for 14 days, on this iPhone, to export or delete, never password, code or card fields. Turn On turns it on; Cancel leaves it off. Turning it off doesn't ask. For testing and training prediction and per-user touch models: the keyboard writes every touch on Hex's keys (raw position, radius, UITouch timestamp, the key it resolved to), every layout with each key's frame and touch target, what each key did to the text (inserts, deletes, autocorrections and undos, suggestion picks, double-space periods, returns), the suggestions shown, the host's text around the cursor after each word and when the host changes it, and each dictation's steps (heard, learned fixes, cleanup, inserted) with its outcome. One NDJSON file a day in the App Group (TypingLogs/), kept 14 days, at most 64 MB a day; Privacy › Advanced › Typing logs shares a day or all of them (AirDrop to a Mac) and deletes them. Password, one-time code, and card fields write one skipped line and nothing else. Events are buffered on the main thread and encoded and appended on a utility queue every 2 s. Nothing is sent. The keyboards look the same with it on as with it off: it is only the switch. The schema is in docs/typing-logs.md.
  • Settings (ContentView, SettingsPages.swift): Home fits one screen: the Hex header, one status line ("Ready" with a check, or "Finish setup · 1 step ›"), Dictate, and four rows that each open a short page, then Keyboard settings (Hex's page in iOS Settings) and Run setup again, and last Privacy policy, Support and Acknowledgements. Privacy policy (App Review 5.1.1(i)) and Support open in Safari inside Hex (HexLinks, one base URL: https://dev.anoma.ly/hex until the production site serves anoma.ly/hex/privacy). Each row has a one-line description at most. The pages:

    • Dictation: Style, Lowercase, Format lists, Spoken punctuation, End with a period, Dictation language, Show what I say and Type as I speak (iOS 26 and later), Translate what I say, Clean up what I say, Custom vocabulary (its list below the switch), Keep mic warm, and Microphone.
    • Keyboard: Keyboard, Full (the default) or Dictation bar (stored as FullKeyboardSettings.isEnabled, App Group key experiments.fullKeyboard from when the full keyboard was an experiment: unset or true is Full, false the dictation bar, so testers who turned that experiment off keep the bar; an iPad follows the same choice, where until build 117 it always showed the bar), Auto-start (Never, On switch by default, or Always) while the dictation bar is chosen, then Languages, Number row (off by default), Auto-capitalization, Autocorrect (with Spatial autocorrect below it while it's on), Suggest swear words, Vocabulary (Hex's custom vocabulary list, how many words it holds) and Key pop-ups only while Full is chosen, Haptics (the dictation bar uses it too), then Swipe typing and Emoji (Clear emoji history while it's on), which say they need the full keyboard while the dictation bar is chosen, and a field to try the keyboard in.
    • Privacy: "Everything stays on this iPhone", Learn how I type (what it learned below the switch), and under Advanced, Record typing logs (off by default) with its Typing logs viewer while it's on.
    • Experiments: Hinglish (Hindi in Latin letters), Greeklish (Greek in Latin letters), Two-language dictation, Clean up with Apple Intelligence, Adaptive touch (its status and Reset touch learning below the switch), German spatial autocorrect, Korean keyboard (두벌식), Dvorak punctuation keys, Comma and period keys, Quick accents, ABC under the emoji key, Keyboard themes (its Theme menu below the switch while it's on), Two-thumb selection, Delete swiped words letter by letter, Hide word suggestions, Join the keyboard when unfolded, and Multilingual model (Parakeet).

    Simulator scenes: --snap-state settings-home (ready), setup (keyboard added, Full Access not confirmed yet), setup-sheet, setup-sheet-microphone (before iOS has asked for the microphone), and settings-<dictation|keyboard|privacy|experiments>[-on][-end]. Each writes its page's switches to the App Group: off by default, on with -on; switches that are on by default (Spatial autocorrect, Swipe typing, Emoji) stay on. -end starts the page scrolled to the bottom. settings-privacy-confirm opens Record typing logs' consent dialog. --experiment-keyboard-themes turns Keyboard themes on, and --keyboard-theme automatic|black|white|navy|forest|ember picks the theme the full keyboard draws; the dictation bar keeps the system look, and iOS draws its own strip above or below a third-party keyboard, which a theme can't color.

  • Dictation language (Settings › Dictation › Dictation language, iOS 26+): Automatic by default: the first of the iPhone's preferred languages (then its region's locale, then US English) that Apple Speech transcribes, so a Norwegian iPhone dictates Norwegian with no setup; Nynorsk tries Bokmål, the only Norwegian Apple has. Apple Speech's Spanish includes es-ES, es-MX and es-US (and es-CL on recent systems), and Spanish follows the iPhone rather than always Spain: a region Apple has keeps it (es-CL on a Chilean phone), es-419, es-CO, es-AR and any other Latin American, United States or unlisted region is the Americas' Spanish (es-US, then es-MX), plain es is the iPhone's Region's (es-MX in Mexico, es-US in the United States, Puerto Rico and Latin America, es-ES elsewhere), and only Spain and Equatorial Guinea stay on es-ES (DictationLanguageSwitch.spanishCandidates, spanish(forRegion:)). Before iOS 26 SFSpeechRecognizer lists es-CO and es-419 itself, so the on-device pick keeps them, then falls through es-US, es-MX and Spain's. While the keyboard switches between two or more languages, Automatic follows the one it shows (Keyboard languages, below). The menu also lists every locale SpeechTranscriber or DictationTranscriber supports, by name. SpeechTranscriber is used wherever it has the language (as before); DictationTranscriber (preset longDictation) covers the rest (SpeechLanguagePicker, AppleSpeechLanguages, AppleSpeechModule). The setting lives in the App Group (settings.dictationLanguage, nil for Automatic) so in-keyboard dictation uses the same rule and the same downloaded assets; both record the language they resolved (settings.dictationLanguage.resolved). Picking a language prepares (and if needed downloads) its model at once, or after the dictation in flight. An iPhone without SpeechTranscriber (A13: iPhone 11, SE 2nd generation, all 3–4 GB) dictates English with DictationTranscriber on iOS 26+: loading Parakeet there sat at 50% (Core ML's compile) until iOS ended Hex, at every launch. Parakeet stays where it fits (6 GB) and before iOS 26, for English only, on an iPhone without SpeechTranscriber: an iPhone that has it keeps English on Apple Speech, on DictationTranscriber too while SpeechTranscriber's English is set aside, and goes to Parakeet only once neither Apple module's English installed twice in a row (TestFlight build 126: one failed install sent an iPhone 15 Pro Max to a Parakeet download that stalled, until a reinstall). When Apple Speech doesn't answer, English stays on it there too, and any other language stays on Apple Speech, which says it can't dictate it, rather than going to Parakeet as English (DictationController.appleSpeechLanguage). After Hex ended twice in a row while loading Parakeet, the next launches wait for Set up instead of loading it again, for a day and on that build only (ParakeetLoadGuard: iOS ending Hex in the background or a swipe away mid-load once is no evidence), and a Parakeet download that sits at one fraction for 90 s ends with a message. A preparation that says nothing for 30 s ("Checking the speech model": Speech or its asset store not answering), or nothing new for 120 s after, ends with "The speech model isn't ready. Try again." (watchPreparation), as does the keyboard model's check behind setup's last step, whose row then offers Try again. Apple's reservation and asset status are watched at 60 s the same way (SpeechModelDeadline), a reservation or release that never answers is forgotten so the next attempt asks again, and a keyboard check that hangs counts no strike toward Hex's handoff (SpeechModelNeedsHex.keyboardError). When a model won't install (SpeechModelInstall, shared by the app and in-keyboard dictation): Apple lists some languages whose SpeechTranscriber asset never arrives ("transcription.ru asset not found after attempted download"), so Hex then installs the same language on DictationTranscriber, and after the second such failure in a row goes straight to it for three days. A download whose progress doesn't move for 60 s is given up on. When neither module installs while the iPhone is online, twice in a row, Automatic sets the language aside for three days and dictates the next one ("Russian isn't available on this iPhone yet. Using English."); a language picked by hand says so and offers that one ("Try English", with Use English in the app). A model that installs clears the failures so far. Only Hex's own failures set anything aside: in-keyboard dictation's are often its own (each process holds its own reservation, one released, removed or over Apple's limit reads as not installed there, "Too many asset locales allocated"). With nothing else to move to (an English-only iPhone), Automatic tries the set-aside language again rather than none, so "Apple's speech recognition can't dictate this language" means neither module lists any of the iPhone's languages. Offline or stalled, it says the download failed and tries again next time. In the keyboard, a model it can't get itself says "Hex couldn't transcribe that. Try again." the first time, and the second time in a row (within 6 h, no model in place since) "Open Hex to download English.": Dictate and the Hex key then open Hex, which downloads it, until a model installs (in either process; Hex checks again each time it comes to the front), a language is chosen, or a day passes (SpeechModelNeedsHex, settings.dictationLanguage.keyboardNeedsHex); a move to another language keeps its own sentence. Coming back to Hex after it was in the background starts over a check that failed or never answered, and a check that gives up bumps its generation so a late answer can't reopen the row (KeyboardSpeechPreparation). Reservations make room under AssetInventory.maximumReservedLocales by releasing others first. Only Hex's own error text reaches the app or keyboard (HexDescribedError); Apple's and other packages' descriptions go to logs, however readable. The setup sheet's Speech model row names the language ("Norwegian Bokmål · Apple"). Steps that only know English follow the language: Clean up what I say (an English tagger) uses word rules in Spanish, German and Norwegian and skips other languages; Clean up with Apple Intelligence runs only when the on-device model supports the language; the styles trim English fillers (um, uh, er, ah) only in English, Nordic ones (eh, ehm, øh, hm) in Norwegian, Danish and Swedish, Spanish ones (eh, em, mmm) in Spanish, German ones (äh, ähm, öhm) in German, and none elsewhere (Norwegian "er" is "is", Portuguese "um" is "a"); translation's source is the dictation language rather than a guess. Custom vocabulary works in any language.

  • Two-language dictation (Settings › Experiments › Two-language dictation, iOS 26+): Off by default (experiments.twoLanguageDictation, App Group). While on, and while dictation uses Apple Speech's SpeechTranscriber, one recording is read twice: in the language dictation resolved (unchanged) and in the first other language in Settings › Keyboard › Languages whose base language differs from it, mapped by the same rule the keyboard-follow language uses (LanguageSettings.dictationLanguage(for:)). Both must resolve to SpeechTranscriber, and the second language's model must already be installed (AssetInventory.status): dictation never downloads for it, and a language whose SpeechTranscriber model was set aside as failing is skipped. Two SpeechAnalyzers read the same WAV, each with its own AVAudioFile, prepared with prepareToAnalyze(in: nil) before starting, and a SpeechTranscriber(locale:, transcriptionOptions: [], reportingOptions: [], attributeOptions: [.audioTimeRange, .transcriptionConfidence]); both get custom vocabulary's contextual strings. Each final result's timed runs give the merge its words (the run's own characters, leading whitespace included, start, end, confidence), and a run without letters or without a time range keeps its characters on the word before it. TwoLanguageMerger cuts both readings at the earlier of two word ends within 150 ms, scores each reading in a segment by its confidence-weighted seconds clipped to the segment, and picks a reading per segment with a two-state Viterbi that charges 0.35 s of confident audio for each language change, ties going to the primary. A word the new reading starts with is dropped only when it repeats the last word kept, case and punctuation aside, and the two spans overlap by at least half of the shorter one. The winning runs are joined with their own characters, inserting one space at a language change only when neither side has whitespace there. Whenever the second language wins no piece, the primary's original text is used unchanged, so Chinese, Thai, punctuation and untimed runs are exactly as Apple gave them. The text then runs through the same pipeline with the primary language. No second installed language, a second analyzer that throws, or no timed words in either reading dictates exactly as before; a primary reading that finished without hearing anything reports empty so its caller throws its own no-speech error without transcribing again; with the switch off no two-language work starts. TwoLanguageMergeTests cover the merge and the run extraction. The app, keyboard, widgets, and tests built on a macOS 26 GitHub runner with 33 tests passing (run 45005571072, the latest observed verification); nothing here has been measured on an iPhone, and the simulator can't transcribe.

  • Switching the dictation language in the keyboard (iOS 26+, Full Access): the dictation bar shows the language beside the style ("Norsk ⌄") and opens a tray of chips; the full keyboard's Hex key menu has a Language row with its own page, and the space bar names the language faintly ("English", "Norsk"), unless the keyboard switches between two or more languages: then it names the keyboard's. Both offer Automatic plus up to five languages: the one set, the ones picked lately (settings.dictationLanguage.recent), then the iPhone's preferred languages, each matched to one Apple Speech has (DictationLanguageSwitch; the keyboard lists Apple's languages once per process and caches them in settings.dictationLanguage.supported). One variant per language, the first of the current, the recents and the preferred winning, so Spain's Spanish and Mexico's never both take a chip; picking one drops its siblings from the recents (Norwegian's nb/nn/no aliases stay one language). Chinese stays apart, so Mandarin's zh-CN, Taiwan's zh-TW and Cantonese's zh-HK each keep a chip and a pick keeps the others in the recents. Names are each language's own ("Español", "Deutsch", "עברית"), with the region where Apple Speech has more than one of that language ("Español (US)" or "Español (ES)", "English (GB)"; Norwegian stays "Norsk"), so the menu and the dictation bar's chips say which variant dictation actually uses. Under Automatic, Settings › Dictation language says the same itself while the keyboard follows: "Follows keyboard: English (US) · Spanish (US)" names each keyboard language's dictation language, and is refreshed when Hex comes back to the foreground, not only when the row appears. A long press on a language (a chip, or a row on the Hex key menu's Language page) shows Remove, which takes it off (settings.dictationLanguage.removed, so one of the iPhone's own languages stays off too) and puts the one set back on Automatic; Settings › Dictation language's Remove from keyboard does the same (TestFlight build 56: a language couldn't be removed, and one could only be added in Hex). More languages… lists every other language Apple Speech has, the iPhone's own first, and picking one there adds it (in the dictation bar the list takes the keys' place). Removing or adding one never changes Automatic, which still follows the keyboard's language. A pick writes the same App Group setting as Hex's menu, so in-keyboard dictation uses it at once; a warm Hex session follows it within 100 ms and prepares the language between dictations, and a handoff or the Action Button picks it up before recording.

  • Translate what I say (Settings › Dictation, iOS 26+): Off by default. Translates each dictation on the iPhone with Apple's Translation framework into the language picked under Insert in, once its pack is downloaded. When it can't (no pack, a pair Apple can't translate, or a translation that fails twice, each in a new TranslationSession, never one kept), the transcript goes in as spoken and the keyboard says so for 8 s: "Couldn't translate, pasted as spoken" over the fix ("Download Chinese in Settings"), whose Open Hex (the dictation bar: a tap) opens Settings › Dictation (hex-dictation://keyboard/translation). Settings keeps the last failure (experiments.translateDictation.lastFailure, with Apple's error code, never the text) beside the download until a dictation translates. With Hinglish on, Hindi going into another language is translated from its Devanagari, and romanized only if it goes in as spoken; Greeklish does the same for Greek, translated from Greek letters. Where you dictate, a "Translate to English" switch turns it on or off without opening Hex (testers asked for a button, not a setting): a row in the full keyboard's Hex key menu, lit with a checkmark while on, and a chip in the dictation bar's language tray. It writes Settings' own switch (experiments.translateDictation), the only switch: nothing else turns translation off (build 56's "Paste in Hindi" left experiments.translateDictation.pastesTranslation false with Settings still on; that reads as off, and the next switch removes it), and Settings re-reads it when it shows. Below it, Translation language lists up to six targets (the current one, English, the dictation and preferred languages, then common ones, only those Apple translates into); picking one sets experiments.translateDictation.target and turns translation on. Settings keeps the full language list and the pack download under Insert in. While dictations paste translated, the Hex key wears the target's code ("EN") in its corner, the dictation bar shows "→ EN" beside the Hex mark, and the space bar reads "Español → English". The App Group carries the switch, so in-keyboard dictation, a warm session, and the Action Button all follow it. The keyboard shares the app's String Catalog, so the Hex key menu is in Chinese on a Chinese iPhone.

  • Keyboard languages (Settings › Keyboard › Languages, with Full chosen): Automatic (the default, as before: the dictation language's layout, below), or one or more layouts ticked and dragged into order (KeyboardLanguagesPage, App Group settings.keyboardLanguages, which reads the one-layout settings.keyboardLayout until first written and writes its first language there). The layouts of the iPhone's own keyboards and preferred languages come first ("On this iPhone"); ticking one from Automatic keeps Automatic's language beside it, and unticking the last goes back to Automatic. With two or more, the space bar names the current one ("Français", with the region only beside another French layout) and a quick swipe along it switches (left the next, right the one before): letters, accents and autocorrect. English and Español chosen together are Apple's bilingual "English & Spanish" keyboard (KeyboardLanguageList.layout): both type on the first one's keys and long presses, so English first is QWERTY with ñ on a held n in both, Español first keeps its ñ key in both, and the swipe switches the language they autocorrect and dictate in (black box, iOS 27.2 simulator: Spanish added to Apple's English keyboard has no ñ key and English's long presses; English added to its Spanish keyboard keeps the ñ key and Spanish's). English and Deutsch together are its "English & German" keyboard the same way (iOS 27.2 and 26.5 simulators): German added to the English keyboard is QWERTY with English's long presses, English added to the German keyboard keeps its layout, QWERTZ – Deutsch's ü ö ä keys or, on QWERTZ, ten keys a row with ä ö ü and ß held. Apple and Gboard switch with a long press on the globe, but a Face ID iPhone shows no globe in a third-party keyboard and Hex's space-bar slide is its cursor trackpad, so this follows SwiftKey: with two or more languages the cursor slides once the space bar was held still for 0.35 s (the hold that starts Apple's trackpad, below), a slide before that is a swipe, and one shorter than 24 pt types a space (FullKeyboardEngine.trackpadHoldDelay, languageSwipeDistance). The globe key is unchanged. The keyboard comes back in the language it was left in, in every app and field, as Apple's keyboards do (black box: a switch in Messages held in Safari and back; settings.keyboardLanguages.current; without Full Access an iPhone may keep it only while the keyboard runs). The input mode says what it types: Hex sets UIInputViewController.primaryLanguage to the language's BCP 47 identifier (KeyboardLanguage.inputModeLanguage) as the language changes (unit-tested), meant to make the host spell and autocorrect in the language typing; Apple documents it as superseding the extension's Info.plist en-US, but whether the host follows it (Notes's red underlines, for one) is not verified on a device. Switching from the full keyboard to the dictation bar leaves the last reported language in place. A field gives a third-party keyboard nothing else to go on: before this change documentInputMode was observed as nil in Messages and the plist's en-US in a field that asks for English; what it reports with the runtime language is unverified. Dictation follows the keyboard: with two or more languages and Dictation language on Automatic, dictation is in the language the space bar shows, in the iPhone's region where Apple Speech has it (LanguageSettings.keyboardDictationLanguage, KeyboardLanguage.dictationLanguage; none for Srpski), in the keyboard and in Hex, which prepares the new language between dictations (TestFlight build 56: with the keyboard on Español, dictation stayed English and Spanish came out "incomprehensible"). Spanish is the iPhone's Spanish, not always Spain's: a region Apple has keeps it (es-CL on a Chilean phone), es-419, es-CO and any other Latin American or United States region is es-US, then es-MX, plain es and no Spanish preferred language follow the iPhone's Region (es-MX in Mexico, es-US in the United States, Puerto Rico and Latin America, es-ES elsewhere), and Spain and Equatorial Guinea stay on es-ES. A Spanish variant picked by hand for that language is used again when the space bar comes back to it: the recent pick goes ahead of the preferred languages, so es-US or es-MX survives the switch away and back. It doesn't where the iPhone's own language has no layout (Hebrew), as dictating is how that language gets written. Switch dictation too (off by default, shown with two or more on iOS 26+) follows there too, and puts a dictation language picked by hand back on Automatic at each switch (it used to write the keyboard's language as a pick, which stayed after Spanish was unticked or the switch turned off). Bilingual autocorrect: the first other chosen language with Latin letters and an iOS dictionary (KeyboardLanguageList.partner: English beside French, French beside English; none for Русский, Українська, Ελληνικά, Srpski, or two French layouts) is checked on the same keys, with its Spatial model where it has one, as Apple's "Français et English" keyboard does: a word either language knows as typed is never changed, a typo only one can fix gets that fix ("teh" → "the" while French types), and when both can, the language of the last words only one dictionary knows wins, else the clearly cheaper slip, else the keyboard's own; the strip shows fixes and completions in the language the words before are in, never the other's, even where that language has none (TestFlight build 56, Čeština and English: "autocomplete suggests English words in Czech too"; the iOS 27.2 simulator's Czech dictionary completes nothing, and English filled the strip), and with nothing before to tell, both languages', in turn. In the simulator the English keyboard alone changed 34 of 77 common French words and the French keyboard alone 27 of 80 English words; with both chosen, none. Any two Latin-script languages with an iOS dictionary pair this way (English with Français, Deutsch, Italiano, Português, Polski, Čeština, Norsk, Română, Español, or two of those), whatever their order; Polski's simulator dictionary takes every word, so the any-word guard (#2143) leaves it out there. The words before count for a language when only its dictionary knows them, or, from three letters, when both do but only its typing model has them among its commonest (iOS's French and German dictionaries take "the", "you", "your" and "please"). Once they are the keyboard's own language, a word the other dictionary knows still gets its sure fixes (a missing apostrophe or common slip from unambiguousFixes, a two-letter slip); a word without a vowel never counts as the other's ("dk", "tkm" are abbreviations there); the other's name guesses ("Zuri" for "Ziri") are never fixes; Spatial autocorrect may still read the taps as another word, counting the other's word as one it changes only on its valid-word margin; and at a sentence's start, "Im" and "Dont" are revised to "I'm" and "Don't" once the next word leans English (bilingualRevision; German "Im Haus" stays). A word kept because only one language knows it as typed takes the other's accents once the next word leans that one, as Apple's keyboard puts "nao sei" right at "sei" (TestFlight build 56, English and Português): "voce esta" becomes "você esta" ("voce" is an English word), only for a lowercase guess of that dictionary's that adds accents alone, and not when the words before lean the other way (accentRevision). "nao" alone was already "não" beside Português. Held-out English benchmark (2,000 Tatoeba sentences' synthetic taps, Spatial autocorrect on, iPhone 17 simulator, iOS 27.2, keys as of #2180): English alone 5.75% WER; with Français beside it 7.56% before, 5.79% now (99 correct words broken against 125 before and 113 alone); with Deutsch 7.53% before, 5.79% now; with the German Spatial experiment on, 7.18% before, 5.70% now. Typed at a key's centre after "we said that", 1 of 80 common French words changes ("dont" to "don't") and 3 of 72 German ones ("auch", "ist", "habe"), none before; none at a message's start or after French or German words, and a slip in a French or German word at a message's start is fixed as often as before (26 of 50, 18 of 40). Per-letter engine p95 with a partner 4.1-5.4 ms (3.9-5.1 ms before), 2.2 ms alone. Black box (iOS 27.2 simulator): Settings › Keyboards offers "Add to English (US) Keyboard" for French (France, Canada, Switzerland), "Type in English and French on the same keyboard", which makes one "English & French" bilingual keyboard on English's QWERTY with no language switch; Portuguese, Polish and Turkish offer "Add to Existing Keyboard". There it fixed "I dont know" and "Dont do it" to "don't", kept "est" and "merci" after English words, and wrote "Im" as "I m"; the simulator's French word data is thin ("je pense" became "he pence"), so how it weighs French there is unverified. 한국어 (두벌식): Settings › Experiments › Korean keyboard (off by default, experiments.koreanKeyboard) adds 한국어 to the list: the standard Dubeolsik rows with shift's ㅃㅉㄸㄲㅆ and ㅒㅖ, and no qwerty/azerty choice. HangulComposer composes each syllable in the field (한, then delete to 하, ㅎ, nothing; ㅘ splits to ㅗ; 닭 + ㅗ → 달고, 한 + ㅏ → 하나), the keyboard replacing only the run's changed characters through deleteBackward/insertText (no marked text). Space, return, punctuation, a page or language switch, emoji, dictation, paste, undo, a selection action, the delete-word swipe, a cursor move or hiding the keyboard commits the run. The cursor check records the last 32 characters before the run, the text after the cursor and the field, plus the host's selected text, so a move to another place whose text before it ends alike commits too while a host that trims the front of its window still matches if the window holds at least those 32 characters plus the run; a host that trims more than the tail or reports no selection is best-effort, and a host that reports no text before the cursor joins syllables only while jamo come within about half a second (a cursor move followed by a jamo inside that window can't be detected). A field whose text after the cursor changes while you type (inline autocomplete) commits each jamo, so syllables don't join there. Korean has no accent holds, spelling dictionary, autocorrect, word suggestions or typing model, and with the experiment off the language isn't listed, Automatic never picks it, and a stored "ko" is ignored. Unit tests only (HangulComposerTests, HangulKeyboardTests, KoreanKeyboardGatingTests): no CI compiles the iOS targets and no device run checks it.

  • Layout per language (Settings › Keyboard › Languages › Layout): each language the keys show (every chosen one, Automatic's while none is, and only the first of English and Español together) gets a menu of the layouts Apple's Keyboard Layout list offers for it (iOS 27.0 simulator, black box), its own (today's) first: English QWERTY, AZERTY, QWERTZ, Dvorak; Deutsch "QWERTZ – Deutsch" (with ü ö ä), QWERTZ, QWERTY, AZERTY; Español "QWERTY – Español" (with ñ), QWERTY, AZERTY, QWERTZ; Français (France) AZERTY, QWERTY, QWERTZ, (Canada) QWERTY, AZERTY, QWERTZ, (Suisse) QWERTZ, AZERTY, QWERTY (Français (France)'s AZERTY is Apple's "AZERTY – French", the first layout Apple lists for France and Belgium, with the apostrophe key; Canada's QWERTY and Switzerland's QWERTZ are Apple's "QWERTY – French" and "QWERTZ – French", the first each lists, with the key ending row 2; every other French layout is the plain one); Italiano and Português QWERTY, AZERTY, QWERTZ; Polski and Română QWERTY, QWERTZ, AZERTY; Čeština "QWERTZ – Čeština" and "QWERTY – Čeština", Apple's "QWERTZ – Czech/Slovak" and "QWERTY – Czech/Slovak" (iOS 27.2 simulator), both its own rows with the ´ and ˇ keys, Z and Y swapped on the second; Magyar "QWERTZ – Magyar" and "QWERTY – Magyar", the standard Hungarian QWERTZ with every accented vowel on a held key, Z and Y swapped on the second (Apple's own Hungarian layout list wasn't read on a simulator for this change); Srpski QWERTZ (Hex's, though Apple now adds it as QWERTY), QWERTY, AZERTY. Norsk, Русский, Українська and Ελληνικά keep their one layout (Apple also offers a phonetic "Russian – QWERTY", not built). The plain layouts are the same 26 letters in every language, with the language's bottom row and 123 pages; only the letters move (KeyboardLetterLayout, KeyboardLanguage.letterRows(_:)). Holds and autocorrect stay the language's: English on QWERTZ (the TestFlight request, build 56) holds what English QWERTY does, and a plain layout without the language's own letter keys holds them right after their base letter, as Apple's does (German QWERTZ: A Ä À…, O Ö…, U Ü…; Spanish QWERTY: N Ñ Ń). Apple's Dvorak is P Y F G C R L and delete, A O E U I D H T N S from the edge, shift and Q J K X B M W V Z, on keys narrowed so that row fits (38.3 pt pitch on a 402 pt iPhone); split on the Duo it keeps delete on row 3 (Apple's split Dvorak unobserved). Settings › Experiments › Dvorak punctuation keys (off by default, experiments.dvorakPunctuation) puts , and . ahead of P on row 1, whose nine keys narrow to start at the left margin and end at the same delete key, puts ; just left of the space bar where it can give up a letter's width, and types <, > and : with a shift you press or caps lock (auto-capitalization leaves the comma); on the split Duo the two punctuation keys join only the left half, on a pitch narrowed so it keeps its right edge (its top-row keys are narrower), and ; takes from the left space bar. A URL or email field keeps its own . key instead of row 1's, and Experiments › Comma and period keys adds no second pair beside the space bar where this experiment put them on row 1. Off, every layout is key for key the one it was. Kept per language in the App Group (settings.keyboardLanguages.layouts, LanguageSettings.letterLayout(for:)); unset, not offered, or unreadable (an iPhone that keeps the App Group from a keyboard without Full Access), a language types on its own layout. Spatial autocorrect, the touch decoder, swipe typing, next-key growth and the neighbouring-key cost read the keys as drawn, and Adaptive touch learns each arrangement as its own key plane; typing logs' layout lines name it (letters).

  • Keyboard layouts (each language's letters): English, Norsk (å ending row 1, ø æ row 2), Español (ñ ending row 2; beside English, the first of the two's keys, above), Deutsch (QWERTZ with ü, ö ä), Italiano and Português on the English letters (Apple's keyboards for them are QWERTY), Français (France) on Apple's "AZERTY – French" (rows of 10, 10 and 7 keys: W to N and the apostrophe key, a letter wide and centred between shift and delete at their English size, as on Apple's; Hex's keycaps fall on Apple's to the pixel on an iPhone 17 Pro and SE, upright and turned, iOS 27.2 simulator). The apostrophe key (FrenchTyping) shows ’ right after a letter and types the apostrophe, curled as the field curls quotes (l’homme, qu’il), and holds ' ’ ‘ ; anywhere else it shows and types ´ (never a dead key) and holds ´ ˋ ˆ ¨, as Apple's does. It is no letter, so the decoder, Adaptive touch, next-letter odds and swipe typing never read it, and French autocorrect fixes a slip on either side of an elision's apostrophe as a word of its own ("l'hpmme" → "l'homme", "k'homme" → "l'homme", frenchElisionFix). The iPad keeps its own ' and : key instead; Français (Canada) on QWERTY and Français (Suisse) on QWERTZ, as on Apple's "QWERTY – French" and "QWERTZ – French": the same apostrophe key ends row 2, ten keys a letter wide edge to edge under row 1's, its rules and holds France's (keycaps on Apple's to the pixel on an iPhone 17 Pro and SE, upright and turned) (automatic picks them for fr-CAandfr-CH, AZERTY for any other French, Belgium included), Русский (Apple's ЙЦУКЕН: 11, 11 and 9 letters, ё on a held е, ъ on a held ь), Українська (the standard Ukrainian ЙЦУКЕН, ДСТУ 4862:2007, in the rows the iPhone's own Ukrainian keyboard shows on the iOS it runs on, because Apple changed them in iOS 27: from iOS 27 (27.0 and 27.2 simulators) 12, 12 and 10 keys, йцукенгшщзхї, фівапролджєʼ, ячсмитьбюґ, with ʼ the U+02BC apostrophe Ukrainian spells with, and only ʼ offers anything when held, the other apostrophes (' ’ ‘ and the backtick); before iOS 27 (26.2 to 26.5 simulators) Russian's 11, 11 and 9, йцукенгшщзх, фівапролджє, ячсмитьбю, every letter the two share on Russian's key, ї on a held і and ґ on a held г. On both the 123 and #+= pages' apostrophe key is ʼ, held like the letter, and ₴, which the system keyboard offers on its held currency key, is held on $; no ы, э, ъ or ё), Polski (QWERTY, as the standard Polish layout is, on the English letters; it matches the system keyboard's observed long-press options: Polish's own letter right after the base on A, C, E, L, N, O, S and Z, so ą ć ę ł ń ó ś ż, with ź after ż, and nothing on U, I or Y; its currency key is €, held for € $ £ ¥ ₩ ₽ and no zł, which Hex holds on the #+= page's €), Română (QWERTY on the English letters, as Apple's iPhone Romanian keyboard is: ă and â held on A, î on I, ș on S and ț on T right after the base, the comma-below ș and ț (U+0219, U+021B), never the cedilla ş ţ, and nothing on the other letters; Apple's Romanian layout with keys of its own for them is the iPad's; its currency key is €, held for € $ £ ¥ ₩ ₽ as Polish's is, and its held " offers " „ ” » « “), Ελληνικά (it matches the standard Greek layout and the system keyboard's observed long-press options, in the rows the iOS 27.2 simulator's own Greek keyboard shows: ερτυθιοπ´, ασδφγηξκλ, ζχψωβνμ, nine, nine and seven keys, with no ; or ς key; the tonos key ´ (U+00B4) accents the vowel right before it, α´ → ά, Α´ → Ά, ϊ´ → ΐ, and after anything else, an accented vowel included, types ´ itself, as the system keyboard was observed to with real touches in Messages (no dead key, no ¨ key); held, each vowel offers its tonos form, ι and υ also ϊ ΐ and ϋ ΰ, σ offers ς and κ the kai sign ϗ, and in capitals Σ offers nothing and Ι, Υ only Ί Ϊ, Ύ Ϋ, each callout laid out as the system's, the key's own on the left (AccentCatalog.arrangement, as for English and Spanish); σ types ς while it ends the word and turns back into σ when another letter follows, as the system keyboard does; its 123 page has the ano teleia · in ;'s place and ; (the Greek question mark, which also starts the next sentence with a capital) in ?'s, its #+= page ? in |'s, its currency key is $ held as English's, and its held " offers French's order), Čeština (it matches the standard Czech layout and the system keyboard's observed long-press options, in the rows the iOS 27.2 simulator's own Czech keyboard, "QWERTZ – Czech/Slovak", shows: qwertzuiop, asdfghjkl´, yxcvbnmˇ, ten keys edge to edge on rows 1 and 2 and shift and delete a letter wide on row 3; the ´ (U+00B4) and ˇ (U+02C7) keys are no dead keys: with real touches in Messages, each accents the letter right before it, ´ giving á ć é í ĺ ń ó ŕ ś ú ý ź and ˇ č ď ě ľ ň ř š ť ž and ů after u, capitals too (A´ → Á, Uˇ → Ů), and after anything else, an accented letter included, types itself ("aˇ", "á´"); delete then takes the accented letter whole (CzechTyping); held, E offers ě é ê è ę ë ē ė, R ř ŕ, T ť, Z ž ź ż, U ú ů û ü ù ū, I í î ï ì į ī, O ó ô ö ò õ œ ø ō, A á à ą â ä æ ã å ā, S š ß ś, D ď, L ł ľ ĺ, Y ý ÿ, C č ç ć, N ň ñ ń and the ˇ key the ring ˚ (which makes ů and å), each callout laid out as the system's (AccentCatalog.arrangement), and every other key, ´ included, nothing; its 123 and #+= pages and their held keys are English's), Magyar (the standard Hungarian QWERTZ, so qwertzuiop, asdfghjkl, yxcvbnm, on the English keys' QWERTZ; its nine accented vowels are all held, á on A, é on E, í on I, ó ö ő on O and ú ü ű on U, the base first and then the neighbouring Latin accents, ő and ű the double-acute letters U+0151 and U+0171, and every consonant holds nothing; QWERTY is offered too) and Srpski (Latin: Apple's QWERTZ option for it; Apple's iPhone Serbian Latin keyboard has no keys of its own for č ć đ š ž, they are held on c, d, s and z), as Apple's iOS keyboards lay them out (KeyboardLanguage: letter rows, long-press accents, spelling dictionary). Automatic follows the dictation language (Hebrew, Cyrillic Serbian and other languages without a layout get English; Serbian only in Latin script, sr-Latn); before any dictation language is known, the first iPhone keyboard, then preferred language, that has a layout. It doesn't read back the input mode Hex reports (primaryLanguage), which is derived from this choice. Rows of 11 keys narrow only the letters: shift, delete and the bottom row keep their English sizes, and row 3's letters sit centred between shift and delete. Russian's nine-letter row 3 wouldn't fit beside them, so there shift and delete narrow instead, as on Apple's; Ukrainian's rows (twelve keys from iOS 27, Russian's before it) leave them one letter wide, as on Apple's. Long-press options match each language's system keyboard and its observed long-press options (iOS 27.2 simulator: every letter, 123 and #+= key held for about 3 s with a real touch, the callout read from the accessibility tree; KeyboardLanguage.letterHoldsandsymbolHolds, AccentCatalog): a key the system keyboard holds nothing on holds nothing (Spanish Z, Serbian E), letters with their own key drop out (Norwegian A has no å or æ, and Æ and Ø offer Ä and Ö; Spanish N offers only ń), German keeps ß on a held S, and Hex's callout shows the options in the system's left-to-right order with the base letter at the end under the thumb, in one row where the system wraps a long list (English A) into two. Autocorrect and suggestions use the layout's UITextChecker dictionary (nb, es, de, it, fr, pt_BRthenpt_PT, pl, cs, hu, ro, ru, uk, el; Serbian would use sr_Latn, which iOS 27.2 doesn't have, nor a Croatian one); where the iPhone has none they stay off except your own text replacements, and the English contractions and typo list (i→I) apply only to English. Neighbouring-key costs follow the layout (on QWERTZ, Z sits beside T), and an accent or tilde you left out is the cheapest fix of all ("aqui" → "aquí", "manana" → "mañana"; a word the dictionary knows, such as "esta", is never changed). Italian also restores common accented spellings independently of the checker's guesses and a bilingual partner's dictionary, where the plain spelling is no word of its own ("perche" → "perché", "citta" → "città", the days, the -ità words; KeyboardLanguage.accentRestorations); "e"/"è" and "ce"/"c'è" stay as typed, because "e" is "and" and "ce" is "to us": the strip offers "è" and "c'è", and space never applies them (KeyboardLanguage.accentSuggestions). In Polish every missing diacritic in a word counts as one such fix ("zolw" → "żółw", "maly" → "mały"). The iOS 27.2 simulator lists pl_PLbut its checker flags no word, so Polish fixes are checked against a stub dictionary there. Greek likewise: every tonos or diaeresis left out counts as one such fix, and so does σ where the word ends in ς ("καλημερα" → "καλημέρα", "λογοσ" → "λόγος"); Greek fixes are checked against a stub dictionary too. Czech likewise: every háček or čárka left out counts as one such fix ("cesky" → "český", "prilis" → "příliš"), checked against a stub dictionary, and Hungarian too ("koszonom" → "köszönöm", "kesobb" → "később"), while a word of its own keeps its spelling ("meg" and "kor" stay though "még" and "kör" exist). A two-letter non-word one neighbouring key (or one swap) from a very common two-letter word is fixed too ("tk" → "to", Spanish "nl" → "no", Hungarian "es" → "és";KeyboardLanguage.commonTwoLetterWords`, English, Spanish, Italian and Hungarian). Auto-capitalization is unchanged. The 123 and #+= pages are English's in every language but Greek (above); their held keys follow each language's system keyboard where it differs (# holds № in Russian and Ukrainian, the digits º and ª in Spanish and Italian, . holds · in French, $ holds ₴ in Ukrainian and kr in Norwegian).

  • Size (full keyboard): On an iPhone the suggestion strip is Apple's prediction bar, as thick as Apple's (iOS 26.5 and 27.2 simulators, iPhone 17 Pro, Air, 17 Pro Max and SE, upright and turned, light and dark, beside Hex in the same field): 44 pt from the keyboard's top edge in every state (suggestions, Undo dictation, Paste, the Full Access hint, the recording bar, the emoji panel's top row), with Apple's 8 pt under it upright (6 pt turned), so row 1's keycaps start 52 pt down (50 turned), the keys stay on Apple's rows, and where iOS adds nothing above the keyboard Hex is exactly as tall as Apple's keyboard: 253 pt on an iPhone 17 or 17 Pro, 260 pt on an SE, 272 pt on an Air or Pro Max (KeyboardMetrics.withAppleBar). On an 844 to 874 pt iPhone from iOS 27 Apple's 43 pt row 4 ends 4 pt below where iOS ends a third-party keyboard's view, so those 4 pt come out of all four rows alike: 42 pt keycaps with Apple's 11 pt gaps, row 1 on Apple's line and rows 2 to 4 1, 2 and 3 pt above Apple's, no row shorter than another; every bottom-row key takes taps down to the view's bottom edge. The gap between rows 3 and 4 is the space bar's across its width (row 3's keys whose middle is over it end at their keycaps), as on Apple's keyboard, whose space bar takes taps up to 2.5 pt into row 3's keycaps (iOS 27.2); over 123, emoji, return and the rest the gap splits, so a low m or delete above return never sends return (Hex's rows sit 1 to 3 pt above Apple's, so a finger trained on Apple's lands lower on Hex's row 3). Apple's space bar takes taps 13 pt below its keycap, down to its own key plane's end; the 17 pt under Hex's view belong to iOS's globe band and never reach a third-party keyboard. The 17 pt band iOS adds above a third-party keyboard in some fields (Notes, Spotlight, a re-tapped Safari field; on iOS 26 even a plain first-focused text view) is an exception Hex doesn't design for: there Hex is 17 pt taller than Apple's. Inside the bar Hex draws what Apple's does: the words, SF 17 pt, centred 25 pt down; a pending autocorrection in Apple's bubble, its whole cell across and a 38 pt capsule from 6 pt down to the bar's bottom; 1 × 24 pt dividers between filled cells, hidden beside the bubble; and the Hex key in a 59 pt cell of its own at its end of the bar, as Apple's writing tools icon has at the other (a TestFlight screenshot, iPhone 17 Pro Max, iOS 27.2.1), with a divider between it and the word beside it; the three cells share the rest. iOS 18 is unmeasured and keeps the 38 pt strip it had (247, 254 and 266 pt). Apple's keyboard draws other rows before iOS 27 (iOS 26.2 and 26.5 simulators, iPhone 12, 16, 17 Pro, Air and 17 Pro Max, beside Hex in the same field): the same keycaps, but upright its panel is 7 pt taller under the keys on an 844 to 874 pt iPhone and 2 pt shorter on a 900 pt or taller one, so on iOS 26 Hex's rows sit 7 pt higher (row 4 whole, 260 pt) or 2 pt lower (270 pt) to stay on Apple's (KeyboardMetrics.appleRowsAreIOS26); the SE is the same on both. iOS 18 is unmeasured (the Studio has no iOS 18 simulator) and keeps the iOS 27 rows Hex drew there before. Turned, the bar is 44 pt with 6 pt to row 1 and the keyboard 204 pt (32 pt keycaps on a 40 pt pitch); on iOS 26 Apple's turned keycaps are 27.33 pt on a 35.33 pt pitch ending 4 pt above the bottom, and Hex's are too, 187.33 pt in all (iOS 18, unmeasured, keeps 204 pt) (the SE keeps iOS 27's turned keys for now: Apple's turned SE on iOS 26 is its upright keyboard across the whole screen). Where iOS gives the keyboard a shorter view than it asks for (Spotlight on an iPhone 17 Pro Max, iOS 27.3, TestFlight: about 20 pt short), the rows fit the height it has: Apple's bar gives up its 6 pt over the old 38 pt strip first, so a view up to 6 pt short keeps every key whole and in place, and beyond that the keycaps and the gaps between rows shrink alike (KeyboardMetrics.fitted), instead of row 4 falling under iOS's globe band. Dictating no longer grows the keyboard. On the iPhone Duo (KeyboardPosture, model iPhone19,4) iOS sets third-party keyboards in an inset platter with a band of its own above them, so the strip has no gap there, and the keys follow Apple's for each screen: 56 pt rows folded, 59.7 pt unfolded and turned, 51.7 pt unfolded upright, starting as far in from the screen's side as Apple's. Folded, the cover screen was measured beside Apple's keyboard on a Duo: its keycaps start 18.5 pt in (54.5 px of 1375 at 2.95 px per point), and row 4 stands on row 3's columns: 123 from the edge to a gap before z, the globe on z (without it, space starts there), space up to return, and return from a gap after m to the edge. A Hex key right of space takes m's column, where Apple's dictation key is; left of space it follows the globe and emoji. Its labels are 0.875 of the iPhone's there, as Apple's are: its 123 is 70 px wide folded against Hex's 80 at 2.95 px per point. The unfolded postures are measured in the simulator.

  • Number row (full keyboard, iPhone): Settings › Keyboard › Number row (off by default, settings.keyboardNumberRow, App Group) puts 1 to 0 in a row of their own above row 1, in every language and posture, upright and turned. Off, nothing changes. On, the keyboard grows upward by the row and every row below stays on Apple's lines (iOS keeps a keyboard on the screen's bottom): Apple's iPhone keyboard has no number row, so it follows SwiftKey's (0.85 of a letter row's pitch, the same row gap; Gboard on iPhone has no row, only digits held on row 1): 34.9 pt keycaps and 45.9 pt more keyboard on a 17 Pro (293 pt, granted in a text view, a Safari form and Spotlight on the iOS 27.2 simulator), 47.6 pt on a Pro Max, 34 pt turned. The digits sit on the 123 page's columns, take the gap under the strip as row 1 did, and hold what the 123 page's digits hold (0 holds °). On the 123 page the row hides, since its row 1 is the same digits, but its band stays so nothing jumps, and that row 1 reaches up through it; the #+= page keeps the row. The digits never reach Spatial autocorrect, next-letter touch help or Swipe typing (a drag from a digit is no swipe), and a word holding digits is never corrected. Where iOS gives less height, the number row shrinks with the others (KeyboardMetrics.fitted). The iPad keeps Apple's rows for now: its docked keyboards would need Apple's own number row on the 13-inch and room above the strip on the others.

  • Comma and period keys (full keyboard, iPhone): Settings › Experiments › Comma and period keys (off by default, experiments.commaPeriodKeys, App Group) puts a comma just left of the space bar and a period just right of it, both a letter wide, on the letters page in a plain text field. On the joined keyboard the comma starts where the bar used to start and the period ends where it used to end, the bar moving inward at both ends; on the Duo's split keyboard the comma takes from the left half's bar and the period from the right half's, each bar keeping its inner edge. Neither is added unless the bar still holds a letter-wide space after them (on an upright iPhone showing both the globe and emoji keys with the Hex key beside space there isn't room, so they're skipped). They type , and . with shift on too (they have no shifted symbols). The 123 and #+= pages, a URL or email field (which keeps its own .), an iPad, and English's Dvorak letters while Dvorak punctuation puts both on row 1 are unchanged. Off, every layout is key for key the one it was. Unit tests only (KeyboardLayoutTests, FullKeyboardEngineTests): no CI compiles the iOS targets, and no physical iPhone run checks it.

  • Word suggestions (full keyboard): The strip above the keys shows the words the engine offers as you type, and after a space the finished word's emoji. Settings › Experiments › Hide word suggestions (off by default, experiments.hideWordSuggestions, App Group) stops the suggestions: the strip keeps its height, the Hex key, Undo, status, notices and the Paste chip, but shows no typing or next-word suggestions, swipe readings, finished-word emoji, or the chip that keeps what you typed while a correction is still ahead (keep literal). Autocorrect still corrects on space, and the chip that puts back what it replaced (.revert) stays after Delete. With no words offered, tapping the left slot to keep what you typed mid-word and holding a word to add it to Custom vocabulary are gone. Off, the strip suggests exactly as it did.

  • Background (full keyboard and dictation bar): Hex draws nothing behind its keys, so the keyboard's background is iOS's own glass panel: the same colour as the band iOS adds above a third-party keyboard and the globe row under it, over any app or wallpaper, with or without that band (no step at Hex's edges on an iPhone 17 Pro, iOS 27.2, in Notes, Safari on a light page and Spotlight over a wallpaper, and in the iOS 27.0 and 27.2 simulators, light and dark). One case gets a cover, because iOS can leave Apple's keyboard under a third-party one. In a field that takes focus as its screen appears and has an input accessory view, opened with Hex, switching to Apple's keyboard never tells Hex it disappeared and keeps its view; switching back puts a new Hex over both, and Apple's keys show through Hex's translucent ones (every switch in the iOS 27.0 and 27.2 simulators, under a bare clear keyboard extension too; one TestFlight report, build 56, iPhone 17, iOS 27.0.1; never in Notes, Safari or Spotlight on an iPhone 17 Pro). Hex can't see the app's views, but it can see its own earlier keyboard still on screen in the same field (KeyboardsOnScreen), and only then draws a thin blurred material under itself (KeyboardBackdropView), through which Apple's keys are an even, slightly lighter tone with no shapes. Opened with Apple's keyboard, or focused by a tap, the same field frees Hex as usual and nothing is left behind. Until build 117 the material was always on: 5 to 9 off the glass in dark mode and neutral grey where the glass takes a wallpaper's tint, so Hex showed as a panel of another colour inside iOS's panel, with a seam under iOS's band.

  • Split keyboard (iPhone Duo): Unfolded and held upright (the inner screen wider than tall), the full keyboard splits as Apple's does there (SplitGeometry): each half at the keyboard's side with Apple's 46.3 pt keys on a 57 pt pitch, row 2 half a key in and row 3's letters a key and a half, an odd row's middle key on both halves (G and V in English), a space bar on each half, and nothing in the middle, where a tap types nothing. The strip sits over the right half, where Apple's predictions are; the recording bar still spans the top. Every layout splits by the same rule (measured against Apple's English split only; AZERTY, QWERTZ, Russian and the rest follow the rule unchecked), and every page (123, #+=). Spatial autocorrect reads taps as on the unsplit keyboard (KeyboardLayout.unsplitX), so either G is one key. Apple's split follows its own Split Keyboard preference, which a keyboard extension can't read, so Hex splits whenever the Duo is in that posture (the iOS 27.2 simulator has Apple's off until KeyboardShouldSplitInDefaultContext is set). Settings › Experiments › Join the keyboard when unfolded (off by default, experiments.joinKeyboardWhenUnfolded, App Group) draws one joined keyboard there instead: unfoldedWide's own 39 pt rows on a 51.7 pt row pitch, 19 pt in from the keyboard's own side as unfoldedTall's keys sit from its screen's, with the iPhone's portrait bottom row (123 and the globe to z's right edge, the emoji key on x's column when the globe shows, space after them, return from m). Off, nothing changes.

  • iPad: The app is iPhone-only (TARGETED_DEVICE_FAMILY 1), but iPadOS runs its keyboard extension as an iPad keyboard across the screen, so the full keyboard shows there too (Settings › Keyboard picks it or the dictation bar, as on an iPhone). Docked, it has Apple's iPad mini rows on every iPad (KeyboardPadRows.swift; the 11- and 13-inch's own, with tab, caps lock and the 13-inch's number row, are a later step): delete ends row 1 and return row 2, a shift key opens and closes row 3, after the letters Apple's two-symbol keys (, and ! then . and ?; AZERTY , and ?, . and !, ' and :), the shifted symbol drawn over the one a tap types and typed with shift (KeyboardKey.shifted), and the bottom row the globe, 123, the Hex key where Apple's dictation key is (whatever Settings › Hex key says: the strip is all suggestions), the space bar, emoji (or 123 when Emoji is off) where Apple's second 123 is, and the hide-keyboard key. The letter keys are as wide as the longest row allows with its delete or return at least 1.236 and 1.24 letters wide (English 54.5 pt, French 53.25 pt on the mini upright), row 2 is set in 0.4 of a key pitch (0.2 where it holds as many letters as row 1, as AZERTY's does), the left shift is a letter wide (1.061 on AZERTY), and delete, return and the right shift take what's left; the bottom row's keys are the same in every language. The 123 and #+= pages follow Apple's: digits and delete, @ # $ & * ( ) ' " (€ £ ¥ _ ^ [ ] { } on #+=) and return, a page key either side of % - + = / ; : and the two two-symbol keys (§ | ~ … \ < > ! ? on #+=), and row 4 as the letters' with ABC; Hex has no undo or redo key there, so ABC takes the room of Apple's undo and second ABC. German is QWERTZ without ü, ö and ä as Apple's iPad German is, holding them on u, o and a. Measured against Apple's iPad mini in English, German and French (every key within a point); the other languages' rows follow the same rule unmeasured. Docked, a held key's accents open in Apple's iPad cells (below). The sizes are Apple's iPad key sizes (KeyboardPadSize, KeyboardPadPlacement, KeyboardMetrics.pad), measured beside Apple's keyboard in the iPadOS 27.2 simulator: docked, Apple's key heights, row pitch, gaps and side inset for the iPad mini (under 800 pt across), 11-inch and 13-inch (1,000 pt and up), upright or turned, so its four rows sit exactly on Apple's bottom four (letters and the bottom row; the 13-inch's number row and the 11- and 13-inch's tab, caps lock and extra keys are a later step), counted from the screen's bottom, where iPadOS leaves 20 pt empty under a third-party keyboard (3 pt under Apple's). Which way up comes from the keyboard's width against the screen, not the size classes, which an iPad keeps regular either way. Above the rows the strip is Apple's 44 pt bar with the words centred, and above it 44 pt stay clear, as the dictation bar keeps them: iPadOS always adds its own 55 pt bar above a docked keyboard (undo, redo, paste; empty in Windowed Apps), and in some apps it draws that bar over the keyboard's top instead (Notes on an iPad mini, iPadOS 26.6, TestFlight). So Hex is taller than Apple's (353 against 282 pt on the mini and 11-inch upright). Docked, the labels keep their upright sizes turned, and keys show no balloons, as Apple's don't. Floating (iPadOS makes every keyboard 320 pt wide, with no bar above and its own globe and dictation band below), Hex is Apple's floating keyboard: 26 × 38.25 pt keys on a 54.3 pt pitch, 35.5 pt shift and delete, 123, space and return on row 3's columns, and the strip where Apple's bar is, 271 pt in all as Apple's; it shows no globe of its own there. iPadOS grants a custom keyboard any height it asks for (up to 900 pt tried); one that asks for none gets 258 pt (343 turned) on the 11-inch. It always lays a third-party keyboard out at that default first and switches to the height asked for about 15 to 20 ms after viewDidAppear, even when the height was set before the view had a window (iPhone: 216 to 247 pt), and a tap while the keyboard still moves lands where the key was a frame earlier. Measured with a scratch build logging every touch and layout (iPadOS 27.2 simulators, three at once): taps 100, 200, 400 and 800 ms after the keyboard appeared, 100 launches each, all went in on an 11-inch docked (400 of 400) and floating (399; one second tap never reached the keyboard), as on an iPhone 17 Pro (399; one first tap at 100 ms typed the key above). Launched into a turn, at the same moment after launch Hex takes taps sooner than Apple's keyboard (1.4 s: 75 of 80 against 2 of 80; from 1.6 s both every one); earlier the keyboard is still sliding in. Docked, a held key's accents show in Apple's iPad cells, half a key wide and 53 pt tall, two rows once there are more than five (AccentMenu.padArrangement, KeyboardMetrics.accentCellWidth); floating, as on an iPhone. In the floating keyboard's emoji panel the category icons shrink to fit its 18.6 pt slots (17 pt in 22 pt and wider).

  • Touch targets (full keyboard): Every point of the key area types a key: gaps split down the middle, the outer keys reach the screen's edges, and row 1 also takes the 8 pt gap under the strip (on the Duo, the strip's bottom 4 pt below its pills, KeyboardLayout.stripTouchBand), so a tap just above a keycap types it rather than picking a suggestion (Apple's keys reach about 8 pt above theirs, measured in the simulator). While you type (a key is down, or one went down or came up in the last 0.25 s) the strip's whole lower half goes to the row-1 key below the finger (FullKeyboardEngine.takesStripTouch): Record typing logs showed every strip pick that soon after a key was an e aimed high landing on the left slot, which kept "th" and typed a space where "the" was meant; nobody reads a new suggestion and taps it that fast. Taps are not shifted: moving every logged tap up by 2 to 8 pt landed no more of them on the meant key, and Apple's row borders sit within about 2 pt of the gaps' middles when nothing is typed. While autocorrect is on in a text field that allows it, the English and Spanish word lists (Spatial autocorrect's models, whether or not it is on) let the likelier next letter claim up to 5 pt of a neighbouring letter's target, 1.25 pt for each e-fold it is likelier (FullKeyboardEngine.letterOdds): after "th" a tap just inside d types e, as Apple's does (there e reaches about 10 pt into d). A finger that rolls within that reach keeps the letter. Other layouts keep fixed targets. The keyboard's window no longer holds back touches near the screen's side edges for system gestures: Record typing logs showed them reaching the keys about 80 ms later than others on an iPhone (whether this removes it is unverified on hardware; the simulator never delayed them).

  • Spaces (full keyboard): A suggestion (a word, what you typed, or an emoji) types its own space, so a space pressed right after it is swallowed, as on Apple's keyboard; it still counts towards double-space for a period. Holding the space bar still for 0.35 s starts the trackpad, as Apple's does, and letting go then types no space (FullKeyboardEngine.trackpadHoldDelay). A tap that slides before then is still a space, however far it goes, as on Apple's (black box, iOS 27.2: taps of 60 to 150 ms sliding 8 to 24 pt, and slides of 40 pt in 150 ms, all typed one); a slide of 10 pt used to start the trackpad at once and type nothing (TestFlight build 56: "missing the space key"). With two or more languages such a slide is a language swipe (Keyboard languages, above). In a rollover space keeps its place: a key going down while the space bar is still held types the space first, as a held letter's does. The full keyboard's trackpad moves as Apple's does (black box, iOS 27.2): the finger drags a point through the text at the text's own scale, in two dimensions, and the cursor goes to the character nearest it. Slowly a point of finger is a point of text, and faster it goes further (1.4 times at 400 pt/s, twice at 1,000, 2.6 times at 2,000, TrackpadCursor.gain); a line takes the font's line height (20 pt), the first half of one; the x carries across short lines; it stops at a line's start and end and at the text's first and last line. A keyboard can only move the cursor by UTF-16 offsets, so TrackpadCursor lays out the text the field shares (the body font's own widths, rows wrapped at the keyboard's width less 40 pt) and moves by the offset to the nearest position, a 👍🏽 in one step. The field shares only the cursor's sentence or line and a little before it, so at that text's edge the cursor waits for the host to show more, and past its first or last line it steps just past it in one move, which the host makes only if there's text there. The keys blank to keycaps 60% of the way to the backdrop, as Apple's, with no haptic while it moves. Apple's second-finger tap to select text has no host counterpart: UITextDocumentProxy (iOS 27.2 SDK) can read the selection but not make one. Off (the default), a second finger does nothing. Settings › Experiments › Two-thumb selection (off by default, experiments.twoThumbSelection, App Group) gives it one: while the space bar is the trackpad, a second finger starts a selection from the cursor, and the finger further left moves the start of the range, the other the end, each with its own TrackpadCursor.State (the trackpad's gain, acceleration and rows). Settling the trackpad at the start can move the host cursor once; after that, no host cursor moves. The strip shows the text around the range with the selected part highlighted, within the text the host shares: the range is clamped to what the field shares around the cursor when the selection starts (the field shares only the cursor's sentence or line). If the host's context hasn't caught up at the start the whole span is kept; either way an action runs only while the host still shares the whole range, and otherwise dismisses without acting. Both thumbs up shows Delete · Cut · Copy · Dictate · Cancel (Cut and Copy only with Full Access, as the strip's clipboard; without it they aren't offered): Delete puts the cursor at the range's end with adjustTextPosition and deletes its characters one by one (a 👍🏽 is one); Cut copies then deletes; Copy copies and puts the cursor at the end; Delete and Cut leave it at the range's start; Cancel changes nothing; and any other key dismisses the actions without acting. Dictate leaves the range in place and starts the keyboard's usual dictation there (the warm Hex session or in-keyboard dictation). When its words are about to insert and the field still shows the range unchanged (the same field, and the text around it), they replace it; otherwise they insert at the cursor as any dictation would, with the range left alone. Cancelling or hearing nothing leaves the range untouched. Where dictation would open Hex instead (no Full Access, or the keyboard can't dictate here), tapping Dictate dismisses the actions and opens Hex (or shows the Full Access hint) with the text left in place. An empty range ends like a normal trackpad lift. Delete and Cut ask the host to delete once per Character: in a script whose host removes one code point per call (some Devanagari and Thai clusters), the last code points of a cluster can stay behind. The dictation bar's space bar is the same trackpad along the line (the same still hold or a 10 pt slide, a character per 7.5 pt, the keys' glyphs blanking, a selection tick per step; TestFlight build 56 asked for it), and a tap still types a space.

  • Smart punctuation (full keyboard): As on Apple's keyboard (black box, iOS 27.2), the
    ' and " keys type opening quotes at the start, after a space, a bracket or an opening
    quote, and closing ones elsewhere (‘Hi’ it’s “yes” 5’9”); a second hyphen makes an em dash
    (a--b is a—b); autocorrect's and the strip's apostrophes are ’ (don’t, I’m). Each language
    types its own quotes (SmartQuotes): „ “ ‚ ‘ in German, « » with no-break spaces in French,
    „ ” ‚ ’ in Polish, « » in Norwegian, Russian and Ukrainian, “ ” ‘ ’ elsewhere. A field's
    smartQuotesType or smartDashesType of .no turns each off (Messages' composer has
    both off: Apple's keyboard types straight quotes and -- there). The keys reason about the
    text with straight quotes (SmartPunctuation.plain). A mark that ends a clause (, . ! ? ;
    )) right after a picked suggestion takes the space the suggestion typed ("Hello,"), as Apple's does with smartInsertDeleteType on or off.
  • Delete after an autocorrection (full keyboard): As on Apple's keyboard (black box, iOS 27.2), delete takes only the space or mark after the corrected words, and the strip's left slot offers what you typed, highlighted with an undo arrow (Apple's text view shows it in a bubble under the word, which a keyboard can't draw); tapping it puts that back, without the space, and turns the fix down for the session. Space keeps the fix. The offer comes back whenever the cursor is at the end of the latest autocorrection again, revisions ("well see" to "we'll see") and joins included. Text replacements offer none, as on Apple's.

  • Key clicks: Every key of the full keyboard (delete, shift, 123 and return included, and each delete repeat), its strip, and the dictation bar's keys ask for the system click on touch-down (UIDevice.playInputClick), which plays only because the keyboard's root view (KeyboardInputView) adopts UIInputViewAudioFeedback: the full keyboard view adopted it before, as a subview, and never clicked. It follows Settings › Sounds & Haptics › Keyboard Feedback › Sound, with no setting of Hex's. Apple's keyboard has other sounds for delete and the modifier keys, but the click is the only keyboard sound a keyboard may play, so every key gets it. In the iOS 27.2 simulator each press reaches the system click (system sound 1104, in the keyboard's log) with Full Access or without, and none with the Sound switch off; on an iPhone, Apple's guide says it needs Allow Full Access.

  • Slides from 123, #+= and shift (full keyboard): As on Apple's keyboard, holding 123 (or ABC, or #+=) shows its page, and sliding shows the pop-up of the key under the finger (or darkens it with Key pop-ups off); letting go types that key and goes back to the page it came from. Letting go back on the page key types nothing and goes back too; off the keys it types nothing and the page stays. Holding shift and sliding to a letter shows and types its capital. None of them is a swipe, and a slid-to key opens no accent menu (Apple's does after a hold).

  • Quick accents (full keyboard, Settings › Experiments): Off by default (experiments.quickAccents, App Group). Holding a letter opens its callout with the letter's main accent selected — the language's preferred one (KeyboardLanguage.quickAccentPrimaries: Spanish o → ó, English e → é, Italian a → à, and Á with shift), else the first variant after the key's own in its holds (AccentCatalog.primaryVariant) — so letting go where the finger landed types it. The cell under the finger when the callout opened is remembered (accentOpenCell), so the preselection stands until the finger moves into another cell, where sliding picks that one as before; letting go far below still cancels, and another key going down while the callout is held commits the preselection. Only letters: the 123 and #+= pages' held symbols, and the mark keys ˇ and ʼ, keep the key's own selected. Off, the callout opens on the key's own letter. Unit tests only (FullKeyboardEngineTests): no CI compiles the iOS targets, and no physical iPhone run checks it.

  • Emoji (Settings › Keyboard › Emoji, for the full keyboard): On by default (stored as experiments.emoji from when it was an experiment off by default: unset or true is on, false off, so testers who turned it off keep it off). An emoji key right of 123 (after the globe) in plain text fields, not web address, email or ASCII-only ones. It opens a panel in place of the strip and the keys: a row of emoji for the text before the cursor, then a grid that scrolls sideways through Frequently used (eight columns: your emoji, then common ones) and the eight categories, at the system emoji keyboard's 46 pt pitch, with ABC, the categories and delete below, drawn without keycaps as the system's are; a tap anywhere under the grid goes to the key under it, each category the full width of its slot. With Settings › Experiments › ABC under the emoji key on (off by default, experiments.emojiLettersUnderEmojiKey, App Group), ABC sits exactly where the emoji key that opened the panel was, its x and width, so switching back to the letters needs no thumb move; a tap left of it still goes to ABC, and the categories take the room between it and delete. Off, or in a panel with no emoji key behind it (the dictation bar's), ABC keeps its leading-end place. As on the system emoji keyboard (iOS 27.2 simulator), a tap on a category jumps the grid there as the finger lifts, and a finger sliding along the bar scrubs the grid live, each icon's slot running through its own category from start to end, with the chip on the category under the finger and a light tick for each new one (EmojiCategoryBar); scrolling the grid moves the chip. The row reads the last eight words (the newest counts most), their plurals and endings, letters held down ("lmaooo"), phrases ("new year") and the word still being typed, matched against Unicode's CLDR names and keywords for the layout's language and English and against Hex's own chat words ("congrats", "bday"), one emoji per family ("woman running", "man running"); your emoji break ties and fill the rest (EmojiSuggester). Search (the magnifier) types with the letter keys into a field in the strip, results above it, by name first. Holding an emoji with skin tones offers five; the one you pick is typed and kept for that emoji. After a space, an empty strip offers the finished word's own emoji ("pizza " → 🍕), never in place of word suggestions. The App Group keeps a use count per emoji that halves every 10 days and each emoji's skin tone, no words (EmojiUsageStore); Clear emoji history empties it. The index is HEXShared/FullKeyboard/Emoji/emoji-index-<lang>.txt, built by scripts/build-emoji-data.py from Unicode's emoji-test.txt 18.0 and CLDR 49 (Unicode License v3, LICENSE-Unicode.txt beside it, which permits commercial use with the notice) and scripts/emoji-aliases-en.txt; the keyboard maps it and binary-searches it in place (about 0.7 MB on the heap), loading it on the first emoji key press or finished word. Emoji newer than 15.1 are listed only where the emoji font draws them. Nothing leaves the iPhone.

  • Screen stays on while dictating in Hex: recording and transcribing set isIdleTimerDisabled, and every way out (done, cancel, failure) hands the screen back to Auto-Lock. iOS honors it only while Hex is in front: the handoff screen and in-app recording, not a warm session or in-keyboard dictation, where another app is in front and a keyboard extension has no public way to keep the screen on.

  • Resilient capture & expiry contract (DictationController):

    • Monotonic jobGeneration, retained transcriptionTask, and unique per-capture WAV filenames ensure an expiring warm session never drops an already-accepted transcription (deliverable for 30s), while a stale winding-down inference can never overwrite a newer capture or delete its audio file.
    • A keyboard's dictation never leaves the keyboard stuck. Hex reads the keyboard's Done and Cancel and heartbeats the job's snapshot until its text is in, in a warm session and in One shot alike (One shot used to ignore them, so a handed-off recording ran to its 60 s limit), and asks iOS for background time to finish the text. The keyboard takes a recording or transcription whose heartbeat is more than 6 s old as over (KeyboardSnapshot.isOrphanedJob): a Hex that ended, crashed or was suspended mid-job used to hold every keyboard in recording for good. A result that missed its 30 s to go in by itself stays offered as Paste dictation for 10 minutes, in the full keyboard's strip and in the dictation bar's top row (on an iPad it once offered nothing). A failure or note from Hex ("Another app is using the microphone. Try again.") shows for 8 s after Hex wrote it (KeyboardSnapshot.messageAt), not for as long as it stays in the App Group; meanwhile both keyboards dictate in the keyboard, and after it a warm session's next start is Hex's Try again.
    • An in-keyboard dictation the keyboard is dismissed from (or another app comes forward) stops its capture at once, keeps what was said, and finishes without the keyboard; its text is held in the App Group (keyboard.heldDictation), goes into the same field if the keyboard comes back there within 30 s, and is offered as Paste dictation anywhere else or later. A capture that goes quiet for 6 s finishes with what it heard, and a transcription stuck for 2 minutes is given up.
    • Phone/FaceTime/Siri audio interruptions finalize captures >= 300 ms so spoken words are not lost, cancel shorter taps cleanly out of Listening, and resume warm keepalive when iOS signals .shouldResume. Hex's own recording (the app, a warm session's keepalive and captures) is playAndRecord with mixWithOthers and A2DP output, never duckOthers, so it doesn't interrupt another app's video or music, and ending a session deactivates with notifyOthersOnDeactivation. It sets no preferred sample rate: playAndRecord runs the microphone and the output at one rate, and the 16 kHz it used to ask for moved everything the iPhone played and recorded (a screen recording's audio too) to 16 kHz while Hex held the session (48 kHz to 16 kHz in the simulator). The recorder writes 16 kHz itself, and the keepalive is input only (an AVAudioEngine input tap, no output node: its output element is off). A keepalive that keeps stopping by itself is restarted at once, then after 1 s, 2 s, … up to 30 s, not every 100 ms. Microphone (MicrophoneChoice, MicrophoneRoute) picks each dictation's input: Automatic (the default) a wired or USB headset's microphone, the car's in CarPlay (the driver's microphone Siri uses, where the iPhone's hears the speakers and the road), else the iPhone's, so AirPods and other Bluetooth headphones keep playing at full quality and nothing waits for call audio; iPhone always the built-in one; Bluetooth also Bluetooth hands-free microphones (AirPods, Bluetooth car kits, which iOS can't tell from headphones), in call audio (8–16 kHz) and still mixing. Only Bluetooth adds allowBluetoothHFP, and the input is always set explicitly. The warm keepalive stays on the iPhone's microphone whatever the choice, so a warm session never holds a car or headset in call audio between dictations; a dictation moves to its input as it starts (a Bluetooth headset's call link can take 0.5–2 s, unmeasured here). The setting replaced Always use iPhone microphone: left on it reads as Automatic, turned off as Bluetooth. In-keyboard dictation asks WebKit for the same input by name (deviceId), read from the keyboard's own session; left to itself WebKit takes the input of the route already playing, the iPhone's while a car or headphones play over A2DP. Its capture session (playAndRecord, videoChat, no mixWithOthers) no app can change: other apps' video and music pause while it records and WebKit ends it without telling them, so once it has ended the keyboard activates and deactivates its own session with notifyOthersOnDeactivation (InterruptedAudio), for players that resume on .shouldResume. Unverified on an iPhone: the simulator has no audio interruptions.
  • Action Button, Control Center, Back Tap & Live Activity:

    • DictateWithHexIntent (AudioRecordingIntent + LiveActivityIntent) and HexDictationControl (ControlWidget) start recording on first press with a Dynamic Island / Lock Screen Live Activity (Stop and Cancel buttons) and transcribe + copy to the clipboard on second press.
    • Only these dictations get a Live Activity; the keyboard and the app show their own recording UI. It shows Copied for 2 s, and a cancel, failure, or empty capture ends it at once, under a background task so a suspending app can't leave it behind (LiveActivityCoordinator). Every end also ends any other Hex activity, and launch, returning to Hex, and ending the warm session (Keep mic warm › Off) end any no dictation owns.

Acknowledgements

The app shows these under Settings › Acknowledgements (HEXMobile/AcknowledgementsView.swift).

  • Google Books Ngram data: Spatial autocorrect's word and word-pair frequencies, German's capitalisation shares and French's list of words outside its model come from the Google Books Ngram Viewer data, version 3 (2020), by Google, licensed under Creative Commons Attribution 3.0. Counts are summed over 1980–2019 and kept only for words in the lists below.
  • SCOWL: English words are checked against SCOWL (Spell Checker Oriented Word Lists) and VarCon by Kevin Atkinson, sizes up to 70 (http://wordlist.aspell.net/). Copyright 2000-2018 by Kevin Atkinson. Permission to use, copy, modify, distribute and sell these word lists, the associated scripts, the output created from the scripts, and its documentation for any purpose is hereby granted without fee, provided that the above copyright notice appears in all copies and that both that copyright notice and this permission notice appear in supporting documentation. Kevin Atkinson makes no representations about the suitability of this array for any purpose. It is provided "as is" without express or implied warranty. SCOWL's inflections come from WordNet 1.6, Copyright 1997 by Princeton University, all rights reserved, provided "as is" without warranties (the full notice is in the app); its other sources are public domain.
  • LDNOOBW: The keyboard's suggestion filter uses the List of Dirty, Naughty, Obscene, and Otherwise Bad Words (LDNOOBW), © 2012–2020 Shutterstock, Inc., and contributors: its English, Spanish, German and Norwegian lists, licensed under Creative Commons Attribution 4.0 International. Hex lowercased and classified the entries, omitted ordinary, medical and search-only terms, and added common slurs, inflections and spelling variants.
  • Parakeet V2: Hex dictates English with NVIDIA's Parakeet TDT 0.6B v2, licensed under Creative Commons Attribution 4.0 International, downloaded as FluidInference's Core ML conversion (CC BY 4.0, encoder weights at reduced precision), used unchanged.
  • Parakeet Ultra: Experiments › Multilingual model downloads Parakeet Ultra by Moondream, a post-training of NVIDIA's Parakeet TDT 0.6B v3, both licensed under Creative Commons Attribution 4.0 International, as FluidInference's Core ML conversion (revision 95eaa59: 8-bit integer encoder weights, decoder and joint re-exported), used unchanged.
  • RLA-ES: Spanish words are checked against RLA-ES, the Spanish spelling dictionaries for LibreOffice by Santiago Bosio and contributors (Spain and Mexico), used under the Mozilla Public License 1.1 (https://github.com/sbosio/rla-es).

Polish's three entries show while the keyboard bundles pl.hextm (AcknowledgementsView.polishEntries):

  • National Corpus of Polish n-grams: Polish word and word-pair frequencies come from the n-grams of NKJP's balanced subcorpus (Institute of Computer Science, Polish Academy of Sciences; http://zil.ipipan.waw.pl/NKJPNGrams), made available under a Creative Commons Attribution licence (the page names no version).
  • ParlaMint: which Polish spellings are missing their marks, and which words are names, is checked against the Polish part of ParlaMint 5.0 (Erjavec et al., CLARIN.SI, http://hdl.handle.net/11356/2004), licensed under CC BY 4.0.
  • PoliMorf: Polish words are checked against PoliMorf 2018-07-23 (http://hdl.handle.net/11321/577), under the 2-clause BSD licence whose notice the app reproduces in full: portions copyright © 2011 Marek Futrega, Piotr Gackiewicz, Mirosław Prywata, Włodzimierz Macewicz, Marcin Miłkowski; © 2007–2018 Zygmunt Saloni, Włodzimierz Gruszczyński, Marcin Woliński, Robert Wołosz, Danuta Skowrońska; © 2011–2018 Institute of Computer Science, Polish Academy of Sciences.

Build & Test

Prerequisites: Xcode 26+ and XcodeGen 2.46.0+.

cd ios
scripts/fetch-cleanup-model.sh   # the cleanup tagger's Core ML model, kept out of git
xcodegen generate
xcodebuild test -project HEXMobile.xcodeproj -scheme HEXMobile -destination "id=<SIM_UDID>"

The emoji index files are generated and committed: after changing scripts/emoji-aliases-en.txt or the sources, run scripts/build-emoji-data.py (it downloads the pinned Unicode and CLDR files, or reads them from --sources).

The cleanup tagger's model (HEXKeyboard/CleanupTagger/HexCleanupTagger.mlpackage, 25 MB) is a GitHub release asset (ios-model-cleanup-tagger-1), not a git file. fetch-cleanup-model.sh downloads it and checks its SHA-256; Xcode Cloud runs it from ci_scripts/ci_post_clone.sh. A new model gets a new release tag and hash.

Replaying real typing

TypingReplayTests runs recordings of real fingers through the keyboard as it runs: the key plane, Swipe typing or taps with Spatial autocorrect, then the engine's spacing. It is opt-in, one test per mode, and the datasets below are not vendored, so convert a recording to one of the two JSON shapes first. xcodebuild passes environment variables to the test runner only when they are prefixed with TEST_RUNNER_, and strips the prefix, so TEST_RUNNER_HEX_SWIPE_REPLAY becomes HEX_SWIPE_REPLAY:

TEST_RUNNER_HEX_SWIPE_REPLAY=/path/swipes.json xcodebuild test \
  -project HEXMobile.xcodeproj -scheme HEXMobile -destination "id=<SIM_UDID>" \
  -only-testing:HEXMobileTests/TypingReplayTests

Swipe recordings are {"sentences":[{"source":String,"session":String,"words":[String], "gestures":[{"x":[Double],"y":[Double],"t":[Double],"start":Double?}]}]}: one gesture per word, x 0…1 across the ten columns of the top letter row, from q's left edge to p's right edge (their centres at 0.05 and 0.95), y 0…1 from the top of the first letter row to the bottom of the third (row centres at 1/6, 1/2 and 5/6), t milliseconds since the finger landed, and the optional start milliseconds from the sentence's first landing to that one, so gestures keep the person's gaps; without it each begins 0.5 s after the previous lift, longer than the two windows the replay's clock drives that a lift leaves open (the double-space period and the strip's touch guard). A start that would put a finger down before the previous one came up is moved to that lift, in both modes, and clamped counts those. A gesture recorded as one point is a tap. Hex's second and third letter rows are offset half a column and one and a half columns, as the format assumes. Tap recordings replace gestures with taps, one {x, y, t} per touch, each array holding that touch's samples (a tap is usually one) on the sentence's clock; y > 1 is the space bar, x 0.25…0.75 across it. TEST_RUNNER_HEX_TAP_REPLAY=/path/taps.json does the same for taps, with Spatial autocorrect reading each word on its space.

Both report a line per source inside TYPING-REPLAY … TYPING-REPLAY-END: total words, words outside the dictionary and their share (a word is in the dictionary when the decoder's model has it or English's contractions in AutocorrectEngine.englishContractions do; the model lookup gets only the word's ASCII a–z letters and apostrophes, so accented, hyphenated and numeric tokens normally count as outside), and accuracy among the in-dictionary words. Scoring lowercases, straightens curly ’, trims leading and trailing apostrophes, and keeps ASCII letters: a tap compares letters, and an apostrophe only when the recorded word has one; a swipe compares letters only. Taps report that as accuracy over every in-dictionary word and also all-words-accuracy over every word, so the gap between the two is what the out-of-dictionary words contribute. Swipes report accuracy and typed-or-offered over the swipe attempts (attempts): gestures that became swipes, ones whose path left the key by a swipe's pitch and weren't promoted, and ones that read nothing. They also report in-dictionary-accuracy over every in-dictionary word — taps included, a skipped gesture counted wrong — and all-words-accuracy over every word, which is the end-to-end number: what the person reading the field saw, with an out-of-dictionary word now able to count as right when the field has it. taps and taps-right are the recordings whose path never leaves its key by a swipe's pitch: the keyboard types that key's letter, and the field keeps it, as for a person typing a one-letter word. Tap mode keeps Swipe typing attached as the keyboard runs it, so a recorded tap whose samples drift can be promoted to a swipe; read-as-swipes counts the recorded taps the plane took that way. not-swipes counts swipe attempts the plane never took as swipes or that read nothing; skipped counts gestures that began off the letter keys, and clamped the touches moved forward to the previous lift; unlike taps and taps-right, which count in-dictionary words, those three count every recorded touch, and read-as-swipes also counts every delivered tap, whatever its word. A tap's words compare an apostrophe only when the recorded word has one; a swipe's by letters only, because a swipe's path runs over letter keys and cannot draw one. The decoder's own median and p90 milliseconds close the line, though a swipe's preview decode can overlap its timed final decode, so a decode-ms sample may include that overlap; the words are unaffected. HEX_REPLAY_LIMIT caps sentences per source; HEX_REPLAY_OUT writes each sentence's intended and typed words (prefix both with TEST_RUNNER_ too).

The test with no recording checks Hex's key geometry against the grid and runs clean gestures through the same mapping to type the right words; it cannot catch a converter that emits wrong coordinates.

Numbers can move between simulator iOS versions because autocorrect also consults the system text checker for word validity and names, and its dictionary changes with iOS.

Public datasets of real typing it is meant for:

See benchmark/typing-replay/README.md for each dataset's download and its convert.py command.

Simulator vs. Physical iPhone Proof Limits

  • Verified in Simulator (HEXMobileTests + --snap-state scenes): Compilation across HEXMobile, HEXKeyboard, and HEXWidgets; App Group command/snapshot framing; the DictationController generation/expiry race and 30s post-close transcript delivery; 10s handoff window; interruption handling; meter recovery after injected startup and mid-capture recorder unavailability (including keyboard levels and heartbeat); deterministic style formatting; UI layouts; and first-run onboarding's launch decision (a fresh install shows it, a saved page resumes, a ready keyboard and microphone skip it).
  • Requires a physical iPhone: Real AVAudioSession background keepalive survival across lock/app switches, live orange indicator behavior, on-device SpeechTranscriber / Neural Engine Parakeet latency, Dynamic Island Live Activity interaction, and hardware Action Button invocation. Transient AVAudioRecorder.isRecording changes during warm handoff or Bluetooth route changes have not been reproduced on hardware; the meter recovery tests inject that condition through ActiveAudioCapture.
  • Screen recordings with the microphone: The simulator can't screen-record, so whether one picks up noise while Hex records shows only on an iPhone: record with the microphone on with Hex closed, Hex open and idle, a warm session on, and during in-keyboard dictation (WebKit's videoChat session, which Hex can't change), and compare.
  • Show what I say and Type as I speak: LiveDictationTests cover how results become the words shown, the switches' defaults, the end of a long dictation, and Type as I speak's typing, revising, taking back, stopping when the field changes or an app reports no text, and private fields. The simulator has no speech models, so there the preview and typing ran on a replay of Apple's results recorded on a Mac (LiveTranscriptReplay), in a host app, light and dark, upright and turned, on an iPhone SE's globe key and with no model at all (no preview). On an iPhone 17 Pro (iOS 27.2): the preview, Type as I speak, a 60 s dictation, memory with and without, and the keyboard hidden mid-dictation with Type as I speak (Home, then back: the held text pasted over its words). Unchecked: VoiceOver on a device (the test runner there has no VoiceOver control), hosts that lag or show part of the text (web views, Notes) and real speech in a room rather than a clip from a speaker.
  • In-keyboard dictation's start time: The simulator's WebKit captures from a mock microphone, and on a Mac without a microphone the warm session's recorder can't start, so how long a real microphone takes to open (getUserMedia, the audio session, the recorder) shows only on an iPhone, in the probe log's first audio and warm start lines.
  • Apple Speech hint: Custom vocabulary's contextual strings are best effort and unverified on a physical iPhone. Apple documents contextual strings only for DictationTranscriber; Hex sets them on both modules, and Parakeet's batch path gets no hint. On macOS 27.2.1 SpeechTranscriber's text didn't change with them (0 of 126 clips of names and terms), and DictationTranscriber wrote the listed spelling in 28 of 132 places against 4 without them, so for English on a recent iPhone SoundAlikeSpelling is what writes a name the listed way.
  • Dictation languages: SpeechLanguagePickerTests cover the picking rule and DictationControllerTests a language change during a dictation, Automatic following the keyboard's language, and Spanish never reaching Parakeet. The simulator can't transcribe: SpeechTranscriber.isAvailable is false there and DictationTranscriber fails with "Model not available", so AppleSpeechClipTests (skipped unless HEX_SPEECH_CLIPS names a folder of <language>.wav clips) reports that. The same code (AppleSpeechTranscriber, AppleSpeechModule, the picker) built for macOS 26 transcribed say clips word for word: German and Spanish through SpeechTranscriber, Norwegian (Nora) and Hebrew (Carmit) through DictationTranscriber, which downloaded each model in 11–16 s. DictationTranscriber returned Norwegian and Hebrew without punctuation under every preset (the short-form presets returned nothing), so Polished only capitalizes and ends the sentence. Ukrainian is DictationTranscriber's only (macOS 27.2 lists uk-UA there and not for SpeechTranscriber): a say -v Lesya clip came back word for word in Cyrillic, without punctuation, after an 18 s model download. Polish is too (macOS 27.2.1 lists pl-PL for DictationTranscriber only): say -v Zosia clips came back with every diacritic (ą ć ę ł ń ó ś ź ż), numbers as digits ("10 min", "19:30"), and no punctuation. Czech is too (macOS 27.2.1 lists cs-CZ for DictationTranscriber only, a 12 s model download): say -v Zuzana clips came back with every háček and čárka ("Příliš žluťoučký kůň úpěl"), numbers and times as digits ("10 min.", "19:30", "8:15") and no punctuation. It never goes to Parakeet, which is English-only. Greek is too (macOS 27.2 lists el-GR for DictationTranscriber only): say -v Melina clips came back with every tonos, final ς, numbers as digits ("10 λεπτά", "7:30"), and no punctuation. Lowercase lowers with Unicode's context-aware mapping, so a word-final Σ becomes ς ("ΟΔΟΣ" → "οδος"), as desktop's to_lowercase does. Spanish variants have unit tests only: a region Apple has keeps it (es-CL on a Chilean phone), es-419 and es-CO resolve to es-US then es-MX, before iOS 26 SFSpeechRecognizer keeps es-CO and es-419 as themselves and falls to es-US, then es-MX, then Spain's for an unlisted region, plain es by the iPhone's Region, a picked variant survives a swipe away and back, the switch offers one variant per language with Chinese's variants kept apart, and the Hex key menu names the region. No iPhone or simulator ran the Spanish variants for this change, and the Settings line and menu labels weren't captured. Unverified on a physical iPhone.
  • Speech model fallback: SpeechModelInstallTests and StallWatchTests run the fallback, set-aside, offline, stall and keyboard-handoff rules against fakes of Apple Speech's calls (the simulator has no speech models), SpeechLanguagePickerTests the set-aside and no-SpeechTranscriber picks, and --snap-state full-keyboard-language-unavailable, keyboard-language-unavailable, language-unavailable and language-switched render the messages. Which module fails for which language on iOS 26.x, whether a stalled Apple download ever moves again, the reservation limit on a full iPhone, and Parakeet's memory on 4 GB iPhones are unverified on hardware.
  • Keyboard languages: KeyboardLayoutTests cover the list (resolution, switching, space-bar label, pairing, migration, the dictation pick, English & Spanish's keys, dictation following the keyboard, Spanish's Americas variants and the iPhone's Region), HangulComposerTests the two-set composer and the keyboard typing through it, FullKeyboardEngineTests the swipe against the hold-then-slide trackpad and switching in a FullKeyboardView (English & Spanish too, and a picked Spanish variant surviving the swipe away and back), and AutocorrectEngineTests bilingual autocorrect on stub dictionaries and iOS's own (testABilingualKeyboardKeepsBothLanguagesWords prints the break counts). Hungarian's QWERTZ rows and held vowels, typing through them in a FullKeyboardView, and its accent fixes are unit tests too (testHungariansRowsAndHoldMatchItsLayout, testHungarianTypesAndSwitches, testHungarianRestoresMissingDiacritics); no simulator or iPhone typed on its keys for this change. Checked live in Messages in the simulator (English, Français, Русский). Memory (Release, iPhone 17 Pro simulator, iOS 27.2, Full Access, three runs, footprint after typing a sentence): Français alone 28.7–28.8 MB, with English beside it 29.7–29.9 MB; English alone 28.9–29.9 MB, with Français 29.5–29.9 MB; resting 18.7–19.1 MB and peaks 37–41 MB either way, so the second language costs about 1 MB (the simulator counts no GPU memory). A real iPhone, the strip's latency with two Spatial models (English and Spanish), and whether a keyboard without Full Access keeps the current language are unverified.
  • Keyboard layouts: KeyboardLayoutTests check every language's rows, widths, hit frames (tiling the key area and row 1's band above it with no gaps or overlaps), balloons and accent menus at 320–440 pt, in landscape and in the iPhone Duo's postures, and the iPad's (testPad…: each size docked either way up against Apple's rows, floating, the hide-keyboard key, targets); AutocorrectEngineTests and FullKeyboardEngineTests check typing, dictionaries and English-only fixes. Whether the simulator or an iPhone has a UITextChecker dictionary for nb decides Norwegian autocorrect; how good those corrections are is unmeasured.
  • Learn how I type: Dictation edit inference (span location, word diff, outcomes, the learning threshold, and applying fixes) is covered by unit tests only. How much documentContextBeforeInput and documentContextAfterInput each host app shows, and whether it calls textDidChange after the keyboard's own edits, varies by app and is unverified on a physical iPhone; when the context is cut short or missing, the dictation is settled on where it was last seen or not learned from.
  • Adaptive touch: FullKeyboardEngineTests and AutocorrectEngineTests cover what teaches it (words that stood, picks, undone fixes, retypes; never autocorrections, swipes, claimed letters, strip touches or sensitive fields), contexts, the evidence floor, the grip fallback, the file (size, backup exclusion, damage, reset) and the guard (no new rewrite, no new join, names and your words kept). The autocorrect benchmark's held-out sentences (Tatoeba text, synthetic taps; iPhone 17 simulator, iOS 27.2) with the Experiment off, or on without evidence, match the keyboard without it item for item (2,000 of 2,000 outputs and strip hits, English and Spanish). With a profile learned from the 300 dev sentences, English WER goes from 5.24% to 5.16% (16 words fixed, 5 broken, none typed right) and Spanish from 8.34% to 8.26% (12 fixed, 3 broken); the strip's hit after two letters rises 0.8 and 0.3 points. With the personal centres deciding alone it would be 5.04% and 8.00%: the guard holds back most of the gain. Learn how I type's journal alone does better (5.00% and 8.02% in the same replay, offline): with both on, 5.04% and 8.04%. In an offline replay of a grip change (learned on taps 6 pt right, typed 6 pt left), the profile without the fallback broke 38 English words stock got right; the fallback stops at the stock centres and breaks none, and then learns the new grip. Per-letter engine p95 changed by -0.09 to +0.04 ms against the Experiment off (simulator noise), peak footprint by at most +0.25 MB. Real iPhones, real grips and hands, host apps' text reports, Full Access off on a device (whether the switch is readable, the extension's own container), and how often people retype or undo are unverified.
  • Record typing logs: TypingLogTests cover the line format, day files, the cap, retention, the skip rule, and a real FullKeyboardView typing through the logger, whose edits replay to the field's text. Touches there are synthetic samples fed to the same path UITouches take; real touch radii, sampling rates, how often hosts call textDidChange, and the extension's write cost on an iPhone are unmeasured.
  • Clean up what I say: CleanupTaggerTests check, in the simulator (CPU only), the tokenizer against Hugging Face on 1,891 words, reconstruction against reconstruct.js on 670 cases, and the model on the 137 hand-written sentences at 0.98. Its labels match the fp32 checkpoint's on all 1,128 words, and 136 of 137 outputs match the ONNX int8 pipeline. The other sentence differs on two words within 0.01 of the threshold, which ONNX's int8 activations keep and the checkpoint deletes. In the simulator the model loads in about 150 ms, cleans a sentence in about 3 ms (p95 under 8 ms), and adds about 3 MB to the process footprint, because the CPU leaves the int8 weights memory-mapped from the bundle. The simulator's GPU path took 0.8 s to load and added 30 MB, so Hex doesn't use the GPU. Neural Engine load time, latency, and the keyboard extension's footprint next to Apple Speech are unmeasured on an iPhone; keyboard-capture-probe.log records the footprint around each in-keyboard dictation.
  • In-keyboard dictation memory: Measured only in the simulator (Release, iPhone 13, iOS 27.2, which enforces no limit and leaves out GPU memory; three runs each, the same before and after the full keyboard became the default): the full keyboard rests at 18.4–18.6 MB with Full Access and 18.0–18.3 MB without, and its first dictation reaches 30.8–31.8 MB within 3 s of the tap (peak 32.1–33.0 MB, so about 13 MB); the dictation bar rests at 20.7–20.8 MB and its first dictation reaches 27.3–27.9 MB (about 7 MB). Later dictations add 1–5 MB. The 32 and 24 MB thresholds rest on that plus the Metal scope's GPU memory, not on an iPhone's; keyboard-capture-probe.log records each step's footprint, peak and headroom, and keeps only its newest 256 KB once it passes 512 KB.
  • Without Full Access: Verified in the simulator with the keyboard in a real hosted field (--snap-state hosted-keyboard) and Full Access off: a fresh install mounts the full keyboard and offers no in-keyboard dictation; keys typed through the engine autocorrect ("thw" became "the", "tk" "to", and with Spatial autocorrect, on by default, "ofthw" "of the"); the Hex key shows the hint. Scenes: full-keyboard-no-access[-hint][-dark]; --keyboard full|dictation-bar picks the keyboard. The simulator's keyboard reads the App Group without Full Access; whether an iPhone's does is unverified, which is why the keyboard mounts the full keyboard without Full Access whatever was chosen. Full Access in a simulator: a kTCCServiceKeyboardNetwork row for the containing app (anomaly.hex.mobile, auth 2) in the shut-down simulator's data/Library/TCC/TCC.db. Unverified on a physical iPhone.
  • First run: Reading Full Access from the keyboard on page 3, the keyboard-switch signal behind "Full Access looks off" (UITextInputMode.currentInputModeDidChangeNotification), and Settings opening on Hex's page are unverified on a physical iPhone. Before iOS 26 without a warm session, dictating on page 3 takes the cold-start path, whose handoff screen replaces the page.
  • Emoji: EmojiSuggesterTests cover the index, ranking, search, the strip and the history, and score 100 sentences (HEXMobileTests/Fixtures/emoji-context-sentences.json) for a sensible emoji in the top five: 30 hand-written to tune (30/30, top-1 27), 30 hand-written and held out (30/30, top-1 27; written by the same hand as the chat words, so optimistic), and 40 written by a separate agent that saw no code or word lists (37/40, top-1 28 before any change; 38/40, top-1 29, now). CLDR's words alone scored 26, 22 and 30. Apple's on-device word embedding (EmojiSemanticIndex, app and tests only) changed none of these, cost 5.3 MB that stayed after release and 100–120 ms to build, so the keyboard doesn't use it; Apple's sentence embedding scored 18/40. In the keyboard extension (Release, iPhone 17 Pro simulator, iOS 27.2, Full Access, a hosted field, no GPU memory, no headroom reading, footprint resident and peak): drawn as text, each emoji's image (about 48 KB) stayed in a cache of the shared emoji font that nothing public empties, so browsing every category reached 62.7 MB (peak 67.5) and scrolling past every emoji 137.7 MB (peak 142.3), still held once closed. The panel now draws its emoji itself (EmojiGlyphRenderer): a private copy of the emoji font, from the same memory-mapped file, replaced every 32 emoji and once drawing pauses for 0.3 s, and a 2 MB least-recently-used cache of the bitmaps, freed with the panel and on a memory warning. Two runs: typing 30.6–34.6 MB, the panel open 36.4–40.0 MB, every category 39.4–43.0 MB, every emoji 40.0–42.8 MB (peak 44.3–47.9), 31.9–35.3 MB once closed, and an in-keyboard dictation after it no higher. Measured again with the bar's scrubbing (one run, same setup): tapping every category peaked at 44.5 MB and scrolling past every emoji at 44.4; one slide across five categories peaked at 46.0, three slides back and forth across the bar at 47.5 and ten at 48.3, each back to 42–43 MB resident after. EmojiSuggesterTests check that a cell lands where a label drew its emoji and that drawing every emoji stays in the cache's limit; --snap-state full-keyboard-emoji-probe prints the app's footprint, the renderer's counts and each frame's work scrolling every emoji at 40 pt a frame (p50 1.6–1.8 ms, p95 2.5–6 ms against 0.6–1.0 ms drawn as text). The panel opens in 24–29 ms the first time and 15–17 ms after (8–9 ms drawn as text, which reused the shared font's images).
  • No iOS CI: Custom vocabulary is covered by unit tests only. No CI job compiles or tests the iOS targets, so these checks run only in a local xcodebuild test.

Internal TestFlight releases

Hex Mobile in App Store Connect uses Anomaly's signing team. The GitHub Release Hex Mobile workflow checks dev at noon and 6 p.m. Chicago time, or on manual dispatch, and cuts an ios-beta-* prerelease when iOS inputs have changed. These tags are separate from desktop releases. The marketing version stays fixed across beta builds. MARKETING_VERSION stays 9.0.0 for TestFlight, so every dev build reaches testers without a new beta review; it's never submitted to the App Store. App Store releases are 1.0.x and use their own version. Apple rejects any upload below the highest approved App Store version, so TestFlight's version must stay above every App Store release. To build one, run scripts/marketing-release.sh <version> [commit], which pushes the tag ios-beta-appstore-<version> (e.g. ios-beta-appstore-1.0.0) on that dev commit: it matches the Xcode Cloud start condition, and ci_scripts/ci_post_clone.sh stamps that version into project.yml before generating the project. Once Apple approves a version, later uploads must be higher than it. Hex Mobile is iPhone only (TARGETED_DEVICE_FAMILY 1). An iPad still installs it (as an iPhone app) and runs its keyboard as an iPad keyboard: the full keyboard or the dictation bar, as Settings › Keyboard chooses (see iPad under the full keyboard; until build 117 an iPad always showed the bar). Docked, the bar keeps its top 44 pt clear: iPadOS adds its shortcut bar (undo, redo, paste) to a custom keyboard, usually above it but in some apps over its top (Notes, iPadOS 26.6), where it covered the bar's top row. A floating keyboard (iPhone-wide) has no shortcut bar and no strip. The bar's top row is 40 pt tall around its 26 pt pills, and every control in it takes taps across the row's full height, at least 44 pt wide, with the Hex mark and each tray's close reaching the screen's edges.

The Xcode Cloud workflow needs a tag start condition with prefix ios-beta-, container ios/HEXMobile.xcodeproj, shared scheme HEXMobile, an iOS Archive action, and a TestFlight post-action for Hex Internal. Cloud owns the build number. Verify the run's source tag and the processed build in that group.