mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-22 21:23:02 +01:00
* iOS keyboard: a word of your own in capitals is offered, never put in for a lowercase one
What changed: Spatial autocorrect's space and Swipe typing no longer put in one of the
person's own words written with capitals after the first letter (all caps such as "DONT",
or capitals inside), from UILexicon (Contacts, text replacements) or words Hex learned, for
a word typed or swiped without such capitals. The strip still offers it. Space puts in the
next reading that clears the margin, or keeps what was typed; a swipe types the model's
lowercase word of the same letters when there is one ("in" for an own "IN", which takes the
model word's place in the swipe decoder), else its first ordinary reading. Caps lock, words
typed in capitals, names with only a first capital, and the vocabulary list's own spellings
("GitHub") behave as before.
Why: two days of Record typing logs from one tester (build 106/115). An own word in capitals
replaced "don't" three times ("dknt" -> "DONT"; reverted, re-picked or deleted) and three
swipes typed capitalised words from the same list instead of common words. Own words cost
11 nats in both decoders, and the English model prices contractions far lower than that
("don't" 14.4 nats), so the own word won. That model pricing is a separate data change.
Where: AutocorrectEngine+Distance (isOwnCapitals, decisionOfferingOwnCapitals,
readingsOfferingOwnCapitals); AutocorrectEngine.decoderDecision calls it (three lines
changed, none added: the file is at its cap); SwipeTyping.finish orders its readings.
Verified (iPhone 17 Pro simulator, iOS 27.2):
- New cases in AutocorrectEngineTests and FullKeyboardEngineTests fail on the base commit
("DONT ZORBIX", "HELO THE") and pass here; AutocorrectEngineTests, FullKeyboardEngineTests
and SwipeDecoderTests pass (240 tests).
- The tester's touches replayed through the keyboard (792 words and swipes): with no own
words, identical output for every word. With the five capitalised words Hex put in on his
phone seeded as his own list: 9 outputs differ; 3 now match what he left ("don't" twice,
"in"), 2 he had kept in capitals now come out without them (the bar offers them), 2 stay
wrong (a sloppy swipe, a literal he protected), scorable word errors 38 -> 37 of 680.
- English, Spanish and German held-out gates (iOS replay of #2374's 2,000 sentences each):
every output identical; en 5.91% WER (82.5% fewer than raw taps, 0.29% correct words
changed), es 8.91% (77.8%, 0.20%, accents 94.1%), de 6.52% (83.3%, 0.12%), critical cases
pass. English with 133 all-caps acronyms seeded as own words: 6.21% before and after
(4 sentences differ, 2 better, 1 worse, 1 wrong either way), 0.31% changed both.
* iOS keyboard: "i'm", "i'll", "i've" and "i'd" typed with the apostrophe take the capital
What changed: with Autocorrect and Auto-capitalization on (the defaults), space after "i'm",
"i'll", "i've" or "i'd" typed with their apostrophe now puts in "I'm", "I'll", "I've", "I'd",
with a straight apostrophe or smart quotes' ’ alike, as "i", "im", "ill", "ive" and "id"
already took theirs. Auto-capitalization off still adds no capital ("i'm" stays), as for "i".
Why: a post on X said Hex "refuses to capitalize I's in sentences". With defaults on
origin/dev, real key taps through the key plane capitalised a lone "i" mid-sentence, at a
sentence start, before a comma, in fields with and without auto-capitalization and beside a
second language, and "im"/"ill"/"ive"/"id" too; but every apostrophe form stayed lowercase.
Nothing handled a token with an apostrophe in English, and with smart quotes the ’ the key
types ends the token, so the word was never seen whole.
Where: AutocorrectEngine+Dictionaries.pronounCapital reads them from the text before the
cursor; replacementOnSpace asks it after text replacements (one line changed, none added).
FullKeyboardEngine.finishWord now deletes, logs and records what the fix replaced
(fix.original), which equals the token in every case before this one.
Verified: a new AutocorrectEngineTests case fails on the base commit ("so i'm in and i'd go
i’ll say i’ve") and passes here; AutocorrectEngineTests, FullKeyboardEngineTests and
SwipeDecoderTests pass; the English, Spanish and German held-out gates give the same output
for every sentence (their scoring is case-blind, and no gate sentence changed).
227 lines
12 KiB
Swift
227 lines
12 KiB
Swift
import Foundation
|
||
|
||
/// Case matching (with the strip's and a fix's capitals, and the word before a token) and the
|
||
/// keyboard-distance cost of a one-edit typo: what cautious autocorrect, bilingual choices,
|
||
/// and the strip's spelling guesses measure a slip by. Kept beside the engine so its own file
|
||
/// stays under the cap.
|
||
@MainActor
|
||
extension AutocorrectEngine {
|
||
// MARK: - Case matching & QWERTY distance
|
||
|
||
static func matchCase(of typed: String, onto target: String) -> String {
|
||
guard !typed.isEmpty, !target.isEmpty else { return target }
|
||
if typed.count >= 2 && typed == typed.uppercased() && typed != typed.lowercased() {
|
||
return target.uppercased()
|
||
}
|
||
// Preserve intentional casing in `I`, `I'm`, `iPhone`, etc. unless `typed` started capitalized.
|
||
if target != target.lowercased() {
|
||
if typed.first?.isUppercase == true, let first = target.first, first.isLowercase {
|
||
return first.uppercased() + target.dropFirst()
|
||
}
|
||
return target
|
||
}
|
||
if typed.first?.isUppercase == true, let first = target.first {
|
||
return first.uppercased() + target.dropFirst()
|
||
}
|
||
return target
|
||
}
|
||
|
||
/// Whether `word`, one of your own words with capitals after its first letter ("DONT",
|
||
/// "McQuorv"), is read for `typed`, which has none: the strip offers it, but neither space
|
||
/// nor a swipe puts it in. Record typing logs showed a word from someone's own list in
|
||
/// capitals put in for "don't" and for swiped common words. Words typed in capitals, your
|
||
/// own lowercase words and names, and your vocabulary's spellings ("GitHub") are as before.
|
||
func isOwnCapitals(_ word: String, typed: String) -> Bool {
|
||
word.dropFirst().contains(where: \.isUppercase) && !typed.dropFirst().contains(where: \.isUppercase)
|
||
&& !isVocabularySpelling(word)
|
||
}
|
||
|
||
/// Spatial autocorrect's decision for `token`, putting in no `isOwnCapitals` reading:
|
||
/// space takes the next reading that clears the margin, if any, and the strip keeps them all.
|
||
func decisionOfferingOwnCapitals(
|
||
_ decoder: TypingDecoder, taps: [CGPoint], typed token: String, previous: String?, valid: Bool, keys: TypingDecoder.Keys
|
||
) -> TypingDecoder.Decision? {
|
||
let lower = token.lowercased()
|
||
var decision = decoder.decide(taps: taps, typed: lower, previous: previous, literalIsValid: valid, keys: keys)
|
||
guard let replacement = decision?.replacement, isOwnCapitals(replacement, typed: token),
|
||
let readings = decision?.readings else { return decision }
|
||
decision?.replacement = readings.lazy.map(\.text).first {
|
||
!self.isOwnCapitals($0, typed: token)
|
||
&& decoder.replaces(with: $0, taps: taps, typed: lower, previous: previous, literalIsValid: valid, keys: keys) == true
|
||
}
|
||
return decision
|
||
}
|
||
|
||
/// A swipe's readings, best first, putting in no `isOwnCapitals` one unless caps lock is
|
||
/// on: the model's word of the same letters in lowercase ("in" for "IN", which takes its
|
||
/// place in the decoder), else the first ordinary reading, goes first; the strip offers yours.
|
||
func readingsOfferingOwnCapitals(_ readings: [String], capsLock: Bool, model: TypingDecoderModel) -> [String] {
|
||
guard !capsLock, let first = readings.first, isOwnCapitals(first, typed: "") else { return readings }
|
||
if model.index(of: first.lowercased()) != nil { return [first.lowercased()] + readings }
|
||
guard let index = readings.firstIndex(where: { !isOwnCapitals($0, typed: "") }) else { return readings }
|
||
var ordered = readings
|
||
ordered.insert(ordered.remove(at: index), at: 0)
|
||
return ordered
|
||
}
|
||
|
||
/// `word` without a capital first letter it only has from where it was learned: a fix
|
||
/// you picked at the start of a sentence ("Helli" to "Hello"), a guess iOS knows
|
||
/// capitalised, a Contacts name that is also a common word ("Bell", "Will"). When `typed`
|
||
/// starts in lowercase and the dictionary knows the word in lowercase, it is written in
|
||
/// lowercase, as Apple's keyboard writes "bell" mid-sentence beside a contact Kate Bell
|
||
/// (black box, iOS 27.2). Names the dictionary knows only capitalised ("Taylor"), "I" and
|
||
/// its contractions, words with more capitals ("iPhone"), a word in your vocabulary spelled
|
||
/// as you added it ("Swift") and German, whose nouns take capitals, keep theirs.
|
||
func withoutStrayCapital(_ word: String, typed: String) -> String {
|
||
guard language != .german, hasDictionary, typed.first?.isUppercase != true, word.count >= 2,
|
||
let first = word.first, first.isUppercase, !word.dropFirst().contains(where: \.isUppercase),
|
||
!word.contains(" "), !isVocabularySpelling(word) else { return word }
|
||
let lower = first.lowercased() + word.dropFirst()
|
||
if language.usesEnglishFixes, lower.hasPrefix("i'") || lower.hasPrefix("i\u{2019}") { return word }
|
||
return checker.isWordValid(lower) ? lower : word
|
||
}
|
||
|
||
/// The word before `token` in `before`, lowercased, when only spaces separate them
|
||
/// (nil at the start of a sentence or after punctuation).
|
||
static func previousWord(_ before: String, token: String) -> String? {
|
||
var rest = before.hasSuffix(token) ? before.dropLast(token.count) : Substring(before)
|
||
guard rest.last == " " else { return nil }
|
||
while rest.last == " " { rest = rest.dropLast() }
|
||
var word = ""
|
||
while let c = rest.last, c.isLetter || c == "'" {
|
||
word.insert(c, at: word.startIndex)
|
||
rest = rest.dropLast()
|
||
}
|
||
return word.isEmpty ? nil : word.lowercased()
|
||
}
|
||
|
||
/// The strip's words for `token` with Auto-capitalization off: no capital you didn't type,
|
||
/// as space puts none in ("Mandy" and "I'd" come "mandy" and "i'd"), except your own
|
||
/// spellings; a word that is then what you typed, or twice there, goes.
|
||
func withoutAddedCapitals(_ pool: [String], typed token: String) -> [String] {
|
||
guard !addsCapitals, !token.contains(where: \.isUppercase) else { return pool }
|
||
return pool.map { isOwnSpelling($0) ? $0 : $0.lowercased() }.reduce(into: []) { kept, item in
|
||
if item != token, !kept.contains(item) { kept.append(item) }
|
||
}
|
||
}
|
||
|
||
/// Returns a weighted cost if `source` and `target` are within 1 Damerau-Levenshtein edit; `nil` otherwise.
|
||
static func singleEditKeyboardCost(
|
||
from source: String,
|
||
to target: String,
|
||
language: KeyboardLanguage = .english,
|
||
letters: KeyboardLetterLayout? = nil
|
||
) -> Double? {
|
||
if source == target { return 0 }
|
||
let s = Array(source)
|
||
let t = Array(target)
|
||
// Polish: diacritics you didn't add, however many ("zolw" -> "żółw", "maly" -> "mały"),
|
||
// are as cheap as one missing Spanish accent, a little more for each further one. Greek
|
||
// likewise: a tonos or diaeresis left out ("καλημερα" -> "καλημέρα", "αυπνια" ->
|
||
// "αϋπνία"), or σ where the word ends in ς ("λογοσ" -> "λόγος"). Czech too: háčky and
|
||
// čárky left out ("cesky" -> "český", "prilis" -> "příliš"), and Hungarian: its accented
|
||
// vowels left out ("koszonom" -> "köszönöm", "kesobb" -> "később", "szivesen" ->
|
||
// "szívesen"), ő and ű included, as o and u with the combining double acute.
|
||
if language == .polish || language == .greek || language == .czech || language == .hungarian,
|
||
s.count == t.count {
|
||
func same(_ a: Character, _ b: Character) -> Bool {
|
||
switch language {
|
||
case .greek: Self.sameGreekLetterIgnoringAccents(a, b)
|
||
case .hungarian: Self.sameLetterIgnoringAccents(a, b)
|
||
default: Self.samePolishLetterIgnoringDiacritics(a, b)
|
||
}
|
||
}
|
||
let missed = zip(s, t).filter { $0 != $1 }
|
||
if !missed.isEmpty, missed.allSatisfy({ same($0, $1) }) {
|
||
return 0.15 + 0.05 * Double(missed.count - 1)
|
||
}
|
||
}
|
||
let ds = s.count - t.count
|
||
guard abs(ds) <= 1 else { return nil }
|
||
|
||
if ds == 0 {
|
||
var diffs: [Int] = []
|
||
for i in 0..<s.count where s[i] != t[i] {
|
||
diffs.append(i)
|
||
if diffs.count > 2 { return nil }
|
||
}
|
||
if diffs.count == 1 {
|
||
let i = diffs[0]
|
||
// An accent or tilde you didn't add ("aqui" -> "aquí", "manana" -> "mañana")
|
||
// is the cheapest fix there is, not a far-away key.
|
||
if sameLetterIgnoringAccents(s[i], t[i]) { return 0.15 }
|
||
return areAdjacentKeys(s[i], t[i], language: language, letters: letters) ? 0.65 : 1.15
|
||
}
|
||
if diffs.count == 2, diffs[1] == diffs[0] + 1,
|
||
s[diffs[0]] == t[diffs[1]], s[diffs[1]] == t[diffs[0]] {
|
||
return 0.60
|
||
}
|
||
return nil
|
||
}
|
||
|
||
// Single insertion or deletion.
|
||
let (shorter, longer) = s.count < t.count ? (s, t) : (t, s)
|
||
var i = 0
|
||
var j = 0
|
||
var skippedIndex: Int?
|
||
while i < shorter.count && j < longer.count {
|
||
if shorter[i] == longer[j] {
|
||
i += 1
|
||
j += 1
|
||
} else if skippedIndex == nil {
|
||
skippedIndex = j
|
||
j += 1
|
||
} else {
|
||
return nil
|
||
}
|
||
}
|
||
let extraIdx = skippedIndex ?? (longer.count - 1)
|
||
let extraChar = longer[extraIdx]
|
||
let leftAdj = extraIdx > 0 && areAdjacentKeys(extraChar, longer[extraIdx - 1], language: language, letters: letters)
|
||
let rightAdj = extraIdx + 1 < longer.count && areAdjacentKeys(extraChar, longer[extraIdx + 1], language: language, letters: letters)
|
||
return (leftAdj || rightAdj) ? 0.70 : 0.90
|
||
}
|
||
|
||
/// Whether two letters differ only by an accent, diaeresis or tilde (e/é, u/ü, n/ñ).
|
||
static func sameLetterIgnoringAccents(_ a: Character, _ b: Character) -> Bool {
|
||
guard a != b else { return false }
|
||
let base: (Character) -> Unicode.Scalar? = { String($0).decomposedStringWithCanonicalMapping.unicodeScalars.first }
|
||
return base(a) == base(b)
|
||
}
|
||
|
||
/// Whether two lowercase letters are one Polish letter with or without its diacritic (a/ą,
|
||
/// z/ż, l/ł: ł has no decomposition, so it is named).
|
||
static func samePolishLetterIgnoringDiacritics(_ a: Character, _ b: Character) -> Bool {
|
||
sameLetterIgnoringAccents(a, b) || Set([a, b]) == ["l", "ł"]
|
||
}
|
||
|
||
/// Whether two lowercase letters are one Greek letter with or without its tonos or
|
||
/// diaeresis (α/ά, ι/ϊ/ΐ), or σ and its word-final form ς.
|
||
static func sameGreekLetterIgnoringAccents(_ a: Character, _ b: Character) -> Bool {
|
||
sameLetterIgnoringAccents(a, b) || Set([a, b]) == ["σ", "ς"]
|
||
}
|
||
|
||
private struct LayoutKey: Hashable {
|
||
let language: KeyboardLanguage
|
||
let letters: KeyboardLetterLayout
|
||
}
|
||
|
||
private static let keyCoordinates: [LayoutKey: [Character: (Double, Double)]] = Dictionary(
|
||
uniqueKeysWithValues: KeyboardLanguage.allCases.flatMap { language in
|
||
language.letterLayouts.map { (LayoutKey(language: language, letters: $0), language.keyCoordinates($0)) }
|
||
}
|
||
)
|
||
|
||
static func areAdjacentKeys(
|
||
_ a: Character,
|
||
_ b: Character,
|
||
language: KeyboardLanguage = .english,
|
||
letters: KeyboardLetterLayout? = nil
|
||
) -> Bool {
|
||
let key = LayoutKey(language: language, letters: language.offered(letters))
|
||
guard let keys = keyCoordinates[key], let pa = keys[a], let pb = keys[b] else { return false }
|
||
let dx = pa.1 - pb.1
|
||
let dy = pa.0 - pb.0
|
||
return (dx * dx + dy * dy).squareRoot() <= 1.6
|
||
}
|
||
}
|