hex/android/app/src/main/assets/typing/SHA256SUMS
Ryan Vogel 3f35348581
English typing model: contractions as common as their words, and Hex's chat and tech words (#2408)
What changed: en.hextm (iOS and Android, one file) is rebuilt by build-typing-model.py from
the same Google Books shards and SCOWL, with two additions:
- Each contraction in the builder's CONTRACTIONS counts as often as its expansion's word pair
  ("do not") when that is commoner, and "i'm", "i'll", "i've", "i'd", "here's", "where's",
  "mustn't" and "needn't" are added in lowercase.
- en-extra-words.tsv (new, beside both copies of the builder): 166 chat and tech words written
  for Hex ("app", "lol", "csv"), each at its tier's cost (10, 12 or 13.5 nats); a word the
  books already have only gets commoner.
Both join the vocabulary before pairs are counted, so their pairs are counted too. en.valid and
en.capitals (Android) are rebuilt from the new model, and build-capitals.py holds Hex's own
words at 94% like SCOWL's lowercase words ("plz" and "okie" are capitalised in books, never by
the keys). SHA256SUMS and the benchmark and iOS READMEs follow.

Why: Google Books splits "don't" into "do" + "n't", so the model priced "don't" at 14.4 nats
and "it's" at 16.2, and lacked "i'm"; a rarer reading beat them. Recorded typing logs had
everyday words the model lacked ("app", "csv", "gpus"), and the next-letter odds moved
correct taps off them.

Reproduced: with the earlier rules the same counts rebuild the earlier en.hextm byte for byte
(14370569...4028), and the valid and capitals builders rebuild the earlier en.valid and
en.capitals from it. New en.hextm c23e9c54...df30: 86,247 words, 417,182 pairs, 3,599,995
bytes (inside the 3.6 MB budget).

Verified:
- iOS gate (scripts/typing-gate.py, SWIFT_DETERMINISTIC_HASHING=1): English 773 -> 734 word
  errors (5.57% -> 5.29%, 83.5% -> 84.4% fewer than raw taps), correct words changed 0.27% ->
  0.26%, critical 2/2, PASS. Spanish and German outputs identical.
- Android gate (quality-gate.py): English 718 -> 680 (5.18% -> 4.90%, 84.7% -> 85.5%), correct
  words changed 0.32% both, critical 2/2, PASS; es, de, fr, it, nb identical.
- Per letter on the iOS engine (held-out inputs, p50/p95): 0.189/2.18 -> 0.181/2.08 ms.
- Swipes, every word of the English held-out sentences through the key plane and SwipeTyping:
  92.38% -> 92.46% clean, 89.83% -> 89.89% with finger noise; one word lost in each ("weed"
  now "we'd"). Existing swipe tests unchanged and passing.
- A tester's recorded taps, single words: 37 -> 34 errors, correctly tapped words changed 12 -> 8.
- xcodebuild test (Studio, own simulator): 882 tests, 0 failures, 4 opt-in skips. Android
  testDebugUnitTest (66, incl. NeverSuggestTest on every shipped model): pass. code-shape.py: pass.
2026-11-22 13:40:28 -05:00

17 lines
1.3 KiB
Text

c23e9c546d620fa6d8d395b9dae72cb7825b39050fc1782b548b829729a6df30 en.hextm
02ffcac185eb832d766b300a7454ba069d0b26251469ea7cdc6eda36926ad5ad es.hextm
7854e9a3c6788f9ff568a476d84c9f9a0f7b6b2c794df4375424252860998355 de.hextm
badc2e919a0238926d2935c750b9a21047e264edeb8987a85967133ed1350067 nb.hextm
538383ecde556bfa17717694f01d83c3e6efca038cda64a792de6461e94c3e2c en.valid
aeb1068b671ede1a786b704b622abd10647a59b2b2d55831e004a361015c15b5 en.capitals
90f2e774d0bcf02b1e0667a4e4d207adf7e0d5291aee45203fbe8b7dd8282e98 es.capitals
7b1b24ff18868fb32e049502806ef3737ad91bde685239a4cc28ae38185f3ef9 de.capitals
11315e5087befcb598274a6fb5a05a6ff281db29be97a7999d5f3d6cd544f436 fr.hextm
102b73373eec17d901f64ec4c6f62307fc967ff7bf5fc91f6f2de97cf6835b02 it.hextm
4a1525710d62fc2a5c98951923153c8dfb3c72377c2ea47629d64a7dbf4a15a8 fr.capitals
e37a8663e8fa153574aa9863a76694eb83ac8430d2b2c75ae59dfbaded2ab9d5 it.capitals
fa47df56029370c3e17498d56970478bb9dee0335bda64c441f2a257c8382b08 es.valid
029c329c7eb5f7ee2855bf5717f73b9a66da116c8f3c1524e8f2d83e8116c00c fr.valid
2b8d6f840a7ffd608d6b4e2c28fc6f44db9159f84b9132ebe60e29579a1d676f it.valid
5633ec7bd91a4c43cafb5d62bea57409cebae47c6e8352f7e4b7b806da6916f6 de.valid
fa44f63d8d21411a36fb09e4c3be401420bf0c6b96518824123f2928ea732191 nb.valid