hex/android/benchmark/audio.tsv
Ryan Vogel a8f9b7073d
Hex for Android: on-device dictation keyboard (#2074)
* Build the native Android dictation keyboard

Add a three-page Compose setup and an InputMethodService with QWERTY editing, system IME switching, tap dictation and hold-to-talk. Capture inside the keyboard and decode a checksum-pinned, MIT-licensed Whisper tiny model so a fresh install works offline without an app switch, playback focus request or audio files.

Verified on the Studio API 35 ARM64 emulator: clean assembleDebug, three debug/release unit cases, system permission and keyboard setup, actual key editing, two WAVs through the guest microphone, insertion into a separate editor, cancellation and continued media output. The merged debug manifest has no Internet permission.

* Document Android setup and the recognizer decision

Record the supported toolchain, model and runtime licences, offline-first trade-off, and phase 1 limits. Document the headless emulator microphone path used for verification so the next phase can repeat it.

Checked the Studio build/run commands and both recognizers on the API 35 image with airplane mode enabled and media playing.

* Measure on-device Android dictation across four languages

Replace the phase-one single-phrase evidence with a fixed 40-clip multilingual, partly music-mixed corpus. Record per-clip errors, load, warm inference, memory, two-core sensitivity and delivered platform-service results separately. Keep an independent foreground editor and media track for mic-to-insertion verification.

Verified on an API 35 ARM64 Studio emulator: Whisper tiny/base/small, Parakeet TDT v3 int8 and strictly on-device Android recognition. Small in its short window made 15/285 word errors; the platform service returned 33/40. Both native runners also decoded the notebook clip after tightening the PCM input checks.

* Use verified Whisper small for styled, offline Android dictation

Replace tiny-only English recognition with a pinned small model selected by the multilingual benchmark. Keep bundled tiny for first-run offline use and download failures. Snapshot the style, cleanup, language and translation task per recording so a paste override affects only the intended editor and dictation.

Replace the recording status text and old meter with the iOS-style dot waveform, round cancel/finish controls and in-place spinner. Explain the sole network permission on welcome and bundle verified Inter v4.1 and its OFL notice for Compose and the keyboard.

Verified assembleDebug and tests on the Studio. The Android formatter matched 41 applicable iOS examples. Exercised tiny before download, observed progress and verified small and Inter hashes. The real IME inserted all four styles and both Spanish paste choices in airplane mode, with music continuing in an external editor; checked recording and transcribing in both themes.

* Add offline spatial typing assistance to the Android keyboard

Replace literal-only English and Spanish key handling with a native Kotlin Gaussian tap decoder, context-weighted word pairs, missed-space readings, and confidence margins. Map the pinned, uncompressed iOS-compatible Google Books v3 assets; credit the CC BY counts, SCOWL and MPL word filters. Use hashed local rejection counts so a second undo stops that correction without saving typed text.

Give the idle strip three frequency-ranked slots and add independent default-on key previews and system-aware haptics, plus slide-to-select accents. Commit space on the same key-up as letters: the previous posted click could reorder rapid input. Make a confident known spelling harder to replace than an unknown one.

Verified on Studio API 35: assembleDebug and tests, byte-identical English and Spanish models, the exact centre-tapped typo sentence in both themes, one-backspace undo and suppression after two undos across an IME restart, ranked suggestions, accent insertion, independent switches and the system vibration gate.

* Benchmark Android typing against iOS on synthetic key taps

Replay the 2,000 held-out English sentences through the installed Kotlin decoder and suggestion path, keeping only public synthetic rows from the mixed iOS experiment. Preserve aggregate scores, the unmodified Google Books model builder, and the two model digests; no labelled personal typing, offsets, or raw logs are committed.

Verified 1,461 errors over 13,870 words (10.53% Android; 8.39% iOS without offsets using the byte-identical model; 33.81% raw). On the Studio emulator the real editor received letters in 1.41 ms median and spaces in 3.16 ms across five exact sentence replays. Document the confidence margins, timing boundaries, feedback controls, licences and reproducible aggregate scorer.

* Align Android typing with iOS and add German and Bokmål layouts and models

Apply R1's English typo and contraction rules before spatial search and
match its language weight, shortlist and valid-spelling margins. A mapped
SCOWL spelling list supplies the validity check missing on Android. Scale
vertical spread and the missing-space cost to real Android row geometry so
"seehkw" still splits into "see how". Keep the existing en/es model bytes.

Build the new German and Bokmål HXTM files with the shared iOS writer.
German uses all Google Books v3 shards; Bokmål uses NB book counts filtered
by the complete 2022 Ordbank snapshot. Pin the new assets, credit their
sources and licences, and ship their real layouts, accents and suggestions.

Replace the post-setup list with the iOS-style Home, local Dictate bar and
four settings pages. Follow the system language by default; make the spoken
language switchable on space, and optionally pin an independent layout.

Verified on the Studio API 35 ARM64 emulator: 1,027 English and 1,374 Spanish
word errors vs R1's 1,163 and 1,387 on the same 2,000 synthetic sentences
per language. Ran all five unit tests in debug and release, inspected the
packaged assets, and checked external-editor suggestions, corrections, undo
and offline microphone insertion, plus Home and settings in both themes.
The Swift reader round-tripped both new models; the identical iOS files
are in the separate local commit 230a437e.

* Build a signed, shrunk Android 16 upload bundle

Replace the API 35 release defaults with API 36, an external upload key, R8 and resource shrinking. Omit the debug probes, benchmark receiver and unused permission, silence native speech diagnostics, and guard password-field context at the input connection. Give the download a generic user agent and ship the Android privacy policy offline.

Verified on the Studio: jarsigner, bundletool and byte-identical installed splits on API 31 and API 36; all ELF load segments and APK native offsets meet 16 KB alignment. Fresh model downloads and offline insertion passed on both. Binder traces showed zero reads in four password variants versus ten in an ordinary editor. Five tests passed in each build variant.

* Prepare the Android Play listing and device handoff

Add the en-US listing, Data safety and IARC drafts, four release screenshots, launcher-derived store art and a physical-phone checklist. Replace the obsolete API 35 build instructions with the signed API 36 workflow. Flag that the public privacy URL still needs the Android text and that suggestions can include strong language.

Verified title and description limits, native screenshot and artwork formats, model downloads and offline microphone insertion in an external editor on 2.5 GiB API 31 and 16 KB API 36 emulators. Recorded bundletool download size and sampled memory from the same signed release.
2026-11-20 17:27:17 +00:00

41 lines
2.9 KiB
Text

id samples_16khz sha256
01 48239 3adb293886cc42801c07eea3e82eaeeafbebc80cd37e7b412b37dd8562aed43b
02 44627 348b73cfb52819dc8ccc5b0c4861b5224024919b3a327cb17d01536c63ff6f42
03 35264 fe77dbde6429b925e1ae90cdf39d5052262accaef412c8b403acdfa3e072e91c
04 49694 0bdadf893c484a418b70ca5f0edaaf20cd1db3535f73fb72aec28277c2e8d818
05 55390 f5c353450767e9884a2cfb75e3173592d3a4e91e538c44205fb39e0a25978cbd
06 49808 9ee850748e2f23c6b9662f8716dd8f071c5135ab7b063db068d3823b33f26a1b
07 44295 c51ac314c4647a57e12d6b7f13b635d1734f16768ebb8f763cd414e94ad46b33
08 49864 37e7461d9a3aefb79b08d14ad71bb8ce144ff3c47b10aab133118945419b3130
09 50652 8c3db1f60d6f5bf336f086f7b775028e1c2bc0f9a7f5348b2977f989888650d3
10 47109 6364e9787b3eceee289c922dc9b26e5ff5d656fb99d9b741892297b525b6565e
11 43112 fc5f641b5569631d4ec1bb4395b1bd43910a804aebd00dd70357c2358831295f
12 51930 b56bd20e0a97af0c08f466cb5c6bbb459b132e639dc686dd961f177298265590
13 57940 9d9ea28527480def7a7056e1b85723fbad1b405f94f4b7da7bcf1d51ff18e42f
14 53917 36d4a74e5217cfe1ce685bc13c7503daa16a1e3203c2e1b7554fd860ed04f92e
15 47085 7bd80a01e3dbe9c30cd24082065307abefa6219319d29652c2767c9192f9837d
16 40948 2fc3eef9558df13a032bcc0b07a9909d5192a6547fcc36b614ce0c77a5121078
17 51479 3debdab57a9908eb25c1564f4c8147c55522a79943d5b6120761a7eb10f1b6c2
18 54433 13a537cb1814673499568c4de015e2dfa44bf9547f339a74a0770db1f510a8cf
19 52233 d67e2d378654d521fc00d1741bf34d481beeb85ae8ced1d68bf2c9822b510bb3
20 44293 8d1792c71d54129445730b30bc2e7681b4c0b92f6c500e46bc5a03b2b4fa246c
21 46988 b4d0c577dc7690db9aa79a58c62590ee8c749b4bcea0c42c6489c44551359531
22 51647 00f7afbfa313c3ad1c99df16b5629525b6363be7f01534717870cd88019d3226
23 54629 779e04fd10d515817fe49151fe95e6091c33eff0444614a4739621f490120127
24 59508 465928ecf55f64e6c94744c90c0fc808c5b5b304a35a126e53ba90623ec208b3
25 37775 cd9a84153e533e430725de3ac71b0963255efee399911fbd718fa7e91fd39a42
26 46567 e3486da198e68d172ddb3340d63c7fa71bac4eb80959af03ebbf985f28edcf24
27 37773 7212b299ee2e7690caae46d8dd24d2dc5b6e8b769183fcb25be471e7a50c6b34
28 35421 ec692b11e161625291fd1efbb46b7dee0d8515f6d8319354e7f7e12b53cd6882
29 40386 ffbbd3a56a5df2beca3ff93eaedbb46ae6e8445ed284851404b6a6564280b6f9
30 46705 a14dd33f1fe71567889a17fd8f2e3e955819378d377f5c6ca754c3f60e21b06e
31 35729 e38068852d1f77012017914aed267457d171ec3850f98fa88bd0592b937a3640
32 55069 0b6908582c7910e0a8b3c70b71feef7d8372245ca2482a55822dff9d45d0417a
33 46385 f9482412572d40696ac5b512207349ddf0c4848f095d760bdc4856327760c3b6
34 56752 34f09d1a269882fb55c696facfbfa47164f72000e938455e076bdc5109c157d3
35 55267 4632393807e7276fcdb095c4ce4475929de878a6830284b01389ba27eae4f6a4
36 36641 6a572e656a610d63d44bce49a98ceacf45aa507e24ef103f3d09d65b33a80fbe
37 45078 a05fe92cc57175c2da44a6400e93b3bf7a325bc480f4a3fed4be531144b02bf1
38 44101 c8bf04b992503e89c5117bc2bc36b122cf4e5e0b42f114e9c44e9349ea1ec085
39 46508 5e1065704b758beb0e8f5359829e6e76eb896bed19d6f46619387660d255d83c
40 48011 5f03254d4c24e11a564204767a1f30c03f258d256530e44930cdbd43d12e8f75