mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-22 21:23:02 +01:00
* sort language menus alphabetically * open language menus on their chosen language Whisper's list is longer than its menu shows, so sorting it by name can push the saved language's check below the edge. A select's press now reveals the chosen row through Rune's scroll request, in the kit's Select and Menu alike, and the map names the menu's order where it still described the catalog's. * docs: note language menus open on the chosen language
257 lines
16 KiB
Markdown
257 lines
16 KiB
Markdown
# Your Own Speech Models
|
||
|
||
Hex can run a GGUF speech model that isn't in its catalog: a file you add with
|
||
"Add model file…", or one another app already downloaded to the Hugging Face
|
||
cache. Both are ordinary features on every OS (they left Labs as
|
||
`custom-models`; `hex labs on|off custom-models` still answers and says so).
|
||
Catalog models stay the tested path: Hex vouches for nothing in a file it
|
||
didn't pin.
|
||
|
||
## Add A Model File
|
||
|
||
```ts
|
||
Picker header -> "Add model file…" -> native dialog (GGUF, All files) // setup.custom-models
|
||
├── Header check, before anything loads (transcription::gguf_header): GGUF v2/v3;
|
||
│ general.architecture parakeet | whisper | qwen3_asr | cohere_asr | sensevoice |
|
||
│ gigaam (any other, e.g. llama.cpp's or SenseVoice.cpp's `sensevoice-small`,
|
||
│ says the file was made for another program and needs a transcribe.cpp
|
||
│ conversion, or the catalog's download; #1299);
|
||
│ stt.variant; general.languages with a code Hex knows; only F32, F16, and
|
||
│ Q8_0 tensors (the types every build runs on Metal and Vulkan), plus
|
||
│ Q4_K, Q5_K, and Q6_K while Labs › Smaller model files is on (macOS; the
|
||
│ Vulkan builds embed no shaders for them); the tensor
|
||
│ data fits the file. Anything else -> "<file> can't be imported: <reason>"
|
||
├── Copy -> models/custom/<16 hex>.gguf while hashing; the copy is checked again;
|
||
│ receipt <16 hex>.gguf.verified.json (size, SHA-256, mtime, inode where
|
||
│ the OS reports one, never the volume's device number, which macOS can
|
||
│ renumber at a restart; older receipts' file_id is ignored); manifest <16 hex>.json (name, general.name, family,
|
||
│ variant, languages, detection, file name, licence, repo URL). The original
|
||
│ is never touched; a failed or cancelled import removes only its partial
|
||
│ copy; the same file again keeps the first import
|
||
├── ID custom-<first 16 hex of the SHA-256> (TranscriptionModelId::Custom)
|
||
├── Card -> name from general.name (Rename edits it), languages from the file,
|
||
│ offline only (no streaming), no language prompt for a no-hint family,
|
||
│ Auto when its header says it detects the language (as Parakeet v3-type
|
||
│ files do), no benchmark; Use prepares it like a catalog model (verifies
|
||
│ Hex's copy against its receipt instead of downloading). General's Language menu
|
||
│ lists every language the file declares alongside Hex's usual ones,
|
||
│ alphabetical by name (Norwegian `no`, Norwegian Nynorsk `nn`, …; #1148)
|
||
├── Remove -> Hex's copy, receipt, and manifest only; not the active model
|
||
├── Changed (a copy replaced by a link or hard link to the same file reads as
|
||
│ changed: its file ID and mtime moved) -> "Check again" rehashes the file:
|
||
│ the recorded SHA-256 again -> new receipt, ready; else it stays changed.
|
||
│ Missing or changed -> "Import again…": the same file replaces the copy and
|
||
│ keeps the import's name
|
||
├── The selected import missing or changed -> its card is not In use and
|
||
│ shows only Check again / Import again… (no Rename or Remove); a check or
|
||
│ import that succeeds asks for it again (ChooseModel), so the listener
|
||
│ leaves the fallback for it
|
||
└── Saved import missing or changed (size, mtime, or file ID) -> the choice
|
||
stays saved; the listener runs a downloaded fallback (the system language's
|
||
recommendation if downloaded, else the first downloaded catalog model;
|
||
custom_models::runnable); launch and setup wait only for that fallback
|
||
(onboarding::model_ready), and `hex status` names it under
|
||
model.fallback with its reason; General's model row names the fallback and says
|
||
to check it again in the picker (not "can't listen until downloaded",
|
||
while the fallback is downloaded); the card warns
|
||
```
|
||
|
||
With Labs › Model in use first on, an import in use when the picker opens
|
||
leads Every model, and "Your model files" shows above the language's group
|
||
([labs.md](labs.md)).
|
||
|
||
## Use A Model File Where It Is (Labs)
|
||
|
||
```ts
|
||
Labs › Use model files where they are on (labs.model_in_place, off by default) // labs.model-in-place
|
||
Picker header -> "Use model file where it is…" beside "Add model file…" -> the same dialog
|
||
├── Gate: the button shows only while the toggle is on, and the add starts only
|
||
│ if it is still on when the dialog answers (model_import::in_place_on)
|
||
├── custom_models::add_in_place on a worker: the import's header check, one full
|
||
│ SHA-256, receipt and manifest (with the file's path, no Hugging Face origin)
|
||
│ in models/custom/; nothing copied, nothing written near the file
|
||
├── The same file imported by copy before -> same ID, keeps its name; Hex's
|
||
│ copy is removed only when the OS shows the chosen file is a different
|
||
│ file (links resolved, device and inode or volume and file index); when
|
||
│ unsure, the copy stays
|
||
├── Hex's own copy chosen (a linked folder, a hard link, another case) ->
|
||
│ nothing changes; the import stays a copy
|
||
├── Card -> "Used where it is" · family · languages · the file's name; Use
|
||
│ loads it from its path; later uses rehash only when size, mtime, or file ID
|
||
│ change, and Check again accepts it when the contents still match
|
||
├── Another account changes or removes the file -> Changed or Missing, as above
|
||
└── Forget -> Hex's receipt and record only; the file stays
|
||
```
|
||
|
||
Off, the picker has no such button and Hex copies every added file as before.
|
||
Shared folders and catalog models from a shared folder are not built.
|
||
|
||
## Smaller Model Files (Labs, macOS)
|
||
|
||
```ts
|
||
Labs › Smaller model files on (labs.k_quant_models, off by default, macOS) // labs.k-quant-models
|
||
├── Add model file… and Use model file where it is… accept Q6_K, Q5_K, and
|
||
│ Q4_K tensors too, beside F32, F16, and Q8_0; the header check is the same
|
||
│ one (custom_models::profile, given the types this build can run), so a
|
||
│ file is refused or accepted before anything loads
|
||
├── The import's manifest records `k_quants: true`; manifests written before
|
||
│ the field read as false
|
||
├── The card stays in the picker with Rename and Remove, but no Use, while
|
||
│ the lab is off, and says "Turn on Labs › Smaller model files to use it.";
|
||
│ its availability folds in the lab (as Parakeet Ultra's does), so
|
||
│ `hex model list`, the trays, and the menu bar leave it out
|
||
├── Saving it as the model while the lab is off (or on Linux and Windows,
|
||
│ where the lab does not exist) -> the choice stays saved and the listener
|
||
│ runs the default model, as for a missing import: General says it needs
|
||
│ Labs › Smaller model files, and `hex status` says "its Labs toggle is
|
||
│ off" (custom_models::fallback_reason)
|
||
├── `hex model use <custom-…>` while the lab is off -> refused, naming
|
||
│ `hex labs on k-quant-models`
|
||
└── Off -> exactly the F32, F16, and Q8_0 rule and its message, byte for byte
|
||
```
|
||
|
||
Metal has the K-quant kernels (mul_mat and get_rows); the Linux and Windows
|
||
Vulkan builds embed shaders only for the catalog's types and ggml aborts a
|
||
Vulkan run whose weights use another type, so those builds refuse these files
|
||
whatever the setting says. A file another tool quantized may mix in types
|
||
neither list holds — Q5_0 and Q4_0 are the common ones from old-style
|
||
quantizers — and those are still refused with the type named. Turning the lab
|
||
off while a K-quant model is selected falls back to the default model at once
|
||
(the choice stays saved), and turning it back on runs the saved import again.
|
||
Hex has not measured any import's accuracy; a Q6_K, Q5_K, or Q4_K file may
|
||
transcribe worse than its Q8_0 source.
|
||
|
||
Checks:
|
||
`k_quants_are_known_only_where_the_caller_runs_them` and
|
||
`k_quant_tensors_are_measured_in_their_ggml_block_sizes` in
|
||
[transcription/gguf_header.rs](../../src/transcription/gguf_header.rs), and
|
||
`a_k_quant_import_needs_the_lab_and_falls_back_without_it` in
|
||
[transcription/custom_models.rs](../../src/transcription/custom_models.rs);
|
||
`smaller_model_files_is_its_own_switch_and_macos_only` in
|
||
[labs.rs](../../src/labs.rs). Not observed: a real K-quant file on Metal, and
|
||
the row on a Mac screen.
|
||
|
||
The local digest identifies the file Hex copied; it is not a publisher
|
||
checksum. The selection, recent models, and keyboard-language mappings keep an
|
||
import while it can't run (`models::validate_saved`), and a build that doesn't
|
||
know `custom-…` keeps its default for that field without rewriting
|
||
`settings.json` (#744). `hex model import <path> [--name N]`, `hex model rename
|
||
<id> <name>`, `hex model repair <id>`, and `hex model remove <id>` (also `hex
|
||
models …`) run in the CLI process; a running Hex lists the change at once (the
|
||
registry rereads `models/custom/` when its folder changes).
|
||
|
||
## Found On This Computer
|
||
|
||
```ts
|
||
Picker opens, or onboarding starts -> one scan (found_models::scan) // setup.found-models
|
||
├── Where: $HF_HUB_CACHE, else $HF_HOME/hub, else ~/.cache/huggingface/hub
|
||
│ (Windows %USERPROFILE%\.cache\huggingface\hub); models--*/snapshots/*/*.gguf
|
||
├── Cheap: headers only, through the import's checks; each file's result is kept
|
||
│ by path, size, mtime, and file ID, so an unchanged file is never read twice.
|
||
│ Never in the background, never a full hash during the scan
|
||
├── Exact catalog match: the snapshot links to blobs/<sha256> and that digest and
|
||
│ size are a catalog pin (a plain copy: repository, file name, and size)
|
||
│ -> that catalog model counts as downloaded; its card says "Found on this
|
||
│ computer" and offers Use, never Download
|
||
├── Other compatible files -> "Found on this computer" at the top of the picker:
|
||
│ name · Not tested by Hex · Q8_0/F16/F32 · owner/name@revision · languages,
|
||
│ "It may be too big for this computer's GPU memory." when the tensor data
|
||
│ plus a quarter exceeds the largest GPU transcribe.cpp reports, and
|
||
│ "Compatible doesn't mean accurate: …"
|
||
├── Use -> full SHA-256 once (a catalog file against its pin; another file against
|
||
│ its blob name), then prepared like any model. Receipts live with Hex:
|
||
│ models/found/<model>.json for a catalog file, models/custom/ for another;
|
||
│ later uses rehash only when size, mtime, or file ID change
|
||
├── A failed check or load -> "Model could not be used." with the reason; the
|
||
│ selection and any recorded audio stay as they were
|
||
└── Delete/Remove on a found file -> Forget: Hex's record goes, the file stays.
|
||
Hex never writes, moves, or deletes inside the cache. A forgotten catalog
|
||
file is noted in models/found/<model>.forgotten.json, so later scans no
|
||
longer count it as downloaded and its row offers Download (#1366)
|
||
```
|
||
|
||
`hex model list` scans too and shows each model's source: `download`, `hex`,
|
||
`imported`, or `found: owner/name@abcdef1`; a compatible file not yet used is
|
||
listed as `custom-<first 16 hex of its blob>` with `(not tested by Hex)`, and
|
||
`hex model use <that id>` checks it in full and uses it in place. The local
|
||
API's `/models` and the SDK's `ModelInfo.source` carry the same field.
|
||
|
||
## Onboarding Choice
|
||
|
||
```ts
|
||
Onboarding opens -> scan; its own download waits until it answers // setup.found-choice
|
||
├── Nothing compatible found -> Setting up Hex, unchanged: the recommended
|
||
│ model downloads by itself, and the model step never shows
|
||
└── Something found -> the first run's model step in place of Setting up Hex:
|
||
"Choose a speech model", one card of radio rows, no buttons inside
|
||
├── Hex's recommended model first, selected, [Recommended] -> "731 MB
|
||
│ download", or "On this computer" when its exact file (same SHA-256) is
|
||
│ in the cache (one row, never two)
|
||
├── Then each found file -> its name, "On this computer" (a catalog model)
|
||
│ or "On this computer · not tested by Hex"; no hashes, repositories,
|
||
│ revisions, or compression
|
||
├── Picking a row starts it: ours downloads ("Downloading… 42%" on the row)
|
||
│ or, already here, is checked in place; a found file is used where it
|
||
│ is ("Checking…"). A later pick supersedes one still preparing
|
||
├── Continue starts the selected row if none was picked, then the story
|
||
│ moves on once the pick is under way: ours with its download pill on
|
||
│ top, a found one with no download page or pill at all
|
||
├── A pick that fails asks again on the list
|
||
├── Back on the pages after it returns to the list to change the pick,
|
||
│ even with the model ready
|
||
└── The held download is released once the tour passes setup, so a run
|
||
that skips the step still gets the recommended model
|
||
```
|
||
|
||
Observed on apollo (Linux, Radeon 8060S over Vulkan, 2026-11-01) with an
|
||
isolated data folder and a fake hub holding Parakeet v3's pinned blob and
|
||
Parakeet primeLine Q8_0 (`handy-computer/parakeet-primeline-gguf@cf04224`):
|
||
`hex model list` showed v3 as `found: …@85ac09e` and primeLine as
|
||
`custom-dbd1ec2985aea64d (not tested by Hex)`; `hex model use` loaded each in
|
||
place, `hex model transcribe` transcribed with both, `hex model remove` forgot
|
||
primeLine and the cache's files, sizes, and times were unchanged. The
|
||
onboarding and picker scenes rendered from that real scan. Earlier, the same
|
||
primeLine file imported by copy transcribed the FLEURS German and English
|
||
samples. Not observed: macOS (Metal) and Windows at all, a plain-copy cache
|
||
without symlinks, and the GPU memory warning on a small GPU.
|
||
|
||
Checks: `transcription::found_models::tests` (scan, cache, catalog match,
|
||
in-place use and forgetting, a forgotten catalog file no longer counting as
|
||
downloaded, the full check refusing a mismatched file),
|
||
`transcription::gguf_header::tests`, `transcription::custom_models::tests`
|
||
(including `a_relinked_copy_reads_as_changed_and_check_again_accepts_only_the_same_contents`,
|
||
`a_receipt_survives_a_renumbered_volume_but_not_another_file`, and `a_file_added_in_place_is_recorded_by_path_copied_nowhere_and_forgetting_leaves_it`),
|
||
`desktop::onboarding::tests::a_held_download_waits_for_the_person`,
|
||
`a_found_model_moves_on_without_a_download_and_back_returns_to_it`,
|
||
`the_onboarding_model_list_starts_the_row_picked_or_the_recommended_one`,
|
||
`labs::tests::graduated_features_answer_by_name`, and the goldens
|
||
`rune-picker-custom-models-light`, `rune-picker-found-light`, and
|
||
`rune-onboarding-model-found-light`,
|
||
`rune-onboarding-model-found-downloading-light`,
|
||
`rune-onboarding-model-found-recommended-light`, and
|
||
`rune-onboarding-model-found-back-light`.
|
||
|
||
## A Download Another Process Runs
|
||
|
||
```ts
|
||
models/.download.lock held by `hex model install` (a terminal) or another Hex,
|
||
writing models/<file>.gguf.partial // transcription::models::lock_downloads
|
||
├── Onboarding, Settings, or `hex model install` starts the same model -> it
|
||
│ waits for that lock instead of failing or downloading a second copy
|
||
├── While it waits, downloaded_bytes comes from the partial file's size, so
|
||
│ `hex status --json`, the picker, and onboarding show that process's
|
||
│ progress rather than 0%
|
||
├── The lock covers the whole models folder, so the progress shown is this
|
||
│ model's partial file; while the other process fetches a different model
|
||
│ it stays flat
|
||
├── Cancel stops only the wait; the other process's download continues
|
||
└── The lock is released -> the finished file is verified against its
|
||
receipt and adopted, with no second download
|
||
```
|
||
|
||
Checks: `transcription::models::tests::waiting_for_another_processs_download_reports_its_partial_bytes`
|
||
holds the lock and writes three bytes of the partial, then asserts a waiting
|
||
download reports those bytes and stops canceled without fetching anything.
|
||
That a foreign lock makes `hex model install`/onboarding show it in the real
|
||
service has **not** been run; the Linux app's snapshot was checked by reading
|
||
`linux::app::snapshot`.
|