hex/docs/features/own-models.md
Dax 919f726add
sort language menus alphabetically (#1937)
* sort language menus alphabetically

* open language menus on their chosen language

Whisper's list is longer than its menu shows, so sorting it by name can
push the saved language's check below the edge. A select's press now
reveals the chosen row through Rune's scroll request, in the kit's
Select and Menu alike, and the map names the menu's order where it
still described the catalog's.

* docs: note language menus open on the chosen language
2026-11-17 11:31:28 -05:00

257 lines
16 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Your Own Speech Models
Hex can run a GGUF speech model that isn't in its catalog: a file you add with
"Add model file…", or one another app already downloaded to the Hugging Face
cache. Both are ordinary features on every OS (they left Labs as
`custom-models`; `hex labs on|off custom-models` still answers and says so).
Catalog models stay the tested path: Hex vouches for nothing in a file it
didn't pin.
## Add A Model File
```ts
Picker header -> "Add model file…" -> native dialog (GGUF, All files) // setup.custom-models
├── Header check, before anything loads (transcription::gguf_header): GGUF v2/v3;
│ general.architecture parakeet | whisper | qwen3_asr | cohere_asr | sensevoice |
│ gigaam (any other, e.g. llama.cpp's or SenseVoice.cpp's `sensevoice-small`,
│ says the file was made for another program and needs a transcribe.cpp
│ conversion, or the catalog's download; #1299);
│ stt.variant; general.languages with a code Hex knows; only F32, F16, and
│ Q8_0 tensors (the types every build runs on Metal and Vulkan), plus
│ Q4_K, Q5_K, and Q6_K while Labs › Smaller model files is on (macOS; the
│ Vulkan builds embed no shaders for them); the tensor
│ data fits the file. Anything else -> "<file> can't be imported: <reason>"
├── Copy -> models/custom/<16 hex>.gguf while hashing; the copy is checked again;
│ receipt <16 hex>.gguf.verified.json (size, SHA-256, mtime, inode where
│ the OS reports one, never the volume's device number, which macOS can
│ renumber at a restart; older receipts' file_id is ignored); manifest <16 hex>.json (name, general.name, family,
│ variant, languages, detection, file name, licence, repo URL). The original
│ is never touched; a failed or cancelled import removes only its partial
│ copy; the same file again keeps the first import
├── ID custom-<first 16 hex of the SHA-256> (TranscriptionModelId::Custom)
├── Card -> name from general.name (Rename edits it), languages from the file,
│ offline only (no streaming), no language prompt for a no-hint family,
│ Auto when its header says it detects the language (as Parakeet v3-type
│ files do), no benchmark; Use prepares it like a catalog model (verifies
│ Hex's copy against its receipt instead of downloading). General's Language menu
│ lists every language the file declares alongside Hex's usual ones,
│ alphabetical by name (Norwegian `no`, Norwegian Nynorsk `nn`, …; #1148)
├── Remove -> Hex's copy, receipt, and manifest only; not the active model
├── Changed (a copy replaced by a link or hard link to the same file reads as
│ changed: its file ID and mtime moved) -> "Check again" rehashes the file:
│ the recorded SHA-256 again -> new receipt, ready; else it stays changed.
│ Missing or changed -> "Import again…": the same file replaces the copy and
│ keeps the import's name
├── The selected import missing or changed -> its card is not In use and
│ shows only Check again / Import again… (no Rename or Remove); a check or
│ import that succeeds asks for it again (ChooseModel), so the listener
│ leaves the fallback for it
└── Saved import missing or changed (size, mtime, or file ID) -> the choice
stays saved; the listener runs a downloaded fallback (the system language's
recommendation if downloaded, else the first downloaded catalog model;
custom_models::runnable); launch and setup wait only for that fallback
(onboarding::model_ready), and `hex status` names it under
model.fallback with its reason; General's model row names the fallback and says
to check it again in the picker (not "can't listen until downloaded",
while the fallback is downloaded); the card warns
```
With Labs › Model in use first on, an import in use when the picker opens
leads Every model, and "Your model files" shows above the language's group
([labs.md](labs.md)).
## Use A Model File Where It Is (Labs)
```ts
Labs › Use model files where they are on (labs.model_in_place, off by default) // labs.model-in-place
Picker header -> "Use model file where it is…" beside "Add model file…" -> the same dialog
├── Gate: the button shows only while the toggle is on, and the add starts only
│ if it is still on when the dialog answers (model_import::in_place_on)
├── custom_models::add_in_place on a worker: the import's header check, one full
│ SHA-256, receipt and manifest (with the file's path, no Hugging Face origin)
│ in models/custom/; nothing copied, nothing written near the file
├── The same file imported by copy before -> same ID, keeps its name; Hex's
│ copy is removed only when the OS shows the chosen file is a different
│ file (links resolved, device and inode or volume and file index); when
│ unsure, the copy stays
├── Hex's own copy chosen (a linked folder, a hard link, another case) ->
│ nothing changes; the import stays a copy
├── Card -> "Used where it is" · family · languages · the file's name; Use
│ loads it from its path; later uses rehash only when size, mtime, or file ID
│ change, and Check again accepts it when the contents still match
├── Another account changes or removes the file -> Changed or Missing, as above
└── Forget -> Hex's receipt and record only; the file stays
```
Off, the picker has no such button and Hex copies every added file as before.
Shared folders and catalog models from a shared folder are not built.
## Smaller Model Files (Labs, macOS)
```ts
Labs › Smaller model files on (labs.k_quant_models, off by default, macOS) // labs.k-quant-models
├── Add model file… and Use model file where it is… accept Q6_K, Q5_K, and
│ Q4_K tensors too, beside F32, F16, and Q8_0; the header check is the same
│ one (custom_models::profile, given the types this build can run), so a
│ file is refused or accepted before anything loads
├── The import's manifest records `k_quants: true`; manifests written before
│ the field read as false
├── The card stays in the picker with Rename and Remove, but no Use, while
│ the lab is off, and says "Turn on Labs › Smaller model files to use it.";
│ its availability folds in the lab (as Parakeet Ultra's does), so
│ `hex model list`, the trays, and the menu bar leave it out
├── Saving it as the model while the lab is off (or on Linux and Windows,
│ where the lab does not exist) -> the choice stays saved and the listener
│ runs the default model, as for a missing import: General says it needs
│ Labs › Smaller model files, and `hex status` says "its Labs toggle is
│ off" (custom_models::fallback_reason)
├── `hex model use <custom-…>` while the lab is off -> refused, naming
│ `hex labs on k-quant-models`
└── Off -> exactly the F32, F16, and Q8_0 rule and its message, byte for byte
```
Metal has the K-quant kernels (mul_mat and get_rows); the Linux and Windows
Vulkan builds embed shaders only for the catalog's types and ggml aborts a
Vulkan run whose weights use another type, so those builds refuse these files
whatever the setting says. A file another tool quantized may mix in types
neither list holds — Q5_0 and Q4_0 are the common ones from old-style
quantizers — and those are still refused with the type named. Turning the lab
off while a K-quant model is selected falls back to the default model at once
(the choice stays saved), and turning it back on runs the saved import again.
Hex has not measured any import's accuracy; a Q6_K, Q5_K, or Q4_K file may
transcribe worse than its Q8_0 source.
Checks:
`k_quants_are_known_only_where_the_caller_runs_them` and
`k_quant_tensors_are_measured_in_their_ggml_block_sizes` in
[transcription/gguf_header.rs](../../src/transcription/gguf_header.rs), and
`a_k_quant_import_needs_the_lab_and_falls_back_without_it` in
[transcription/custom_models.rs](../../src/transcription/custom_models.rs);
`smaller_model_files_is_its_own_switch_and_macos_only` in
[labs.rs](../../src/labs.rs). Not observed: a real K-quant file on Metal, and
the row on a Mac screen.
The local digest identifies the file Hex copied; it is not a publisher
checksum. The selection, recent models, and keyboard-language mappings keep an
import while it can't run (`models::validate_saved`), and a build that doesn't
know `custom-…` keeps its default for that field without rewriting
`settings.json` (#744). `hex model import <path> [--name N]`, `hex model rename
<id> <name>`, `hex model repair <id>`, and `hex model remove <id>` (also `hex
models …`) run in the CLI process; a running Hex lists the change at once (the
registry rereads `models/custom/` when its folder changes).
## Found On This Computer
```ts
Picker opens, or onboarding starts -> one scan (found_models::scan) // setup.found-models
├── Where: $HF_HUB_CACHE, else $HF_HOME/hub, else ~/.cache/huggingface/hub
│ (Windows %USERPROFILE%\.cache\huggingface\hub); models--*/snapshots/*/*.gguf
├── Cheap: headers only, through the import's checks; each file's result is kept
│ by path, size, mtime, and file ID, so an unchanged file is never read twice.
│ Never in the background, never a full hash during the scan
├── Exact catalog match: the snapshot links to blobs/<sha256> and that digest and
│ size are a catalog pin (a plain copy: repository, file name, and size)
│ -> that catalog model counts as downloaded; its card says "Found on this
│ computer" and offers Use, never Download
├── Other compatible files -> "Found on this computer" at the top of the picker:
│ name · Not tested by Hex · Q8_0/F16/F32 · owner/name@revision · languages,
│ "It may be too big for this computer's GPU memory." when the tensor data
│ plus a quarter exceeds the largest GPU transcribe.cpp reports, and
│ "Compatible doesn't mean accurate: …"
├── Use -> full SHA-256 once (a catalog file against its pin; another file against
│ its blob name), then prepared like any model. Receipts live with Hex:
│ models/found/<model>.json for a catalog file, models/custom/ for another;
│ later uses rehash only when size, mtime, or file ID change
├── A failed check or load -> "Model could not be used." with the reason; the
│ selection and any recorded audio stay as they were
└── Delete/Remove on a found file -> Forget: Hex's record goes, the file stays.
Hex never writes, moves, or deletes inside the cache. A forgotten catalog
file is noted in models/found/<model>.forgotten.json, so later scans no
longer count it as downloaded and its row offers Download (#1366)
```
`hex model list` scans too and shows each model's source: `download`, `hex`,
`imported`, or `found: owner/name@abcdef1`; a compatible file not yet used is
listed as `custom-<first 16 hex of its blob>` with `(not tested by Hex)`, and
`hex model use <that id>` checks it in full and uses it in place. The local
API's `/models` and the SDK's `ModelInfo.source` carry the same field.
## Onboarding Choice
```ts
Onboarding opens -> scan; its own download waits until it answers // setup.found-choice
├── Nothing compatible found -> Setting up Hex, unchanged: the recommended
│ model downloads by itself, and the model step never shows
└── Something found -> the first run's model step in place of Setting up Hex:
"Choose a speech model", one card of radio rows, no buttons inside
├── Hex's recommended model first, selected, [Recommended] -> "731 MB
│ download", or "On this computer" when its exact file (same SHA-256) is
│ in the cache (one row, never two)
├── Then each found file -> its name, "On this computer" (a catalog model)
│ or "On this computer · not tested by Hex"; no hashes, repositories,
│ revisions, or compression
├── Picking a row starts it: ours downloads ("Downloading… 42%" on the row)
│ or, already here, is checked in place; a found file is used where it
│ is ("Checking…"). A later pick supersedes one still preparing
├── Continue starts the selected row if none was picked, then the story
│ moves on once the pick is under way: ours with its download pill on
│ top, a found one with no download page or pill at all
├── A pick that fails asks again on the list
├── Back on the pages after it returns to the list to change the pick,
│ even with the model ready
└── The held download is released once the tour passes setup, so a run
that skips the step still gets the recommended model
```
Observed on apollo (Linux, Radeon 8060S over Vulkan, 2026-11-01) with an
isolated data folder and a fake hub holding Parakeet v3's pinned blob and
Parakeet primeLine Q8_0 (`handy-computer/parakeet-primeline-gguf@cf04224`):
`hex model list` showed v3 as `found: …@85ac09e` and primeLine as
`custom-dbd1ec2985aea64d (not tested by Hex)`; `hex model use` loaded each in
place, `hex model transcribe` transcribed with both, `hex model remove` forgot
primeLine and the cache's files, sizes, and times were unchanged. The
onboarding and picker scenes rendered from that real scan. Earlier, the same
primeLine file imported by copy transcribed the FLEURS German and English
samples. Not observed: macOS (Metal) and Windows at all, a plain-copy cache
without symlinks, and the GPU memory warning on a small GPU.
Checks: `transcription::found_models::tests` (scan, cache, catalog match,
in-place use and forgetting, a forgotten catalog file no longer counting as
downloaded, the full check refusing a mismatched file),
`transcription::gguf_header::tests`, `transcription::custom_models::tests`
(including `a_relinked_copy_reads_as_changed_and_check_again_accepts_only_the_same_contents`,
`a_receipt_survives_a_renumbered_volume_but_not_another_file`, and `a_file_added_in_place_is_recorded_by_path_copied_nowhere_and_forgetting_leaves_it`),
`desktop::onboarding::tests::a_held_download_waits_for_the_person`,
`a_found_model_moves_on_without_a_download_and_back_returns_to_it`,
`the_onboarding_model_list_starts_the_row_picked_or_the_recommended_one`,
`labs::tests::graduated_features_answer_by_name`, and the goldens
`rune-picker-custom-models-light`, `rune-picker-found-light`, and
`rune-onboarding-model-found-light`,
`rune-onboarding-model-found-downloading-light`,
`rune-onboarding-model-found-recommended-light`, and
`rune-onboarding-model-found-back-light`.
## A Download Another Process Runs
```ts
models/.download.lock held by `hex model install` (a terminal) or another Hex,
writing models/<file>.gguf.partial // transcription::models::lock_downloads
├── Onboarding, Settings, or `hex model install` starts the same model -> it
│ waits for that lock instead of failing or downloading a second copy
├── While it waits, downloaded_bytes comes from the partial file's size, so
│ `hex status --json`, the picker, and onboarding show that process's
│ progress rather than 0%
├── The lock covers the whole models folder, so the progress shown is this
│ model's partial file; while the other process fetches a different model
│ it stays flat
├── Cancel stops only the wait; the other process's download continues
└── The lock is released -> the finished file is verified against its
receipt and adopted, with no second download
```
Checks: `transcription::models::tests::waiting_for_another_processs_download_reports_its_partial_bytes`
holds the lock and writes three bytes of the partial, then asserts a waiting
download reports those bytes and stops canceled without fetching anything.
That a foreign lock makes `hex model install`/onboarding show it in the real
service has **not** been run; the Linux app's snapshot was checked by reading
`linux::app::snapshot`.