mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-22 21:23:02 +01:00
* labs: add hud height for the bottom dictation hud on macos (#2411) * labs: keep hud padding when raising the bottom gap * docs: describe hud height window raise
711 lines
44 KiB
Markdown
711 lines
44 KiB
Markdown
# `hex` control CLI
|
||
|
||
The control commands drive the one running Hex: start or stop a dictation or a
|
||
meeting, change any setting, switch models, open the window, quit, or read its
|
||
status. They are meant for agents and scripts as much as people, so every
|
||
command takes `--json`, is idempotent, and exits with a stable code.
|
||
|
||
One command tree (`control::cli`) is compiled on macOS, Linux, and Windows,
|
||
and one versioned protocol carries it on every OS through the running Hex's
|
||
local API: the owner-only discovery file (`local-api.json`) and the
|
||
bearer-authenticated `POST /control` (`local_api`).
|
||
|
||
On Linux and Windows one shared host (`control::host`) carries out the
|
||
commands through each OS's thin adapter (`linux/app`, `windows/rune`); macOS's
|
||
host is `macos_rune::control`. A command the running Hex can't do on its OS
|
||
answers `unsupported` at runtime (exit 1) instead of being missing from the
|
||
CLI; `hex status --json` lists the OS's `capabilities`. Today that is
|
||
playground stories (`window story`) outside macOS and the local `model`
|
||
tools on macOS.
|
||
|
||
On Windows a terminal runs `hex` as `hex.com`, a console program that runs
|
||
the app's `hex.exe` and waits for it, so commands print, return their exit
|
||
code, and can be captured like any other (`docs/windows.md`).
|
||
|
||
The read-only `hex history` and `hex meetings list|show|chapters|search` commands, and
|
||
`hex mcp`, read files rather than the running Hex; see `docs/transcript-cli.md`.
|
||
|
||
## Commands
|
||
|
||
```ts
|
||
hex
|
||
├── status what Hex is doing (see Status below)
|
||
├── start open Hex if it isn't running (macOS Hex.app, else `hex app --hidden`); wait until it answers
|
||
├── quit [--force] refused while a meeting or dictation is active, or unconfirmed
|
||
├── restart [--force] quit and reopen (opens Hex when it isn't running)
|
||
├── dictation start|stop|toggle
|
||
│ ├── start --print (macOS) answer with the transcript when the recording ends
|
||
│ ├── stop --print print the transcript instead of pasting it (below)
|
||
│ └── start|toggle --raw paste the transcript as heard, skipping the Output steps
|
||
├── paste-last paste the last dictation (Option-Shift-V); refused while recording
|
||
├── typed-text Labs › Transform typed text (Linux): replace the focused field's text
|
||
├── transcribe <file> [--model <id>] [--language L] print an audio or video file's transcript
|
||
├── meetings
|
||
│ ├── record [--title T] as the menu bar's Record Meeting; needs meetings.enabled
|
||
│ ├── stop Stop & Transcribe
|
||
│ ├── copy-last copy the last meeting transcript to the clipboard
|
||
│ ├── delete <id> move a finished meeting's Markdown and recording to the Trash
|
||
│ └── list | show | search read-only (docs/transcript-cli.md)
|
||
├── settings
|
||
│ ├── list every setting: key, value, kind, choices, writable, about
|
||
│ ├── get <key|prefix> one setting, or every key under a prefix (`meetings`)
|
||
│ └── set <key> <value>
|
||
├── model
|
||
│ ├── list every model, its languages and size, installed or not
|
||
│ ├── use <id> [--language L] installed models only; never downloads or repairs
|
||
│ ├── install [id] [--language L] download, then switch to it (default: the selected model)
|
||
│ ├── cancel stop a download or load in progress
|
||
│ ├── import <path> [--name N] Labs: check and copy in a GGUF; prints its custom-… id
|
||
│ ├── rename <id> <name> Labs: an import's picker name
|
||
│ ├── repair <id> Labs: rehash Hex's copy; ready again if it still matches
|
||
│ ├── remove <id> Labs: Hex's copy of an import (not the selected model)
|
||
│ └── status|devices|check|transcribe <wav> Linux and Windows, run in this process
|
||
├── commands
|
||
│ ├── init create ~/.config/hex (local, no running Hex needed)
|
||
│ └── check validate ~/.config/hex/hex.config.ts transformations
|
||
├── window
|
||
│ ├── show [settings|models|modes|history|meetings]
|
||
│ ├── hide close the window; Hex stays in the menu bar
|
||
│ └── story <story> [--variant V] [--play] the playground (Labs): open a story
|
||
├── onboarding reset (alias restart) replay the first run now; every OS (below)
|
||
├── history clear --yes delete every retained dictation
|
||
├── vocabulary the shared vocabulary (features/vocabulary.md)
|
||
│ ├── list [--json] every term and its "sounds like" spellings
|
||
│ ├── add <term> [--sounds-like a,b] add a term, or new spellings to an existing one
|
||
│ └── remove <term> any case
|
||
├── annotate (alias capture) Annotate (features/capture.md)
|
||
│ ├── start|stop|cancel|replay|status record a capture, end or discard it, save the last 15 s, or report
|
||
│ ├── list [--limit N] the newest captures in the captures folder
|
||
│ ├── show <capture> one capture's words, files, and where it went
|
||
│ ├── send <capture> [--to <session>] [--always] send it to an OpenCode session
|
||
│ ├── route <capture> [--ask-jev] what routing sees and decides; sends nothing
|
||
│ └── inspect|zoom|frames|skill Labs: read a capture for a coding agent (off by default)
|
||
├── screenshot [--pick] Labs (macOS): shoot the display under the pointer and print the PNG's path; --pick opens the ⌘⇧4 picker
|
||
├── labs [on|off <feature>] list Labs features or turn one on/off; edits settings.json (features/labs.md)
|
||
│ └── link <feature> macOS: print hex://labs/<feature>, the link that opens Hex's Labs page on that lab's group
|
||
├── check-for-updates check for an update now
|
||
├── launch-at-login [on|off] report or change Launch at Login (macOS: installed Hex.app only; Windows Run key; Linux XDG autostart)
|
||
├── completions <shell> print a completion script for hex
|
||
└── logs [--process] [-n N] [-f] tail live.ndjson (or process.log); no running Hex needed
|
||
```
|
||
|
||
`dictation start` locks the recording, as a double tap would: `dictation stop`
|
||
or the next shortcut press finishes it, and `dictation cancel` discards it.
|
||
A CLI start has no press to judge, so its recording counts as
|
||
intentional at once: the start tone plays once the microphone is open, and the
|
||
recording behavior (mute or pause media, idle-sleep prevention) and the live
|
||
stream start as they do after a held shortcut. A `stop` replies `done` only
|
||
when the clip went to transcription; a clip with no audio yet (the microphone
|
||
was still opening) or a silent one is dropped with `discarded`, and nothing
|
||
shows as transcribing.
|
||
|
||
`dictation stop --print` stops the recording at once, exactly as `stop`
|
||
does, so a following `start` isn't blocked; only this command then waits, up
|
||
to 60 s, for the transcript and writes it to stdout instead of pasting it. The
|
||
text gets the same processing as a paste (global replacements, the mode's
|
||
corrections, OpenCode rewriting, and transformations). History keeps it as
|
||
`printed`, and Paste Last skips it. No speech (nothing recorded yet, a silent
|
||
clip, or an empty transcript) exits 4 with code `no-speech` and prints
|
||
nothing; with nothing recording it refuses with `not-recording`. A failure,
|
||
or the 60 s timeout prints nothing, exits 1, and never
|
||
falls back to pasting. `--print` with `toggle` or `cancel` is a usage error
|
||
(exit 2). The command is the same on Linux and Windows, where `hex dictation`
|
||
and `hex paste-last` reach the listener's own queue as its shortcut would;
|
||
Linux's `hex typed-text` does too.
|
||
|
||
`dictation start --print` (macOS) records a dictation that belongs to the
|
||
command instead of the person's focused app: the command blocks and answers only when the recording ends,
|
||
however it ends, with the transcript (`transcript`, exit 0) instead of pasting
|
||
or copying it. It starts exactly as `start` does and locks the recording; the
|
||
shortcut press that finishes it, `dictation stop`, `dictation toggle`, and a
|
||
microphone loss that keeps the audio all hand the text to that command, never
|
||
to whatever app has focus. A discarded or silent clip answers `no-speech`
|
||
(exit 4), a cancel (`dictation cancel` or Toggle's cancel shortcut) answers
|
||
`failed` (exit 1), and any other end, such as a microphone loss with no kept
|
||
audio or suspended shortcuts, answers `failed` too, so the command never
|
||
hangs. A printing `start` while anything already records is refused with
|
||
`busy`: it never takes over the person's own dictation or a programmatic one.
|
||
A plain `dictation stop` of a print-only recording answers as any plain stop
|
||
does (`done`, or `discarded` for no audio or silence) and the transcript still
|
||
goes to the start; `stop --print` of it answers `done` too, and both commands
|
||
wait for the one transcript. On Linux and Windows it answers `unsupported`.
|
||
The text gets the same
|
||
processing as a paste and History keeps it as `printed`, as `stop --print`
|
||
does; Paste Last skips it. A printing start waits at most 31 minutes; if it
|
||
gives up, the recording stays print-only and is never pasted. `--raw` is
|
||
allowed: the transcript comes back as heard. `--print` with `toggle` or
|
||
`cancel` is a usage error (exit 2).
|
||
|
||
`dictation start --raw` and `toggle --raw` start a dictation that pastes the
|
||
transcript as heard, skipping the Output pane's steps, as the Dictate without
|
||
processing shortcut (`raw_dictation_hotkey`) does. History keeps it as `raw`,
|
||
and Paste Last pastes it. `--raw` with `stop` is a usage error
|
||
(exit 2); `stop --print` of a raw dictation prints the transcript as heard.
|
||
|
||
`dictation cancel` discards the recording in any shortcut mode: nothing is
|
||
transcribed, pasted, or saved to History, and dictations already transcribing
|
||
finish. With nothing recording it answers `unchanged`. In Toggle mode the
|
||
cancel shortcut (`cancel_hotkey`, Escape by default) does the same; on Wayland,
|
||
bind `hex dictation cancel` to a key in the desktop instead (#786, #1022).
|
||
`meetings cancel` doesn't exist because Hex has no way to discard a
|
||
meeting recording.
|
||
|
||
`onboarding reset` (or `onboarding restart`) opens the window on the
|
||
first-run onboarding at once, as on a new install, for trying it again, on
|
||
macOS, Linux, and Windows. It removes only the onboarding markers (`onboarding-complete`, `onboarding-step`,
|
||
`onboarding-relaunch`) from the data folder. Settings, history, models, and
|
||
permissions are untouched, so the run finds them ready: the download passes at
|
||
once and the permission step is skipped. Labs' Run setup again row does the
|
||
same.
|
||
|
||
`paste-last` needs a listening Hex (`not-ready` during setup) and is refused
|
||
with `busy` while anything records or a shortcut gesture is in progress. The
|
||
menu bar's Paste Last Dictation does nothing in that case.
|
||
|
||
On a Mac with [Labs › Paste Last picker](features/dictation.md#paste-last-picker-labs-macos)
|
||
on, the same command immediately pastes the newest eligible dictation or
|
||
Annotate capture. It does not open a picker or send the capture anywhere. Its
|
||
reply acknowledges queue admission, not receipt by the focused app.
|
||
|
||
`typed-text` (Linux) runs the same job as Labs › Transform typed text's
|
||
shortcut, for desktops without the shortcuts portal. It needs a listening Hex
|
||
and the lab on (`disabled` when it is off), and refuses while Terminal paste
|
||
shortcut is on (`disabled`), because its Ctrl+V would become Ctrl+Shift+V. It
|
||
sends Ctrl+A and Ctrl+C, so it is not meant for terminals. While a capture
|
||
records or finishes, it answers done but the listener drops the job.
|
||
|
||
`transcribe <file>` prints a recording's transcript to stdout with the
|
||
selected dictation model, or the one `--model` names; `--language` picks a
|
||
language the model supports. It reads WAV, AIFF, CAF, FLAC, MP3, Ogg (Vorbis
|
||
and Opus), WebM/Matroska audio, and MP4/M4A/MOV audio (AAC-LC, HE-AAC, ALAC),
|
||
from audio files or a video's soundtrack, decoded in this process with no
|
||
external tool. The file is opened and its first audio decoded before anything
|
||
else runs, so a file Hex can't read is refused with `unsupported`. A
|
||
recording of any length is decoded and sent in pieces of at most 30 seconds,
|
||
so Hex holds about one piece rather than the whole file; the pieces' text is
|
||
joined with one space.
|
||
|
||
Each piece goes to the local API's `POST /transcriptions` with that model
|
||
named. With Hex running, that is the app's own API: a piece that short runs
|
||
on Hex's dictation engine when it already holds that model, so no second copy
|
||
loads and dictation can run between pieces. With Hex stopped, the command
|
||
starts its own `hex service --embedded` (the TypeScript SDK's direct child),
|
||
uses it for every piece, and ends it on every exit; it never publishes it or
|
||
starts a second service beside a running Hex. A model that isn't downloaded
|
||
is offered once, `Download <model> (<size>)? [Y/n]`, only when stdin and
|
||
stderr are terminals and `--json` is off; yes downloads, verifies, and loads
|
||
it, then the piece is tried again. Otherwise the command answers
|
||
`model-unavailable` and names `hex model install`.
|
||
|
||
The service skips silence (a piece the speech detector hears nothing in, once
|
||
it is installed), and only a file with no speech at all answers `no-speech`
|
||
(exit 4). stdout carries only the transcript or the `--json` reply (the same
|
||
`transcript` status `dictation stop --print` uses); a progress line goes to
|
||
stderr only when it is a terminal. Ctrl-C ends the command and its service,
|
||
printing nothing, with exit 130. It never pastes, records History, runs mode
|
||
processing, or logs the transcript.
|
||
|
||
`completions <shell>` prints a completion script for `hex` to stdout and
|
||
nothing else. The script is
|
||
generated at runtime from the same clap definitions every command parses, so
|
||
a new command or flag is in it without anyone writing the script out again;
|
||
`bash`, `zsh`, `fish`, `elvish`, and `powershell` are the shells.
|
||
|
||
```sh
|
||
hex completions bash > ~/.local/share/bash-completion/completions/hex
|
||
mkdir -p ~/.zfunc && hex completions zsh > ~/.zfunc/_hex
|
||
hex completions fish > ~/.config/fish/completions/hex.fish
|
||
```
|
||
|
||
For zsh, put `~/.zfunc` on `fpath` before `compinit` runs, in `~/.zshrc`:
|
||
`fpath=(~/.zfunc $fpath)`. Elsewhere, write the script into a directory
|
||
already on `fpath`.
|
||
|
||
`model import`, `rename`, `repair`, and `remove` run in this process on every
|
||
OS; `hex models` is an alias for `hex model`. An import's id is `custom-` and
|
||
the first 16 hex digits of its SHA-256, and every model command takes it. One
|
||
that is missing or changed answers `model-unavailable` with the reason.
|
||
`model list` also looks in the Hugging Face cache and says where each model
|
||
comes from (`download`, `hex`, `imported`, or `found: owner/name@abcdef1`); a
|
||
compatible file there that isn't a catalog model is listed as `custom-<first 16
|
||
hex of its blob>` `(not tested by Hex)`, and `model use` with that id checks it
|
||
in full and uses it where it is. `remove` on a found file only forgets it
|
||
([own models](features/own-models.md)).
|
||
|
||
`model use` switches only to a downloaded model and checks it on disk; it never
|
||
downloads or repairs one (`model install` does). It keeps that model's own
|
||
remembered language, as the menu bar does; `--language`
|
||
overrides only the language, and must be one the model supports.
|
||
|
||
`vocabulary` edits the [shared vocabulary](features/vocabulary.md) through the
|
||
running Hex, which saves settings and reloads them as a window edit would.
|
||
Terms match in any case; `add` of a term already listed only merges new
|
||
spellings, and replies `changed: false` when there are none.
|
||
|
||
`quit` and `restart` without `--force` are refused with `busy` when the
|
||
listener doesn't answer whether it's recording; no listener, or one waiting for
|
||
setup, counts as idle. On Linux a Hex its systemd unit runs restarts through
|
||
the unit (after writing this desktop's `session.env`); elsewhere `restart`
|
||
waits for the old Hex to drain and exit, then runs `hex start`.
|
||
|
||
On Linux, an older Hex an update left running (from before `hex` commands
|
||
moved to the local API) answers only on its `service.sock`. Every control
|
||
command and `hex start` then answer `outdated` (exit 1) instead of
|
||
`not-running`, and `hex restart` restarts its systemd unit into the installed
|
||
build and waits for it to answer (#915); one not run by the unit has to be
|
||
quit from its window or tray.
|
||
|
||
`model install` with Hex running downloads in the background, as Settings
|
||
does; with no Hex running it downloads here (on macOS through an embedded
|
||
service, as `hex transcribe` does) and then selects the model, printing progress (the source-install step `hex model
|
||
install`). A Hex that starts while the download runs makes the switch instead.
|
||
`model status`, `devices`, `check`, and `model transcribe` are local tools on Linux
|
||
and Windows and take the selected model unless one is named; macOS answers
|
||
`unsupported`.
|
||
|
||
`start` and `restart` keep the `HEX_*` environment (`HEX_RETAIN_DICTATION_AUDIO`,
|
||
`HEX_ENABLE_LLM`, and so on): macOS opens apps without the caller's
|
||
environment, so Hex passes each `HEX_*` variable to `open --env`. `start`
|
||
takes them from the shell that runs it, `restart` from the running Hex. The
|
||
values are never logged.
|
||
|
||
`annotate` (or its old name `capture`) drives [Annotate](features/capture.md),
|
||
in every macOS, Linux, and Windows build; with `capture.enabled` off it refuses with
|
||
`feature-off`. On Linux, `list`, `show`, `route`, and `send` run in the
|
||
`hex` process itself and `start`, `stop`, and `cancel` are handed to the
|
||
running Hex's listener, as is `status`. On Windows every `annotate` command
|
||
goes to the running Hex. `capture stop` replies at once with the bundle
|
||
path while the words and files are written in the background; `capture replay`
|
||
(macOS 27+, Dev Lab `annotate-replay` on) exports the rolling 15 s
|
||
`SCClipBufferingOutput` display buffer to `<bundle>/recording.mp4` and replies
|
||
with the bundle path while keyframes, clipboard copy, and routing finish on the
|
||
same queue (`unsupported` outside macOS 27+, `feature-off` while
|
||
`labs.annotate_replay` is off); `capture status`
|
||
lists bundles still writing and the last outcome (`last`: `bundle`, `stage`
|
||
of `done`, `failed`, or `cancelled`, and for a written bundle `words`, `gaps`,
|
||
`missing_transcript` (why it has no words), and `audio_stopped_at_ms` when Hex
|
||
had to stop keeping its audio, a streamed capture past its 8 h disk guard or a
|
||
failed audio write). A start or replay while two captures
|
||
are still writing is refused with `busy`. Every action is idempotent.
|
||
|
||
`<capture>` is a bundle name (`1792933491953.scrawl`, the extension
|
||
optional), a path, or `latest`. `list`, `show`, and `route` read the captures
|
||
folder and need only the experiment; `send` needs captures on and a listening
|
||
Hex. `send` with `--to` takes a session ID; for the newest capture it also
|
||
takes a name its route lists, and it acts as the capsule's pick, so the
|
||
capsule's Undo follows it. Without `--to` it goes where the capture's route
|
||
(its rule, or its first guess) or its last send says. `--always`, for the
|
||
newest capture only, also saves a rule sending its site or app there, as the
|
||
capsule's ⇧⏎ does. Its reply says what became of it (`Sent to …`,
|
||
`Queued for …` when OpenCode didn't answer, which Hex retries). `route` is
|
||
Scrawl's `route --explain` (`--explain` is accepted); `--ask-jev` asks Jev
|
||
again, one network call. An unknown capture is refused with `not-found`.
|
||
|
||
`inspect`, `zoom`, `frames`, and `skill` are Labs › Agent tools for captures
|
||
(`labs.annotate_agent_tools`, off by default; `hex labs on
|
||
annotate-agent-tools`). They run in the `hex` process on every OS, with Hex
|
||
closed: they only read the bundle and decode its video, never starting a
|
||
capture, sending one, or writing a setting. `frames` extracts with `ffmpeg`
|
||
from PATH when one is installed, as the engine's own `extract_frames_range`
|
||
does. While the lab is off they refuse
|
||
with that switch's name. `inspect [<capture>]` prints `summary.md`; `--marks`
|
||
prints one line per mark (`#2 ✎② 0:09 circled · m2`), or `No marks`; `--json`
|
||
prints `scrawl.json` pretty, or with `--marks` the marks as a JSON array.
|
||
`zoom [<capture>] (--at <M:SS|secs|phrase> | --mark <n|id> [--frame N])
|
||
[--span secs] [--frames N] [--full]` writes a dense sheet of frames around
|
||
that moment into `<bundle>/zoom/` and prints its path, `0:09.500 → 0:11.500,
|
||
12 frames: #1 0:09.500, …`, and `Said: “…”` when anything was said over the
|
||
window; for a mark it prints the mark's line first. `frames [<capture>]
|
||
--from <M:SS|secs> --to <M:SS|secs> [--fps 4]` writes one PNG per frame into
|
||
`<bundle>/zoom/range-*/`, clamped to the recording's length and at most 240
|
||
frames a call, and prints each path. `skill` prints the
|
||
[`hex-annotate`](../skills/hex-annotate/SKILL.md) skill this Hex ships: the
|
||
same guide the Annotate pane's Agents section writes into a coding agent's
|
||
folder while Labs › Install agent skill from Annotate is on.
|
||
|
||
A CLI-started meeting is the same recording the menu bar starts: the menu-bar
|
||
state, the recording capsule, and Stop & Transcribe all apply to it.
|
||
`meetings record` replies once the recording has started (or after 8 s);
|
||
`meetings stop` replies once transcription has begun.
|
||
|
||
`meetings delete <id>` is the Meetings pane's Delete, carried out by the
|
||
running Hex on every OS: it moves the meeting's exported Markdown, any
|
||
separate notes file Hex tracks, and then its recording folder to the Trash,
|
||
never removing anything outright, so an open Meetings window drops the row at
|
||
its next scan and the files stay recoverable from the Trash. The id is
|
||
exactly the one `hex meetings list --json` prints (no prefixes). An unknown
|
||
id, or a meeting that is recording, transcribing, writing notes, or waiting for
|
||
the final transcript of a run Hex quit during, is refused with its own code
|
||
(see Error codes) and exit 1, and nothing moves. It asks no confirmation and
|
||
works whether or not Meetings is on, so a caller should ask the person first
|
||
and offer the same two-step confirmation the pane does.
|
||
|
||
## Exit codes
|
||
|
||
```ts
|
||
0 the command ran, or there was nothing to change (`changed: false`)
|
||
1 Hex refused or failed it (an error reply), or the transport failed
|
||
2 usage error (clap)
|
||
3 Hex isn't running (code `not-running`); control commands never launch it
|
||
4 `dictation start --print`, `dictation stop --print`, or `hex transcribe` heard no speech (code `no-speech`); nothing printed
|
||
130 `hex transcribe` or `hex model install` ended by Ctrl-C; nothing printed
|
||
```
|
||
|
||
## Transport
|
||
|
||
The CLI reads the owner-only discovery file
|
||
`~/Library/Application Support/hex/local-api.json` (`HEX_APPLICATION_SUPPORT_DIR`
|
||
overrides the folder), checks that its pid is alive, and sends one
|
||
`POST /control` to `127.0.0.1:<port>` with `Authorization: Bearer <token>`.
|
||
The route is always on in the macOS desktop app; the headless `hex service`
|
||
doesn't offer it. Linux and Windows Hex publish the same local API (the
|
||
discovery file lives in `~/.local/share/hex` and `%APPDATA%\hex`) for
|
||
transcription and `/streams`, but answer `/control` with
|
||
`404 control-unavailable` until their hosts carry the commands.
|
||
|
||
- Request bodies are JSON up to 64 KiB; replies are at most 1 MiB.
|
||
- The app answers within 10 s (`504 control-timeout` otherwise); the CLI waits
|
||
15 s. `dictation stop --print` gets 65 s, and the CLI waits 70 s;
|
||
`dictation start --print` gets 31 min 5 s, and the CLI waits 31 min 10 s.
|
||
- A waiting `dictation start --print` holds one of the local API's four
|
||
connection workers until its recording ends or it hits that cap, even if
|
||
the CLI that asked is killed: it answers a dropped receiver harmlessly,
|
||
never pastes, and frees the worker only then.
|
||
- Every answer the app gives is `200` with a `ControlResponse`, refusals
|
||
included. Transport-level failures use HTTP status codes:
|
||
`400 invalid-control-command` (malformed or unknown command),
|
||
`404 control-unavailable` (no control host), `413 control-too-large`,
|
||
`503 control-busy` (too many in flight) or `control-unavailable`, and
|
||
`504 control-timeout`. Bad or missing tokens get `401` like every other
|
||
route.
|
||
|
||
`hex transcribe` uses that same owner-only discovery file and bearer token
|
||
(or, with Hex stopped, the endpoint and token its embedded service prints),
|
||
but sends the file one 30 s piece at a time, as 16 kHz mono WAV, to
|
||
`POST /transcriptions?model=<id>&language=<code>` with `Content-Type:
|
||
audio/wav`. The route answers each piece's transcript
|
||
(`{"transcript": "…", "durationMs": …}`) or a code such as
|
||
`model-not-ready`, `unsupported-audio`, `queue-full`, or `resource-exhausted`.
|
||
|
||
## Wire format (version 4)
|
||
|
||
Requests are flat objects tagged by `command` (kebab-case):
|
||
|
||
```json
|
||
{"version": 4, "command": "meeting-record", "title": "Design sync"}
|
||
{"version": 4, "command": "settings-set", "key": "sound_effects", "value": "off"}
|
||
{"version": 4, "command": "dictation", "action": "start"}
|
||
{"version": 4, "command": "dictation", "action": "start", "print": true}
|
||
{"version": 4, "command": "dictation", "action": "stop", "print": true}
|
||
```
|
||
|
||
```ts
|
||
status | quit {force} | restart {force}
|
||
dictation {action, print?, raw?} action: start | stop | toggle | cancel;
|
||
print (start and stop) answers `transcript`
|
||
paste-last
|
||
typed-text
|
||
meeting-record {title?} | meeting-stop | meeting-copy-last | meeting-delete {id}
|
||
settings-get {key?} | settings-set {key, value}
|
||
model-list | model-use {model, language?} | model-install {model, language?} | model-cancel
|
||
window-show {pane?} | window-hide | window-story {story, variant?, play}
|
||
onboarding-reset
|
||
history-clear | check-for-updates | launch-at-login {enabled?}
|
||
vocabulary-list | vocabulary-add {term, sounds_like} | vocabulary-remove {term}
|
||
scrawl-capture {action} action: start | stop | cancel | status; `capture` experiment
|
||
scrawl-capture-list {limit?} | scrawl-capture-show {capture}
|
||
scrawl-capture-send {capture, to?, always?} | scrawl-capture-route {capture, ask_jev}
|
||
```
|
||
|
||
Optional fields may be left out. `hex start` and `hex logs` are local and have
|
||
no wire form. `history-clear` has no confirmation field: the CLI asks for
|
||
`--yes` before it sends it. `hex transcribe` also has no `/control` wire form:
|
||
it sends the WAV itself to `POST /transcriptions` (see Transport above).
|
||
|
||
Replies are tagged by `status`:
|
||
|
||
```json
|
||
{"version": 4, "status": "done", "message": "Recording “Design sync”; …", "changed": true}
|
||
{"version": 4, "status": "error", "code": "feature-off", "message": "Meetings are off; …"}
|
||
{"version": 4, "status": "settings", "settings": [{"key": "sound_effects", "value": true,
|
||
"kind": "bool", "writable": true, "about": "Play recording and error tones"}]}
|
||
{"version": 4, "status": "models", "models": [{"id": "parakeet_v3", "name": "…",
|
||
"languages": ["en", "…"], "size": "…", "installed": true, "selected": true,
|
||
"streams": true, "vocabulary": "bias", "supports_recognition_hints": false}]}
|
||
{"version": 4, "status": "vocabulary", "entries": [{"term": "OpenCode",
|
||
"sounds_like": ["open code"]}, {"term": "Rune"}]}
|
||
{"version": 4, "status": "scrawl-capture", "changed": true, "state": "idle",
|
||
"bundle": "…/captures/1792933491953.scrawl", "elapsed_ms": null,
|
||
"finishing": ["…/captures/1792933491953.scrawl"], "last": null}
|
||
{"version": 4, "status": "scrawl-captures", "captures": [{"id": "1792933491953.scrawl",
|
||
"bundle": "…", "created_at_ms": 1792933491953, "duration_ms": 4000, "mode": "window",
|
||
"app": "Brave", "words": 12, "sent_to": {"session": "ses_…", "title": "Hex"},
|
||
"queued": false}]}
|
||
{"version": 4, "status": "scrawl-capture-detail", "id": "…", "…": "the list's fields",
|
||
"window_title": "…", "transcript": "…", "note": null, "missing_transcript": null,
|
||
"marks": 2, "keyframes": 3, "video": "…/video.mp4", "audio": null,
|
||
"contact_sheet": "…/contact-sheet.jpg"}
|
||
{"version": 4, "status": "scrawl-capture-route", "explanation": "Capture …"}
|
||
{"version": 4, "status": "transcript", "text": "Ship it."}
|
||
```
|
||
|
||
`done.changed` is `false` when the command found things already so ("Already
|
||
recording", "The window isn't open"). Setting `kind` is one of `bool`,
|
||
`number`, `integer`, `choice` (with `choices`), `text`, `shortcut`, `path`, or
|
||
`json`; `value` is `null` for an unset optional setting.
|
||
|
||
### Status
|
||
|
||
```json
|
||
{
|
||
"version": 4, "status": "status",
|
||
"app_version": "2.3.1", "pid": 31708, "memory_footprint_bytes": 412000000,
|
||
"listener": {"running": true, "status": "Listening"},
|
||
"capture": {"state": "idle", "recording": null, "pending_jobs": 0},
|
||
"meeting": {"state": "recording", "title": "Design sync", "elapsed_seconds": 723},
|
||
"model": {"id": "parakeet_v3", "name": "…", "language": "en", "installed": true,
|
||
"preparing": null, "downloaded_bytes": 0, "error": null,
|
||
"backend": {"kind": "vulkan", "device": "AMD Radeon RX 9060 XT",
|
||
"crash_fallback": false},
|
||
"fallback": null},
|
||
"features": {"meetings": true, "release_microphone_while_idle": false,
|
||
"history_retention": "week"},
|
||
"permissions": {"ready": true, "microphone": "ready", "accessibility": "ready",
|
||
"transcription_model": true},
|
||
"window": {"open": false, "pane": null},
|
||
"update": "current",
|
||
"labs": ["live-text"]
|
||
}
|
||
```
|
||
|
||
```ts
|
||
listener.status Enabled | Waiting for setup | No microphone | Stopped (running false)
|
||
listener.shortcuts Wayland portal only: [{"name": "Dictation", "trigger": "Super+D"}], the keys the desktop
|
||
granted (changed in its settings or not); trigger null when it refused one
|
||
capture.state idle | recording | transcribing | unknown (the listener didn't answer or stopped)
|
||
capture.recording dictation | service | null
|
||
meeting.state idle | starting | recording | transcribing
|
||
model the saved selection; installed is about that model
|
||
model.backend {"kind", "device", "crash_fallback"} for the model actually loaded, or null before a
|
||
load and once it unloads; kind is cpu, vulkan, cuda, rocm, metal, or another backend's
|
||
name, device is the backend's own description (usually the processor name on the CPU), and
|
||
crash_fallback is true when the GPU crashed twice starting and this launch stays on
|
||
the CPU. macOS omits it (null)
|
||
model.fallback {"id", "name", "language", "reason": "missing" | "changed" | "labs-off"}, the
|
||
model Hex runs while the saved one can't, or null
|
||
permissions.* ready | needs-request | needs-settings | needs-restart
|
||
window.pane settings | modes | history | meetings | …
|
||
update current | checking | available | unavailable
|
||
```
|
||
|
||
`memory_footprint_bytes` is the physical footprint Activity Monitor shows.
|
||
`permissions.transcription_model` is about the model Hex runs, the fallback
|
||
included, so a missing import with a downloaded fallback is ready.
|
||
`labs` lists the [Labs](features/labs.md) features turned on; older Hex omits it.
|
||
|
||
### Error codes
|
||
|
||
```ts
|
||
not-running Hex isn't running (CLI only; exit 3)
|
||
outdated Linux: an older Hex an update left running; run `hex restart` (CLI only)
|
||
transport the request couldn't reach Hex or its reply was unreadable (CLI only)
|
||
version-mismatch this hex speaks another control version; use the running app's hex
|
||
unknown-setting no such key (the message suggests a near match)
|
||
read-only the key can't be set here (the message says where)
|
||
invalid-value the value doesn't parse or is out of range
|
||
confirmation-required clears history; pass --yes
|
||
shortcut-in-use another Hex shortcut already uses that binding
|
||
not-found no such capture (or it has no scrawl.json yet), or no meeting with that id
|
||
meeting-recording deleting a meeting that is still recording
|
||
meeting-transcribing deleting a meeting that is still transcribing
|
||
meeting-notes deleting a meeting while Hex writes its notes
|
||
meeting-final-pending deleting a meeting that is waiting for its final transcript
|
||
(Hex quit during it; the next launch finishes it)
|
||
unknown-model no such model id (`hex model list`)
|
||
model-unavailable an imported model that is missing, changed, or behind its Labs toggle,
|
||
or one `hex transcribe` named that isn't downloaded or ready
|
||
not-installed `model use` on a model that isn't downloaded (`hex model install`)
|
||
feature-off the command needs an opt-in that is off (Meetings)
|
||
disabled a Labs toggle the command needs is off (the message names it)
|
||
not-ready Hex is waiting for setup (permissions or the dictation model)
|
||
meeting-active quit/restart while a meeting records (pass --force)
|
||
busy something is in progress (recording, transcribing, a full queue),
|
||
or the listener couldn't confirm it's idle
|
||
resource-exhausted a running Hex refused a `hex transcribe` piece as too large or too long
|
||
discarded `stop` dropped the clip: no audio yet, or silent; nothing transcribes
|
||
no-speech `dictation start --print` or `stop --print` heard no speech, or `hex transcribe`'s file had none; nothing printed (exit 4)
|
||
not-recording `dictation stop --print` with nothing recording
|
||
unavailable a part of Hex the command needs isn't available
|
||
unsupported this build or OS can't do it (e.g. restart from a development binary)
|
||
failed Hex tried and failed (the message has the cause)
|
||
internal reading or saving settings failed
|
||
```
|
||
|
||
## Settings
|
||
|
||
Keys are dotted paths into `settings.json`. `hex settings list` prints every
|
||
key with its kind and description; `hex settings list --json` adds `choices`.
|
||
Changes go through the same rules as the window and are saved by the app,
|
||
which then reloads them live; an open window adopts the change unless it has
|
||
an unsaved edit.
|
||
|
||
On macOS, Windows, and Linux on X11, `keyboard_model_switching.enabled` opts
|
||
into switching the shared speech-model picker on keyboard input-source changes.
|
||
The JSON setting `keyboard_model_switching.mappings` maps enabled input-source
|
||
IDs to a model and a language it supports, for example
|
||
`{"com.apple.keylayout.Spanish":{"model":"parakeet_v3","language":"es"}}`.
|
||
Source IDs are macOS input-source IDs, Windows keyboard layout handles
|
||
(`04190419` for Russian), or X11 XKB layouts (`ru`, `de(neo)`). Omit a source
|
||
for "Don't switch"; Apple Speech can't be mapped. Automatic switches load
|
||
installed models only; they never download them. Enabling or editing the active
|
||
mapping applies it; launching Hex does not reapply it. Wayland has no general
|
||
protocol for the active layout, so there Hex keeps the settings but does not
|
||
observe input-source changes or show these controls.
|
||
|
||
On macOS and Windows, `labs.app_model_switching` opts into switching the
|
||
shared speech-model picker per dictation mode. Each mode, Global included, may
|
||
carry `speech`, a model and a language it supports, for example
|
||
`{"model":"parakeet_v3","language":"es"}` in
|
||
`dictation_processing.default_mode` or `dictation_processing.modes[]`; omit it
|
||
for "Don't switch". When the app in front changes, Hex matches the mode by
|
||
application, waits 300 ms, then prepares its choice through the same
|
||
installed-only path as a manual pick; it never downloads a model and a
|
||
recording holds a due switch until it is idle. Matching is by application on
|
||
both macOS and Windows, and a mode's website rules are ignored, so a mode
|
||
chosen only by website never switches. Hex's own window, and a moment with no
|
||
app in front, are ignored: they neither schedule nor cancel a switch, so
|
||
enabling the lab while Hex is in front applies nothing until you next focus
|
||
another app. Editing a mode's choice applies when you next focus an app whose
|
||
mode carries it. With keyboard switching also on, the switch that is due
|
||
later wins, and an app switch wins a tie in the same tick. Linux does not
|
||
support this lab.
|
||
|
||
- Booleans take `on`/`off`, `true`/`false`, `yes`/`no`, or `1`/`0`.
|
||
- Optional settings take `none` to clear them.
|
||
- `microphone` takes a device name exactly as Settings lists it, or
|
||
`system-default` (or `none`) to follow the OS default input.
|
||
- `avoid_bluetooth_microphone` (default on) makes System default record from
|
||
the built-in microphone, else another wired or USB input, when the OS
|
||
default is a Bluetooth headset, so its audio stays out of call quality (on
|
||
Linux, Hex moves its PipeWire stream off a Bluetooth default source after
|
||
opening it, so the headset may still switch to hands-free mode; see
|
||
[Bluetooth headsets](linux.md#bluetooth-headsets)). A saved microphone and
|
||
`--device` are always used as named.
|
||
- Shortcuts take text such as `option`, `right-command`, `option+shift+v`,
|
||
`control+space`, `fn`, `f5`, or `left-option+right-option` (both Options
|
||
held together). `left-` and `right-` apply to shortcuts without a key. On macOS, Windows and X11, `mouse4` and `option+mouse4`
|
||
also work (Mouse 1/2/3 are left/right/middle). Key names resolve on the active keyboard
|
||
layout, the OS's rules for Settings' recorder apply, and a binding another
|
||
Hex shortcut uses is refused with `shortcut-in-use`. `double_tap_only`
|
||
needs a key chord, as in Settings.
|
||
- On Linux with `labs.standalone_special_keys` on (`hex labs on
|
||
special-keys`, off by default), a shortcut may use `menu`, `pause`, or
|
||
`scrolllock`, alone or with modifiers, for example
|
||
`hex settings set dictation_hotkey menu` after remapping Caps Lock to Menu.
|
||
Off, they are refused with `invalid-value`, and turning the toggle off puts
|
||
a shortcut on one back to its default. Elsewhere one alone still needs a
|
||
modifier.
|
||
- `second_dictation_hotkey` (unbound by default; `none` clears it) is a
|
||
second shortcut that dictates exactly as `dictation_hotkey` does, such as a
|
||
key on an external keyboard. It acts only while
|
||
`labs.second_dictation_shortcut` is on (`hex labs on
|
||
second-dictation-shortcut`) and shares no key with another shortcut. Linux
|
||
and Windows need a key in it, as every shortcut there does.
|
||
- `extra_dictation_hotkeys` (macOS, an empty array by default) holds up to
|
||
four more dictation shortcuts, each `{"binding": ..., "mode": ...}`. A row
|
||
acts only while `labs.more_dictation_shortcuts` is on (`hex labs on
|
||
more-dictation-shortcuts`) and shares no key with another active shortcut.
|
||
`mode` is `hold_or_double_tap` (the default), `hold`, `tap`, or `double_tap`;
|
||
`double_tap` needs a binding with a key and otherwise reads as
|
||
`hold_or_double_tap`, except that Labs › Double-tap Option, Command, or Globe also
|
||
allows a binding without one. Turning the Lab off keeps the rows saved.
|
||
- `json` settings (modes, extra dictation shortcuts, meeting providers,
|
||
excluded calendars) take the JSON value the file stores.
|
||
- `ui_language` takes `en`, `pl`, `zh` (Simplified Chinese), `zh-Hant`
|
||
(Traditional Chinese), `ja`, `de`, `es`, `uk`, `pt`, `fr`, `ru`, or `none`
|
||
(follow the system language), the same languages Settings offers. Following
|
||
the system, a Chinese locale with the Hant script or the TW, HK, or MO
|
||
region shows Traditional Chinese; other Chinese locales show Simplified.
|
||
- `updates.channel` takes `stable`, `dev`, or `none` (follow the build: the
|
||
channel it was published on), and checks for updates at once. Choosing
|
||
another channel than the build's offers that channel's current release
|
||
("Switch to Stable 2.3.1" in the window); Hex installs it only once you
|
||
confirm there. Versions are never compared.
|
||
- With the `capture` experiment, `capture.*` keys appear:
|
||
`capture.enabled`, `capture.hotkey` (default `option+shift+s`),
|
||
`capture.list_hotkey` (the capsule's send-to list, default
|
||
`option+shift+v`), `capture.transcriber`, `capture.folder`,
|
||
`capture.media_days`, `capture.bundle_days`, `capture.routing`,
|
||
`capture.show_overlays_in_recordings` (default off; on lets screen
|
||
recordings, Hex's captures included, show its overlays), and
|
||
`capture.capsule_flow` (`list`, the default, `toast`, `menu_bar`, `cursor`
|
||
or `notch`: the capsule's flow after a capture, prototypes to compare). Both
|
||
capture shortcuts need a key, and `capture.enabled on` needs both free; on
|
||
a Mac whose Paste Last is still ⌥⇧V, rebind one of the two first.
|
||
- `meetings.retention` is `day`, `week`, `month`, `forever`, or `none` (the
|
||
default: meetings recorded since the default began are kept 24 hours, and
|
||
older ones stay until you choose). Hex moves meetings that ended longer
|
||
ago than that to the Trash at launch and hourly, Meetings on or off, and
|
||
`hex meetings` and `hex mcp` no longer list them.
|
||
- `meetings.audio_retention` is `day`, `week`, `month`, `forever`, or `none`
|
||
(the default; Meetings › Settings › After a meeting › Keep audio). A
|
||
finished meeting's WAVs move to the Trash on that window while its
|
||
transcript, notes, summary, and Markdown stay; `none` follows
|
||
`meetings.retention`.
|
||
- `transcription.model` and `transcription.language` are read-only here: use
|
||
`hex model use`. `transcription_recents` and `meetings.calendar.connected`
|
||
are read-only (connecting a calendar asks macOS for access from Settings).
|
||
|
||
`recording_audio_behavior` also accepts `duck` (Lower audio): macOS and Linux
|
||
lower the default output, Windows lowers other apps' audio sessions.
|
||
`recording_duck_volume` sets the fraction of the original output volume (0–1,
|
||
default 0.2), applied on the next intentional capture when ducking is selected.
|
||
|
||
Writable keys: `release_microphone_while_idle`,
|
||
`sound_effects`, `sound_effect_volume`, `sound_theme`, `microphone`,
|
||
`avoid_bluetooth_microphone`, `recording_audio_behavior`, `recording_duck_volume`, `double_tap_lock`, `double_tap_only`,
|
||
`shortcut_mode`, `dictation_hotkey`, `paste_last_hotkey`, `raw_dictation_hotkey`,
|
||
`second_dictation_hotkey`, `extra_dictation_hotkeys`, `labs.second_dictation_shortcut`,
|
||
`labs.more_dictation_shortcuts`, `labs.standalone_special_keys`, `labs.app_model_switching`, `show_dock_icon`, `vocabulary_boost`,
|
||
`labs.keep_microphone_open`, `labs.keep_microphone_open_seconds`,
|
||
`labs.hud_height`, `labs.hud_bottom_gap`,
|
||
`dictation_processing.default_mode`, `dictation_processing.modes`,
|
||
`history_retention`, `meetings.enabled`,
|
||
`meetings.output_folder`, `meetings.summaries`, `meetings.opencode_model`,
|
||
`meetings.show_recording_indicator`, `meetings.recording_indicator_corner`,
|
||
`meetings.retention`, `meetings.audio_retention`, `meetings.providers`, `meetings.calendar.excluded_calendars`,
|
||
`meetings.calendar.remind`, `meetings.calendar.lead_minutes`,
|
||
`experimental.hex_microphone`, `experimental.hex_microphone_mode`,
|
||
`experimental.hex_microphone_comfort_noise`,
|
||
`experimental.hex_microphone_strength`, `enable_on_launch`,
|
||
`paste_with_shift`, `ui_language`, `indicator_position`, `updates.channel`.
|
||
|
||
`transcription.recognition_hints` is deprecated: recognition hints merged into
|
||
the [vocabulary](features/vocabulary.md). `get` returns the vocabulary's terms,
|
||
comma-separated, and `set` (or `apply`) adds the comma-separated names to it;
|
||
`list` leaves the key out. Model listings keep `supports_recognition_hints`
|
||
(true where the model reads the vocabulary as its prompt) beside the newer
|
||
`vocabulary`: `prompt`, `bias`, or `none`.
|
||
|
||
`meetings.summaries` chooses who titles and summarizes meetings: `off`, `mac`
|
||
(Apple Intelligence; nothing leaves this Mac, and nothing falls back to
|
||
OpenCode), or `opencode` (sends the transcript to `meetings.opencode_model`,
|
||
or OpenCode's own default model when that is `none`, never another provider).
|
||
Off a Mac it is `off` or `opencode`. The retired
|
||
`meetings.summarize_with_apple_intelligence` and
|
||
`meetings.summarize_with_opencode` are refused with the `meetings.summaries`
|
||
command to use instead.
|
||
|
||
A test (`control::settings::tests::every_persisted_setting_has_a_cli_key`)
|
||
fails when a new persisted setting has no key here.
|
||
|
||
## Compatibility
|
||
|
||
`CONTROL_VERSION` is 4. A change that breaks a request or reply shape
|
||
increments it; adding a command, an optional request field, or a reply field
|
||
does not. A `hex` from a different version gets `version-mismatch`; use the
|
||
`hex` inside the running Hex.app, or restart Hex after updating. Version 2
|
||
removed `permissions.input_monitoring` from the status reply; version 3 removed
|
||
`voice-action` and `features.voice_action`; version 4 removed `commands-retry`,
|
||
`settings-set.confirm`, `window-show` `commands`, `capture.recording` `voice`,
|
||
`features.commands`, and `features.command_model`.
|