7.4 KiB
| name | description | license |
|---|---|---|
| hex-annotate | Read Hex Annotate screen captures, where someone recorded their screen while talking, clicking, and drawing red marks on it. Use whenever a message names or attaches a .scrawl bundle, a capture’s summary.md, contact-sheet.png, or frames/, when the user says "look at my capture" or refers to something they just recorded or drew, or when you need `hex annotate` to inspect a capture, list its marks, zoom into a moment, or pull frames. It also applies when a message starts with a `[hex-annotate …]` header, or the older `[scrawl …]` header from earlier versions. | MIT |
Hex captures
Hex's Annotate records someone's screen, their voice, clicks, and red ink, and
packs the result into a .scrawl folder (a bundle). They sent it to show you a
problem instead of describing it, so the capture is the bug report. Read it
closely before you answer, and don't ask them to explain again what the capture
already shows. If something is unclear, look closer with hex annotate zoom first.
The Labs switch
hex annotate inspect, zoom, frames, and skill are behind Labs › Agent
tools for captures (labs.annotate_agent_tools), off by default. While it is
off they refuse and say how to turn it on. Turn it on with:
hex labs on annotate-agent-tools
or in Settings › Labs, under Annotate ("Agent tools for captures"). It writes
settings.json, so a running Hex takes it at once.
Finding hex
command -v hex finds it when it is on PATH. On macOS, if PATH has none, run
/Applications/Hex.app/Contents/MacOS/hex (or the same path under
~/Applications), or symlink it into ~/.local/bin. These four commands
work with Hex closed: they read the bundle, decode its video, and write only
into its zoom/ folder.
Read the bundle first
contact-sheet.pngshows every keyframe in time order, left to right and top to bottom, each with its numberNin a small chip in a corner (the#Ninsummary.md). Open it, or the individual images inframes/(frame-03.pngis #3), before changing anything: the marks and words only make sense on the images, and an image often shows the problem itself.summary.mdnames what was recorded, its length, and one line per keyframe:#3 (0:10) [clicked "New tab"] "…".⚑marks a moment the person flagged on purpose, and⟨Name⟩a new screen an app reported.scrawl.jsonis the full manifest: keyframes,markswith shape and box,events,transcript.wordswith times, the appcontext, andautocrop. Speech recognition mishears names and code terms, so trust the screen when the two disagree.recording.mp4orrecording.mov, andaudio.wav, are the source media. A.media-pendingfile means the video is still being copied from another Mac.
Markers at a glance: red strokes, circles, underlines, and arrows are the
person's marks and point at the problem; a ripple is a click, a finger trail a
swipe, and a tapering red trail the laser pointer. Coordinates in summary.md
are fractions of the captured area from the top left (x 0.51 is just right of
centre). When several marks sit on one screen the text numbers them in drawing
order (✎① 0:06 …, ✎② 0:09 …); the image shows only the ink, so use
zoom --mark to see which ink is which.
Answer with the moments named the way the summary names them, as in "#3 (0:10)", so the person can find them.
hex annotate
inspect, zoom, and frames take the bundle as their first argument: a
bundle name (1792933491953.scrawl, extension optional), a path, or latest
(the default) for the newest capture in Hex's captures folder. skill takes
no argument.
hex annotate inspect <capture> # summary.md
hex annotate inspect <capture> --marks # one line per mark
hex annotate inspect <capture> --json # scrawl.json, pretty
hex annotate zoom <capture> --at 0:21 # a sheet of frames around a moment
hex annotate zoom <capture> --at "the width changes" # …or a phrase from it
hex annotate zoom <capture> --mark 2 # …or one mark, drawn whole
hex annotate zoom <capture> --mark 1 --frame 4 # ✎① of keyframe #4
hex annotate frames <capture> --from 0:04 --to 0:06 --fps 4
hex annotate skill # this guide, from the installed Hex
inspect --marks
Lists every mark as #2 ✎② 0:09 marked (2 strokes) · m2: the keyframe, the
number the summary shows, the time, the shape and what it was drawn on, and the
mark's id. --marks --json gives the same as JSON, for picking one out
yourself. Use it before zoom --mark when the summary's numbering is
ambiguous.
zoom
Use zoom when the problem is motion (an animation, a jump, a flicker) or
happens between keyframes, or when you can't tell what a mark points at. It
writes a dense contact sheet of frames around the moment into
<bundle>/zoom/zoom-<from>-<to>.png and prints:
…/captures/1792933491953.scrawl/zoom/zoom-9500-11500.png
0:09.500 → 0:11.500, 12 frames: #1 0:09.500, #2 0:09.681, #3 0:09.863, …
Said: “but the width changes when the panel opens”
--mark prints the mark's line first, then the same:
#1 ✎② 0:01 marked “the width” · m2
…/captures/1792933491953.scrawl/zoom/zoom-300-2300.png
0:00.300 → 0:02.300, 12 frames: #1 0:00.300, #2 0:00.481, #3 0:00.663, …
Said: “it blue”
Open that PNG and look at it before you act on the feedback. The cells are
numbered from #1 within the sheet, so they are not keyframe numbers.
--span is the seconds the sheet covers (default 2, a quarter of it before
the moment and the rest after; for a mark, at least the whole mark, centred),
--frames the number of cells (default 12, at most 48), and --full shows
the whole screen instead of the bundle's auto-crop. --at takes a time
(0:21, 21.5, 21) or a phrase from the narration; --mark takes a
mark's number from the summary or its id (m2), and --frame says which
keyframe's ✎① the number counts when several have one.
frames
Extracts one plain PNG per frame across a time range, at --fps frames a
second (default 4, 1 to 240), into
<bundle>/zoom/range-<from>-<to>/frame-001.png and so on, and prints each
path. Use it when you need stills at exact times, or when a zoom sheet's cells
are too small to read. The range must not end before it starts, --to is
clamped to the recording's length, and one call writes at most 240 frames;
narrow the range or lower --fps. Without a video, or over an empty range, it
prints the bundle's existing keyframe PNGs in frames/ that fall in the range
instead.
Leaving the bundle as it is
zoom and frames write only into <bundle>/zoom/; repeating a command
there replaces the frames it generated. Leave every other file and folder in
the bundle alone: Hex and Scrawl manage them, and retention removes the video
and audio after a while.
When the CLI isn't there
If hex is missing, is an older build without these commands, or the Labs
switch can't be turned on, read the bundle directly: contact-sheet.png,
frames/frame-NN.png, summary.md, and scrawl.json. That covers every
keyframe at full size; only zoom and frames add anything, and they need
the video. After retention has removed recording.mp4 and audio.wav, those
two commands can't decode anything, but the sheet, keyframes, and manifest stay.
Install this skill with
npx skills add anomalyco/hex --skill hex-annotate.