Three keys, because a wake word is not reliable enough

There are three different things a person does by voice at a machine, and they are genuinely hard to tell apart in real time.

The first is plain dictation: prose, to go in a document. The second is spelling, or entering a symbol - and the ambiguity there is real, because if I say "dollar" it is not obvious whether I want the word typed or the $ character. The third is a command aimed at the operating system rather than at the document: switch screen, move down five lines.

Recognition here is always triggered by hand. There are no wake words, because they are not reliable enough, and that constraint turns out to be an advantage: if a person is already pressing something to talk, they can press a different something for each of the three jobs. The mode is declared rather than guessed at.

The three keys

Caps Lock records, then transcribes the whole recording. AltGr streams, decoding as the audio arrives, and its output goes through the grammar. Print Screen is the new one: hold it, say one symbol or one editing instruction, release.

The Caps Lock lamp is lit for the duration of a Print Screen hold and returns to whatever it was before on release. All three modes share the one microphone and cannot record at the same time, so a press of one while another is listening is refused rather than queued.

A closed vocabulary, and what that buys

Print Screen accepts a closed vocabulary and never passes an unrecognised phrase through as prose or as an application call. That refusal is the whole design. A symbol mode that falls back to typing what it thought it heard is a mode that occasionally types a sentence into the middle of a document.

The characters are digits zero or nought through nine, A to Z as single letter names or NATO phonetics - Alfa and Alpha both, Juliett and Juliet, X ray - and a full punctuation set: brackets of all three kinds, slash and backslash, hyphen and minus and dash, pling as well as exclamation mark, pound sterling, dollar, caret, tilde, pipe, backtick, euro. A result consisting of one literal punctuation mark is accepted too.

Movement is the other half: left, right, up, down, next and previous word, the starts and ends of line, sentence, paragraph and document, page up and down, new line, new paragraph, tab, space, backspace. Go to, move to, jump to and cursor to can precede a boundary. Counted movement takes words, lines, characters, sentences or paragraphs, so forward five words and left 4 characters both work, with counts limited to one to twenty, and a counted phrase is one operation rather than a repeat.

Undo reverses the last symbol-mode operation, up to thirty-two of them in sequence, and one undo reverses an entire counted movement. The window server records the field text, caret, selection anchor and goal column before it acts. If something else has changed the field in the meantime, the restore is refused rather than overwriting that work.

Guiding a general recogniser, without forcing it

The first implementation reuses the streaming decoder's own weights and biases them towards the answer. Print Screen selects a separate recognition profile that favours token sequences capable of completing a symbol or movement phrase.

The guard is a threshold: the best allowed token has to be within 2.5 logits of the model's free choice. Beyond that, decoding stays free and the result is normally thrown out by the symbol parser. That matters because the alternative - always steering towards the nearest legal phrase - turns arbitrary speech into a confident action, which is the worst failure this mode can have. The parser remains the authority on whether anything runs, and the guidance alone never dispatches a command.

The kernel log says how many token choices the guidance actually changed in each recording, so the profile can be measured on hardware rather than judged by how it feels.

A recogniser of its own

The second implementation does not use Whisper at all on this path, and it is a smaller idea than it sounds.

A closed vocabulary of short phrases does not need a general speech model. The microphone sends 16 kHz blocks to the speech core while the key is held; every 200 ms the core compares the current prefix against phrase templates and publishes an accepted candidate as tentative text, with a final candidate on release. There is no post-recording pass.

The features are 25 ms Hamming windows, 10 ms hops, 40 mel bands and twelve normalised cepstral coefficients, matched with duration-banded dynamic time warping. The model holds quantised features and labels rather than recordings, which is about 364 KiB for the 360 clips it currently has. The guided Whisper stream stays as the fallback for when that file is absent or invalid, and AltGr is untouched by any of it.

The training audio and the derived model are deliberately outside the repository. They are recordings of one person's voice in one house, which is private and also specific enough to be worthless to anybody else.

Teaching it

Every held-key sample is kept, including the failures, which are the useful ones. Each row carries the 16 kHz mono WAV bytes, which mode was used, the words that were decoded, and a writable field for a human transcript of what was actually said. Timing and peak level sit beside them.

A failed recognition with a correct transcript next to it is a training example. A failed recognition that was discarded is an anecdote.

What is not built

Low confidence gives no command, and unrelated speech has not yet been tested against that gate. That is the obvious attack on this design and it is untested.

The held key and the lamp were tested on the X1 without a panic, but on a build that predates the guided profile - so the recognition accuracy of the thing described above still needs a key-and-voice trial on the laptop, and nobody has sat down and done one.

A USB keyboard's Caps Lock lamp has no output-report driver, so the light only works on the built-in PS/2 keyboard. The store loads collection binary fields into memory rather than lazily, so the live sample collection is capped at 256 entries until that changes. And the installer does not yet distribute the optional template model, so it has to be put on the EFI partition by hand.