Voice Input

Overview

Voice Input lets you dictate into any agent's hybrid input bar and into the Daintree Assistant. Microphone audio streams to a transcription service over an encrypted connection, and text lands in the input bar as you speak. An optional AI correction pass cleans the dictated passage up after you stop, fixing punctuation, filler words, and technical spellings.

Two things shape how well it works, and both are covered below: which transcription provider you pick, and how much project context Daintree can hand the transcriber to bias recognition toward the words you actually use.

Choosing a Transcription Provider

Transcription sits behind a provider abstraction, and the provider is yours to pick in Settings > Voice Input > Transcription provider. The choice decides which API key you need and how speech is segmented into utterances.

On the OpenAI path, Daintree sends turn_detection: null so the server never auto-commits, and drives segmentation itself with a Silero v5 voice-activity detector running on a worker thread. It commits at the real end of speech after a short holdover, clears the server's buffer on speech onset so accumulated silence isn't transcribed, and replays roughly 300 ms of pre-roll audio so clearing that buffer never clips the start of the utterance it just detected. A backstop forces a commit if speech runs past eight seconds with no detected pause, which is also the only commit cadence if the detector fails to start.

That replaced a blind two-second commit timer, which cut words mid-pause and added up to two seconds of latency at end of speech.

What leaves your machine

The settings panel carries a data-flow note that changes with the provider you select. Audio is streamed over an encrypted connection to the provider you chose, using your own API key.

• OpenAI: audio is not used for model training. OpenAI may retain audio in abuse-monitoring logs for up to 30 days.
• Deepgram: Deepgram does not retain streaming audio or transcripts by default.

Keyterm biasing sends more than audio: your custom dictionary, project name, current branch tokens, and identifiers scraped from recent terminal output all travel with the session. That is what makes recognition accurate on your codebase, and it is worth knowing before you enable it on a sensitive repository. See Security for how Daintree treats keys and outbound data generally.

Setup

Enable Voice Input

1. Open Settings > Voice Input and toggle Voice input on.
2. Grant microphone permission if you haven't already (see below).
3. Choose your transcription provider.
4. Pick a microphone, or leave it on System default.
5. Enter the API key for the provider you chose. The OpenAI key is validated when you save.
6. Choose your language: English, Spanish, French, German, Japanese, Chinese, Korean, Portuguese, Italian, or Russian.
7. Choose a paragraph break strategy and a recording mode.

Dictation becomes available once the feature is enabled and the selected provider has its key. Those two conditions are the whole gate: microphone permission is not part of it, so the mic button can appear before the OS has granted access.

Grant Microphone Permission

Daintree needs microphone access from the operating system before it can capture audio. The settings panel shows the current permission status and gives instructions for your OS.

Select a Microphone

The Microphone dropdown defaults to System default, which follows whatever input device your OS is using. Pick a specific device to pin dictation to it regardless of the system default. The list comes from the OS and refreshes when devices change, so plugging in a headset mid-session adds it. A refresh button beside the dropdown re-enumerates inputs manually.

Starting and Stopping Dictation

There are four routes into dictation:

• Mic button. Every agent panel has one in its input bar. It always behaves as a toggle regardless of the recording mode below, since a click has no key-release to end. Clicking it while paused resumes rather than stops.
• Shift-Command-V ⇧⌘V Control-Shift-V Ctrl+ Shift+ V Control-Shift-V Ctrl+ Shift+ V . Dictates into the Daintree Assistant when the Assistant's input owns focus, and into the focused agent panel otherwise.
• Option-Shift-Command-V ⌥⇧⌘V Control-Alt-Shift-V Ctrl+ Alt+ Shift+ V Control-Alt-Shift-V Ctrl+ Alt+ Shift+ V . Dictates into the Daintree Assistant from anywhere, opening it if it's closed.
• Action palette. Search for "Toggle Voice Dictation".

Toggle or push-to-talk

Recording mode in settings decides what the shortcut does:

• Toggle (default): press to start, press again to stop.
• Push to talk: hold the shortcut to record. Releasing the key stops recording without submitting.

Push-to-talk has three safety nets, because a held key is easy to lose track of: releasing the trigger key stops the session; releasing Cmd also stops it, since macOS swallows the trigger keyup while a modifier is still down; and the window losing focus stops it, since Chromium sends no synthetic keyup on a Cmd+Tab mid-press. The mic button and the palette keep working as toggles in this mode, because neither produces a keyup.

Pause, resume, and cancel

Control-Shift-Space ⌃⇧Space Control-Shift-Space Ctrl+ Shift+ Space Control-Shift-Space Ctrl+ Shift+ Space pauses and resumes. A paused session holds its connection open and stops forwarding audio, so resuming is instant, but a pause that runs past 60 seconds auto-stops the session rather than holding a socket open indefinitely.

Escape stops an active session and keeps everything dictated so far. Enter in an agent input bar stops the session and submits (see below).

Toolbar indicator

While a session is live, a mic icon appears in the global toolbar with an orbit ring and the elapsed time in M:SS. Its tooltip names the project and worktree being dictated into, and tells you which shortcut stops or resumes it. Clicking the indicator focuses the panel where recording is happening; it doesn't stop the session.

Session States

A dictation session moves through eight phases. Most are invisible in the happy path, but the mic button and toolbar indicator distinguish all of them, and knowing the set makes an unexpected state legible.

Where Dictation Lands

By default dictation follows focus: the Assistant input if it owns focus, otherwise the focused agent panel. Only panels with a real terminal behind them can receive dictation: a browser or dev-preview panel has nowhere for a transcript to go.

Two controls override that, both on a terminal panel's right-click menu:

• Lock dictation to this panel pins the target, so changing focus mid-thought doesn't redirect your next utterance. Unlock dictation releases it. A lock whose panel has since closed is cleared automatically and dictation falls back to focus.
• Recent dictation targets lists the last few panels you dictated into and starts a session on one directly. The list survives a restart; the panel identity does not, so entries that no longer resolve are dropped rather than shown.

Text as It Arrives

Interim transcription (the in-progress guess that keeps changing as you speak) is rendered as a ghost widget at the end of the line rather than inserted into the document. That distinction matters: text that was never inserted can't be undone, so a session's worth of interim churn leaves your undo history intact. The ghost is suppressed during IME composition so it can't displace a composition overlay.

Finalized segments are written into the input bar for real. If a session produces more than one paragraph, each lands as its own insertion with the break between them.

Submitting while recording

Pressing Enter while recording stops the session, syncs the settled transcript into the editor, and sends. The input bar goes read-only with a spinner for the moment that takes. AI correction is deliberately not waited on: it is fire-and-forget after a graceful stop, so Enter never blocks on a model call. Press Escape to cancel the submit and keep the text for editing.

Paragraph Breaks

Spoken commands (default)

With the spoken-command strategy selected, formatting phrases you say while dictating are rewritten in post-processing and stripped from the transcript:

Only a trailing chain of commands at the end of an utterance is converted, so "I'll add a new paragraph here" stays literal and you can still talk about the things themselves. Chains work: "hello comma new paragraph" becomes a comma followed by a blank line.

Manual Enter only

Paragraph breaks come from Enter, and spoken formatting commands are disabled entirely. Choose this if you dictate in a non-English language, or if you want the phrases to stay literal.

Context Biasing

Generic speech models mishear codebase vocabulary, and no amount of post-correction fully recovers a word the transcriber never considered. Daintree biases the transcriber up front instead, assembling a keyterm list from four tiers of project context and sending it with the session: as literal keywords plus a mirrored prompt on the OpenAI path, and as Nova-3 keyterms on the Deepgram path.

The list is capped at 50 terms and 100 characters per term, with the terminal-derived tier capped at 30 so the higher tiers always keep headroom. Common shell commands and language keywords are blocklisted: biasing toward "const" or "grep" would cost accuracy rather than add it. The branch and terminal reads each race a 500 ms timeout, so a slow git call delays the start of dictation by at most half a second and then proceeds without that tier.

Custom Dictionary

The dictionary holds up to 100 domain-specific terms: product names, internal package names, unusual abbreviations. Add them in Settings > Voice Input; each appears as a removable pill.

Dictionary terms do double duty. They are the top-priority tier of the keyterm bias above, and they are also given to the AI correction pass as preferred spellings. The correction prompt asks for them as preferences, not substitutions: when both the transcribed word and a dictionary term are plausible, the transcribed wording wins. Forcing them in produced worse output than leaving a near-miss alone.

Learning from corrections

Learn words from corrections (on by default) watches the edits you make to dictated text before sending, and proposes the words you keep fixing as dictionary terms. Suggestions appear as pills under the dictionary with an Add and a dismiss control; hovering one shows what was heard instead. Nothing is added without you accepting it.

AI Text Correction

With correction enabled, Daintree sends the whole dictated passage for a cleanup pass once the session stops, then swaps the corrected version into the input bar in place. It fixes punctuation, filler words, technical spellings, and homophones.

One model backs every voice AI call (whole-passage correction, file-command detection, and ambiguous-file reranking) at the lowest reasoning tier, because these are fast, strictly-shaped tasks. There is no correction-model dropdown. The earlier tiered choice was retired and existing settings were migrated to the current model on upgrade.

While the request is in flight, the dictated range carries a green dotted underline. When the response lands, the corrected text replaces it. Edit that text mid-flight and Daintree keeps your version rather than overwriting it.

The pass runs after a graceful stop with correction enabled and text to correct. It is skipped when the session ended on an error, and when you stop one recording only to immediately start another: there is no settled passage to correct in either case.

Custom instructions

The Custom Instructions textarea appends project-specific rules to the core correction prompt: "always capitalize ProductName as one word", "React component names use PascalCase". They sit below the built-in correction rules in priority. Inspect core prompt shows the full base prompt read-only, so you can see what your instructions are being added to. Your project name, repository directory name, and dictionary are injected automatically.

File reference resolution

With correction on, Daintree detects spoken file references and resolves them into @file links. Say things like "link to the auth helper", "at file the button component", "reference the user model", "add file the config service", or "open the main layout". The description is matched against the project file tree and the best candidate is picked, then rendered as a clickable file chip.

If resolution fails, the text falls back to @?description so you can see what was being looked up and fix it by hand. Toggle the behavior with Resolve file references; it is on by default once an OpenAI key is configured.

Settings Reference

Everything lives in Settings > Voice Input.

Speech-to-text

AI text correction

This section appears once Voice Input is enabled; its options appear once an OpenAI key is set.

Keyboard Shortcuts

Two more keys act on a live session without being bindings of their own: in an agent input bar Enter stops the session and submits, and Escape stops dictation, or cancels a submit already in progress.

All three voice bindings are rebindable. See Keyboard Shortcuts for the full reference.

Troubleshooting

The mic button isn't there

The button is hidden until Voice Input is configured, which means both of: the feature is enabled in Settings > Voice Input, and the selected provider has its key. A Deepgram provider with only an OpenAI key saved counts as unconfigured.

The session keeps saying "Reconnecting"

Transient faults (a rate limit, a dropped socket) don't end the session. Daintree retries with backoff while buffering audio, and the mic button says so rather than raising an alarm. After several failed attempts it gives up with "Connection failed after several retries", which is fatal and ends the session.

"Connection timed out"

The transcription socket didn't open within 10 seconds. Check your connection, that the key for the selected provider is valid, and that the account has credit.

"Rate limit exceeded: check your API key quota"

The provider rejected the session outright rather than throttling it. This is the fatal variant; the transient one reads "Rate limited, reconnecting" and recovers on its own.

Spoken paragraph commands aren't working

Check that Paragraph Breaks is set to Spoken commands and the language is English. Remember that only a trailing chain at the end of an utterance converts: a command in the middle of a sentence stays literal by design.

AI correction isn't running

Correction needs to be toggled on, an OpenAI key saved, and dictated text present at stop. It is skipped when the session ended on an error, and when you stop one recording and immediately start another.

Recognition keeps missing the same word

Add it to the custom dictionary. It becomes a top-priority keyterm on the next session, which biases the transcriber before the mistake happens, a stronger fix than leaving it to the correction pass. If Learn words from corrections is on, fixing the word by hand a few times will surface it as a suggestion anyway.

File references show as @?

@?description means resolution found no close match in the project file tree, or the description was too vague to disambiguate. Be more specific when you dictate the reference.