Voice Input
Dictate into agent and Assistant input with OpenAI or Deepgram, project keyterms, microphone readiness cues, ordered transcripts, and optional OpenAI text correction and file references.
Overview
Voice Input lets you dictate into any agent's hybrid input bar and into the Daintree Assistant. Microphone audio streams to a transcription service over an encrypted connection, and text lands in the input bar as you speak. An optional AI correction pass cleans the dictated passage up after you stop, fixing punctuation, filler words, and technical spellings.
Two things shape how well it works, and both are covered below: which transcription provider you pick, and how much project context Daintree can hand the transcriber to bias recognition toward the words you actually use.
Choosing a Transcription Provider
Transcription sits behind a provider abstraction, and the provider is yours to pick in Settings > Voice input > Speech-to-text > Transcription provider. The choice decides which API key you need and how speech is segmented into utterances.
| OpenAI (default) | Deepgram | |
|---|---|---|
| Model | gpt-live-transcribe, over the Realtime transcription endpoint | Nova-3, over the streaming endpoint |
| Key | OpenAI API key (sk-...) | Deepgram API key |
| Turn detection | Client-side. Daintree runs its own voice-activity detection and commits at end of speech. | Server-side. A 300 ms pause finalizes the current segment. |
| Language | The chosen language, or no language at all for Auto-detect | The chosen language, or multi for Auto-detect |
| Keyterm biasing | Sent as literal keywords plus a mirrored prompt | Sent as Nova-3 keyterms |
| Spoken-command formatting | Applied in post-processing (see Paragraph Breaks) | Applied the same way |
On the OpenAI path, Daintree sends turn_detection: null so the server never auto-commits, and drives segmentation itself with a Silero v5 voice-activity detector running in a separate utility process, so a native crash in the detector ends only that process. It commits at the real end of speech after a short holdover, clears the server's buffer on speech onset so accumulated silence isn't transcribed, and replays roughly 300 ms of pre-roll audio so clearing that buffer never clips the start of the utterance it just detected. A backstop forces a commit if speech runs past eight seconds with no detected pause, which is also the only commit cadence if the detector fails to start or stops, so dictation keeps working either way.
That replaced a blind two-second commit timer, which cut words mid-pause and added up to two seconds of latency at end of speech.
What leaves your machine
The Transcription provider setting carries a data-flow note that changes with the provider you select. Audio is streamed over an encrypted connection to the provider you chose, using your own API key. Every voice request goes directly from your machine to the provider; none passes through a Daintree server.
- Transcription, OpenAI: audio streams to OpenAI's Realtime transcription service.
- Transcription, Deepgram: audio streams to Deepgram's streaming transcription service.
- Key check: Check and save on the OpenAI key makes one request to OpenAI to confirm the key works. The Deepgram key is saved without a check.
- AI text correction (off by default, OpenAI only): the dictated passage and its context go to OpenAI. See AI Text Correction.
- File reference resolution (with correction on, OpenAI only): an utterance that sounds like a file command, and sometimes candidate file paths, go to OpenAI. See File reference resolution.
Provider retention and training terms depend on the provider and account; this local implementation cannot establish them, and the notes in Settings are the provider's stated defaults, not something Daintree verifies. Keyterm biasing sends more than audio: your custom dictionary, project name, current branch tokens, and identifiers scraped from recent terminal output all travel with the session. That is what makes recognition accurate on your codebase, and it is worth knowing before you enable it on a sensitive repository. See Security for how Daintree treats keys and outbound data generally.
sk-proj-) rather than a legacy user key. Settings shows only a saved key's kind and last four characters.Setup
Enable Voice Input
- Open Settings > Voice input and turn on Dictation. The rest of the page appears once it is on.
- Choose a Transcription provider: OpenAI (the default) or Deepgram.
- Paste the provider's API key. The OpenAI key is saved with Check and save, which confirms it with OpenAI first; a Deepgram key is saved as is and checked the first time you dictate. A legacy OpenAI user key (not
sk-proj-) also shows Organization ID and Project ID fields. - Under Microphone, grant access if the Access row asks for it (see below), and pick an Input device or leave it on System default.
- Under Behavior, choose your Language: Auto-detect (multiple languages), English, Spanish, French, German, Japanese, Chinese, Korean, Portuguese, Italian, or Russian.
- Choose a Paragraph breaks strategy and a Recording mode.
Dictation becomes available once Dictation is on and the selected provider has its key. Those two conditions are the whole gate for the microphone button: microphone permission is not part of it, so the button can appear before the OS has granted access. If something will still stop a session from starting, the Dictation row says so, for example "Can't start yet: add your OpenAI API key and allow microphone access".
Environment key overrides
If you launch Daintree with WHISPER_API_KEY, its value overrides the OpenAI key saved in Settings. DEEPGRAM_API_KEY does the same for Deepgram. These values are read from the app's environment and take precedence on every settings read, so editing or removing the saved key does not change an environment override. Restart Daintree with the intended environment when changing one.
Grant Microphone Permission
Daintree needs microphone access from the operating system before it can capture audio. The settings panel shows the current permission status and gives instructions for your OS.
Open System Settings > Privacy & Security > Microphone and enable Daintree. If permission hasn't been requested yet, the Access row in Daintree shows a Request access button that triggers the system dialog.
If permission was denied earlier, click Open System Settings in the Daintree settings panel to go straight to the Microphone privacy settings. Toggle Daintree on, then click Re-check back in Daintree.
Open Windows Settings > Privacy & security > Microphone and confirm desktop app access is allowed. Daintree shows a Request access button if permission hasn't been determined yet.
If access was denied, click Open Windows Settings in the Daintree settings panel to jump to the Microphone privacy page. Enable access, then click Re-check in Daintree.
Grant microphone access through your system audio settings. Linux has no in-app Request access button, so you'll set permissions through your desktop environment or audio manager (PipeWire, PulseAudio, and so on).
Once permissions are granted externally, click Re-check in the Daintree settings panel to confirm access.
When access is denied, Open sound settings tries your desktop's audio panel, then standalone mixers such as pavucontrol. If none opens, Daintree says Couldn't open the settings app. Open it yourself, then re-check.
Select a Microphone
The Input device dropdown defaults to System default, which follows whatever input device your OS is using. Pick a specific device to pin dictation to it regardless of the system default. The list comes from the OS and refreshes when devices change, so plugging in a headset mid-session adds it. Refresh list, under the dropdown, re-enumerates inputs manually.
Starting and Stopping Dictation
There are four routes into dictation:
- Mic button. Every agent panel has one in its input bar. It always behaves as a toggle regardless of the recording mode below, since a click has no key-release to end. Clicking it while paused resumes rather than stops.
- Command-. Control-. Control-Alt-, . Dictates into the Daintree Assistant when the Assistant's input owns focus, and into the focused agent panel otherwise. If a session is already running anywhere, the same press stops it instead of moving it to whatever has focus now.
- Option-Shift-Command-V Control-Alt-Shift-V Control-Alt-Shift-V . Dictates into the Daintree Assistant from anywhere, opening it if it's closed.
- Action palette. Search for "Toggle voice dictation".
Toggle or push-to-talk
Recording mode in settings decides what the shortcut does:
- Toggle (default): press to start, press again to stop.
- Push to talk: hold the shortcut to record. Releasing the key stops recording without submitting.
Push-to-talk has three safety nets, because a held key is easy to lose track of: releasing the trigger key stops the session; releasing Cmd also stops it, since macOS swallows the trigger keyup while a modifier is still down; and the window losing focus stops it, since Chromium sends no synthetic keyup on a Cmd+Tab mid-press. The mic button and the palette keep working as toggles in this mode, because neither produces a keyup.
Pause, resume, and cancel
Control-Shift-Space Control-Shift-Space Control-Shift-Space pauses and resumes. A paused session holds its connection open and stops forwarding audio, so resuming reuses that connection, but a pause that runs past 60 seconds auto-stops the session rather than holding a socket open indefinitely.
Escape stops an active session and keeps everything dictated so far. Enter in an agent input bar stops the session and submits (see below). A stop that lands while Daintree is still connecting does not throw the opening audio away: it waits up to three seconds for the connection, then transcribes what the microphone captured.
Toolbar indicator
While a session is live, a mic icon appears in the global toolbar with an orbit ring. Its tooltip names the project and worktree being dictated into, shows the elapsed time as M:SS, and tells you which shortcut stops or resumes it. Clicking the indicator focuses the panel where recording is happening; it doesn't stop the session.
Session States
A dictation session moves through eight phases. Most are invisible in the happy path, but the mic button and toolbar indicator distinguish them, and knowing the set makes an unexpected state legible.
| State | What it means |
|---|---|
| idle | No active session. |
| arming | The press registered and the target is claimed. The cue paints at once, before the microphone opens, so you get immediate confirmation that the press landed. Escape during arming aborts cleanly. |
| connecting | Opening the transcription connection. The microphone is already capturing, and that audio is held until the provider is ready, then sent in order. Times out after 10 seconds. |
| recording | Connected, receiving audio, transcribing live. The listening cues wait for real audio; see below. |
| paused | Connection alive, audio capture suspended. Auto-stops after 60 seconds. |
| reconnecting | The connection dropped mid-session. Daintree is retrying with backoff and buffering captured audio; from your side the session is still running. |
| finishing | Stop requested, draining the final transcript. Force-closes after 3 seconds if the last segment never comes back. |
| error | The session ended on a fault that needs you (auth, quota, permission, configuration). |
When Daintree says it is listening
An open microphone is not proof that audio is arriving. A Bluetooth headset such as AirPods can hand over silence for a moment while it switches into its call profile, and words spoken into that gap never reached the transcriber. So Daintree starts capturing the moment you press, but only shows that it is listening once the microphone delivers real audio. Until then the input bar reads Starting microphone… and the mic button's orbit holds still; once audio arrives, the orbit spins and the input border takes the accent tint. Wait for that cue before speaking to avoid losing words while the microphone wakes up.
If no audio arrives within three seconds of capture, the bar and the mic button's tooltip say No audio from microphone. Check the Input device in Settings, or that the right microphone is connected and unmuted. The session stays open, so a microphone that wakes up late still lands.
Where Dictation Lands
By default dictation follows focus: the Assistant input if it owns focus, otherwise the focused agent panel. Dictation needs a terminal input surface; a browser or dev-preview panel has no composer for a transcript.
Two controls override that, both on a terminal panel's right-click menu:
- Lock dictation to this panel pins the target, so changing focus mid-thought doesn't redirect your next utterance. Unlock dictation releases it. A lock whose panel has since closed is cleared automatically and dictation falls back to focus.
- Recent dictation targets lists the last few panels you dictated into and starts a session on one directly. The list survives a restart; the panel identity does not, so entries that no longer resolve are dropped rather than shown.
Text as It Arrives
Interim transcription (the in-progress guess that keeps changing as you speak) is rendered as a ghost widget at the end of the line rather than inserted into the document. That distinction matters: text that was never inserted can't be undone, so a session's worth of interim churn leaves your undo history intact. The ghost is suppressed during IME composition so it can't displace a composition overlay, and the input bar grows to fit it the same way it grows for typed text.
Finalized segments are written into the input bar for real, in the order you spoke them: when the provider finishes a later utterance before an earlier one, the later one is held until the earlier one lands. If a session produces more than one paragraph, each lands as its own insertion with the break between them.
Submitting while recording
Pressing Enter while recording stops the session, syncs the settled transcript into the editor, and sends. The input bar goes read-only with a spinner for the moment that takes. AI correction is deliberately not waited on: it is fire-and-forget after a graceful stop, so Enter never blocks on a model call. Press Escape to cancel the submit and keep the text for editing.
Paragraph Breaks
Spoken commands (default)
With the spoken-command strategy selected, formatting phrases you say while dictating are rewritten in post-processing and stripped from the transcript:
| Spoken command | Inserts |
|---|---|
| "new paragraph" | A blank line |
| "new line" | A single newline |
| "period" / "full stop" | . |
| "comma" | , |
| "question mark" | ? |
| "exclamation point" / "exclamation mark" | ! |
Only a trailing chain of commands at the end of an utterance is converted, so "I'll add a new paragraph here" stays literal and you can still talk about the things themselves. Chains work: "hello comma new paragraph" becomes a comma followed by a blank line.
Enter only
Paragraph breaks come from Enter, and spoken formatting commands are disabled entirely. Choose this if you dictate in a non-English language, or if you want the phrases to stay literal.
Context Biasing
Generic speech models mishear codebase vocabulary, and no amount of post-correction fully recovers a word the transcriber never considered. Daintree biases the transcriber up front instead, assembling a keyterm list from four tiers of project context and sending it with the session: as literal keywords plus a mirrored prompt on the OpenAI path, and as Nova-3 keyterms on the Deepgram path.
| Priority | Source |
|---|---|
| 1 | Your custom dictionary |
| 2 | The project name and its tokens |
| 3 | Tokens from the current branch name |
| 4 | Identifiers scraped from recent terminal output, ranked best first |
The list is capped at 50 terms and 100 characters per term, with the terminal-derived tier capped at 30 so the higher tiers always keep headroom. Common shell commands and language keywords are blocklisted: biasing toward "const" or "grep" would cost accuracy rather than add it. The branch and terminal reads each race a 500 ms timeout, so a slow git call delays the start of dictation by at most half a second and then proceeds without that tier.
Custom Dictionary
The dictionary holds up to 100 domain-specific terms: product names, internal package names, unusual abbreviations. Add them under Terms in the Custom dictionary section of Settings > Voice input; each appears as a removable pill.
Dictionary terms do double duty. They are the top-priority tier of the keyterm bias above, and they are also given to the AI correction pass as preferred spellings. The correction prompt asks for them as preferences, not substitutions: when both the transcribed word and a dictionary term are plausible, the transcribed wording wins. Forcing them in produced worse output than leaving a near-miss alone.
Learning from corrections
Learn words from corrections (on by default) watches the edits you make to dictated text before sending, and proposes the words you keep fixing as dictionary terms. Suggestions appear under Suggested from corrections with an add and a dismiss control; hovering one shows what was heard instead. Nothing is added without you accepting it.
AI Text Correction
Turn on Clean up transcriptions in the AI text correction section (it is off by default) and Daintree sends the whole dictated passage for a cleanup pass once the session stops, then swaps the corrected version into the input bar in place. It fixes punctuation, filler words, technical spellings, and homophones, and it is told the language you dictated in and never to translate.
One OpenAI model backs every voice AI call (whole-passage correction, file-command detection, and ambiguous-file reranking) at the lowest reasoning tier, because these are fast, strictly-shaped tasks. The section's description names it (GPT-5.6 Luna in the current build). There is no correction-model dropdown. The earlier tiered choice was retired and existing settings were migrated to the current model on upgrade.
While the request is in flight, the dictated range carries a green dotted underline. When the response lands, the corrected text replaces it. Edit that text mid-flight and Daintree keeps your version rather than overwriting it.
The pass runs after a graceful stop with correction enabled and text to correct. It is skipped when the session ended on an error, and when you stop one recording only to immediately start another: there is no settled passage to correct in either case.
Custom instructions
The global Custom instructions field appends your custom rules to the core correction prompt: "always capitalize ProductName as one word", "React component names use PascalCase". They sit below the built-in correction rules in priority. Inspect core prompt, on the Core prompt row, shows the full base prompt read-only, so you can see what your instructions are being added to. Your project name, repository directory name, and dictionary are injected automatically.
File reference resolution
With correction on, Daintree detects spoken file references and resolves them into @file links. Say things like "link to the auth helper", "at file the button component", "reference the user model", "add file the config service", or "open the main layout". The description is matched against the project file tree and the best candidate is picked, then rendered as a clickable file chip.
Detection is two steps. A local pattern check looks for phrasing like the examples above; only an utterance that passes it is sent to OpenAI, which extracts the file description from it. File matching then starts locally. When several candidates need an AI tie-break, Daintree sends the spoken description and candidate repository file paths to OpenAI using your configured key. File names can reveal project information even though this request does not send the file contents. See the resolver implementation for the exact request.
If resolution fails, the text falls back to @?description so you can see what was being looked up and fix it by hand. Toggle the behavior with Resolve file references; it is on by default, and needs correction on and an OpenAI key available.
Settings Reference
Everything lives in Settings > Voice input. Only Dictation shows until you turn it on.
Speech-to-text
| Setting | Values / notes |
|---|---|
| Dictation | On / Off (default: Off) |
| Transcription provider | OpenAI (default) or Deepgram, with a note on where audio goes |
| OpenAI API key | Shown when the provider is OpenAI. Check and save confirms the key with OpenAI before keeping it. A project key (sk-proj-) is recommended. |
| Deepgram API key | Shown when the provider is Deepgram. Saved without a check; checked the first time you dictate. |
| Organization ID, Project ID | Shown only while a legacy OpenAI user key (not sk-proj-) is saved |
| Microphone > Access | The OS permission status, with Request access, Open System Settings (or Open Windows Settings) and Re-check as they apply |
| Microphone > Input device | System default (default) or a specific input device, with Refresh list |
Behavior
| Setting | Values / notes |
|---|---|
| Language | Auto-detect (multiple languages), English (default), Spanish, French, German, Japanese, Chinese, Korean, Portuguese, Italian, Russian |
| Paragraph breaks | Spoken commands (default) or Enter only. Fixed to Enter only for any language other than English. |
| Recording mode | Toggle (default) or Push to talk |
Custom dictionary
| Setting | Values / notes |
|---|---|
| Learn words from corrections | On by default. Proposes dictionary terms from edits you make before sending. |
| Suggested from corrections | Proposed terms, each with add and dismiss controls |
| Terms | Up to 100 terms. Biases the transcriber and steers correction. |
AI text correction
The section appears once Dictation is on. Its other rows appear once Clean up transcriptions is on, and stay disabled until an OpenAI key is available.
| Setting | Values / notes |
|---|---|
| Clean up transcriptions | On / Off (default: Off) |
| OpenAI API key | Shown here when Deepgram is the transcription provider |
| Resolve file references | On by default |
| Custom instructions | Free-form rules appended to the core correction prompt |
| Core prompt | Inspect core prompt shows the base correction prompt read-only |
Keyboard Shortcuts
| Action | Shortcut |
|---|---|
| Toggle voice dictation | Command-. Control-. Control-Alt-, |
| Toggle voice dictation in Daintree Assistant | Option-Shift-Command-V Control-Alt-Shift-V Control-Alt-Shift-V |
| Pause or resume dictation | Control-Shift-Space Control-Shift-Space Control-Shift-Space |
Two more keys act on a live session without being bindings of their own: in an agent input bar Enter stops the session and submits, and Escape stops dictation, or cancels a submit already in progress.
All three voice bindings are rebindable. See Keyboard Shortcuts for the full reference.
Troubleshooting
The mic button isn't there
The button is hidden until Voice Input is configured, which means both of: Dictation is on in Settings > Voice input, and the selected provider has its key. A Deepgram provider with only an OpenAI key saved counts as unconfigured.
"No audio from microphone"
The session opened but the microphone has delivered nothing above silence for three seconds. Check that Input device points at the microphone you are using, that it is connected and unmuted, and that the Access row reports access granted. A Bluetooth headset can take a moment to switch into its call profile; the session stays open, so wait for the listening cue before you speak.
The session keeps saying "Reconnecting"
Transient faults (a rate limit, a dropped socket) don't end the session. Daintree retries with backoff while buffering audio, and the mic button says so rather than raising an alarm. After several failed attempts it gives up with "Connection failed after several retries", which is fatal and ends the session.
"Connection timed out"
The transcription socket didn't open within 10 seconds. Check your connection, that the key for the selected provider is valid, and that the account has credit.
"Rate limit exceeded: check your API key quota"
The provider rejected the session outright rather than throttling it. This is the fatal variant; the transient one reads "Rate limited, reconnecting" and recovers on its own.
Spoken paragraph commands aren't working
Check that Paragraph breaks is set to Spoken commands and Language is English (not Auto-detect). Remember that only a trailing chain at the end of an utterance converts: a command in the middle of a sentence stays literal by design.
AI correction isn't running
Correction needs Clean up transcriptions on, an OpenAI key available, and dictated text present at stop. It is skipped when the session ended on an error, and when you stop one recording and immediately start another.
Recognition keeps missing the same word
Add it to the custom dictionary. It becomes a top-priority keyterm on the next session, which biases the transcriber before the mistake happens, a stronger fix than leaving it to the correction pass. If Learn words from corrections is on, fixing the word by hand a few times will surface it as a suggestion anyway.
File references show as @?
@?description means resolution found no close match in the project file tree, or the description was too vague to disambiguate. Be more specific when you dictate the reference.
Implementation Reference
The OpenAI adapter, Deepgram adapter, and recording lifecycle define Daintree's behavior. Model names here describe this integration, not a promise of service availability for every account.