Voice notes
Chatbots answer the voice notes customers send on any channel. Each note is transcribed and answered like a typed message, can answer a form's file question, and can be answered in a voice. Agents can play it in the inbox.
When a customer sends a voice note, your chatbot transcribes it and answers what they said, the same way it answers a typed message. Everything in your flow works on the transcript: keyword triggers, intents, forms and AI nodes all see the words the customer spoke.
Voice notes are on by default. A chatbot published before a feature on this page existed needs to be republished to use it.
Channels
| Channel | What the customer sends |
|---|---|
| A voice note, or an audio file they forward | |
| Telegram | A voice message, or an audio file |
| Messenger and Instagram | A voice message |
| Slack | An audio clip. Your Slack app needs the files:read scope, and must be reinstalled after you add it. Without it, the bot can't fetch the clip and asks the customer to type. |
| Web chat widget | A note recorded with the microphone button. See Recording in the web chat. |
Settings
Open the chatbot's Project settings and go to the Voice notes tab.
| Setting | What it does |
|---|---|
| Answer voice notes | On: voice notes are transcribed and answered. Off: a customer who sends one is asked to type their message instead. |
| Transcription | Who transcribes the audio. Buni is the default. You can also choose your own speech-to-text vendor. See Who pays. |
| API keys | Shown when you choose your own vendor. The workspace's key for that vendor is used unless you pick a different stored credential here. |
| Show customers what was heard | Sends the customer the transcript (🎤 "…") before the reply, so they can spot a mis-hearing. It is never sent for an answer to a form field marked sensitive. |
| Reply with a voice too | Answers a voice note with a voice clip of the reply as well as its text. See Replying in a voice. |
Answer voice notes, Show customers what was heard and Reply with a voice too are built into the bot when you publish, so republish after changing them. A change of vendor, voice or key applies to the next voice note, without republishing.
Who pays
| Transcription | Cost |
|---|---|
| Buni | 1 credit for each started second of audio. The whole note is charged, including silence, because the vendor processes all of it. |
| Your own vendor | Your vendor bills you directly. Buni records the seconds on the usage page but charges nothing for them. |
Transcription by Buni comes with paid plans: a package, or Pay as you go with usage billing set up. On the free plan, choose your own vendor; otherwise customers who send a voice note are asked to type instead. The Voice notes tab says when this applies to your workspace. When a package's credits run out and nothing is set up to pay for more, voice notes are paused with the rest of your usage, and customers are asked to type.
You can use Deepgram, OpenAI, ElevenLabs Scribe or AssemblyAI. Each runs on its vendor's current model. Add the vendor's API key on the Credentials page first. A vendor without a key in the workspace can't be saved.
If your own vendor's key is deleted later, voice notes are not moved onto Buni's transcription, because that would change who pays. Customers are asked to type until you add the key back or switch to Buni.
Voice-note seconds appear under Voice transcription on the usage page. Each voice note is also counted as a chatbot message, like any other message.
Branching on a voice note
A flow can tell whether the customer spoke or typed. {{requestAttributes.inputMode}} is voice when the message was a transcribed voice note, and text otherwise. Use it in a Router, for example to keep spoken answers short. It is listed under System in the variable picker.
A voice note as a form answer
A voice note can answer a form's file question, so a customer can describe a problem or leave a statement by speaking. The file question must accept the note's audio type under Accepted File Types:
| Audio type | Voice notes from |
|---|---|
| OGG | WhatsApp, Telegram, and the web chat in Firefox |
| M4A | Messenger, Instagram, and the web chat in Safari |
| WebM | The web chat in Chrome and Edge |
A file question that accepts any file type doesn't take voice notes, so a spoken "I'll send it later" can't stand in for a document. Republish after changing the form.
When the file question takes the note, its answer is the stored audio, the same as an uploaded file, and it is scanned before the form submits like any upload. The transcript is still the message in the conversation. Questions that aren't file questions are answered from the transcript, as before.
Replying in a voice
Turn on Reply with a voice too, then choose an AI voice and a Voice. The voice runs on your workspace's own key for that vendor: ElevenLabs, Cartesia, Rime, Google Cloud TTS, Microsoft Azure, Amazon Polly or Deepgram. Add the key on the Credentials page, or pick a stored key under API keys. Your vendor bills you directly, and Buni charges nothing for it.
When a customer sends a voice note, the bot's text reply is spoken and sent after the text:
| Channel | How the voice reply arrives |
|---|---|
| An audio message | |
| Telegram | A voice message |
| Messenger and Instagram | An audio message |
| Slack | A Voice reply link that plays the clip |
| Web chat | A player under the reply |
- The text is always sent too. If the voice can't be made (the vendor is down, or its key was removed), the text goes on its own.
- Only the words are spoken. Formatting, links and emoji are left out.
- A reply longer than 2,000 characters isn't spoken, because a clip cut short could leave out what mattered.
- The bot doesn't speak while a person is handling the conversation, or after an answer to a form field marked sensitive.
- A reply the bot gives often is spoken once and reused for 30 days, so it costs one vendor call in that time.
When a voice note can't be answered
The customer always gets a reply:
| Situation | What the customer is told |
|---|---|
| The note is longer than 5 minutes or larger than 3 MB | That voice note is too long. Send a shorter one, or type it. |
| Nothing could be made out, or the transcription failed | Their note couldn't be made out. Type it instead. |
| Answer voice notes is off, no transcription is set up, or voice notes are paused | Voice messages can't be listened to here. Type the message instead. |
The note appears in the conversation as "🎤 Voice note". Its audio is deleted, because nothing can tell whether it answered a form field marked sensitive. If a person is handling the conversation, the audio is kept so they can play it.
A bot that waits for a keyword stays silent when it can't make out a voice note, as it does for any message without the keyword.
Recording in the web chat
The web chat widget shows a microphone button in place of send while the message box is empty. The customer taps it, speaks, and taps send, or the bin to discard the note. A note stops recording and sends itself at 5 minutes. The browser asks for the microphone the first time.
The canvas simulator has the same button once the project has been published, so you can test voice notes as you build.
The microphone shows only when Answer voice notes is on and a note can be transcribed: on your own vendor, or on Buni's on a paid plan (see Who pays). It also stays hidden where the browser won't allow the microphone. A widget in front of a bot published before web voice notes existed tells the customer to type until you republish.
If the customer closes or minimises the chat while recording, or the conversation ends, the note is discarded rather than sent.
If you embed the chat in your own iframe, add allow="microphone" to it, or the browser won't let the chat use the microphone. The snippets in the embed manager include it. For the React, Vue and Angular packages, make sure your site has @buni.ai/chatbot-core 1.2.0 or later, which they install alongside them.
In the inbox
A voice note appears in the conversation with its transcript as the message text and a player underneath. The label under the player says whether the note was transcribed. A spoken reply appears under the bot's message as Voice reply.
While an agent is handling a conversation, the bot doesn't reply to voice notes. The agent gets the note with its transcript and can play it, including notes that couldn't be transcribed.
Languages
The language of each note is detected automatically, so customers can speak whichever language they use with your bot. How well a language is recognised depends on the vendor. If your customers use a language that one vendor handles poorly, try another under Transcription.
Privacy
- The audio is stored so agents can play it. It is deleted once it is older than your organization's call-recording retention period (30 days unless you change it), because a recording of someone's voice identifies them in a way the text of what they said does not. The transcript stays in the conversation like any other message. See Privacy and data.
- If a voice note answers a form field marked sensitive, the audio and the transcript are deleted straight away, and the message is masked in the conversation, as a typed answer would be. If the field is a sensitive file question that took the note, the stored answer is kept with the form's other uploads.
- The audio of a note that couldn't be transcribed is deleted straight away, unless a person is handling the conversation.
- When one of the project's forms accepts voice notes, a copy of each note is kept with the form uploads, so a file question can take it. The copy is deleted with the note, and form uploads are removed after 30 days in any case.
- Voice replies are the bot's own words, never the customer's voice. Each is deleted 30 days after it was made; until then, anyone with a reply's link can play it.
- If your organization has switched transcript expiry off, voice notes are kept too.