How to give your Agno agent a voice with Smallest AI
Add text-to-speech to an Agno agent with SmallestTools. Pick a voice and language, then get the audio back as a saved file or as bytes for your app.
Cosette CresslerContent & Product Marketing LeadNovember 16, 20269 min read
To add text-to-speech to an Agno agent, add SmallestTools() to its tools list and set SMALLEST_API_KEY. The toolkit gives the agent two tools backed by Smallest AI's Lightning models: get_voices lists the voices available to it, and text_to_speech turns text into audio.
You pick the model and language when you create the toolkit, along with a default voice. The audio comes back on response.audio, and the toolkit can also save each clip to a folder for you.
What do you need?
The Gemini example below needs two API keys:
- A Smallest AI API key from the Smallest AI dashboard, under Developer, then API Keys. Export it as
SMALLEST_API_KEY. - A Google API key for the agent's Gemini model, exported as
GOOGLE_API_KEY.
Install Agno with its Google model integration:
pip install -U "agno[google]"Then set your keys:
export SMALLEST_API_KEY="your_smallest_api_key"
export GOOGLE_API_KEY="your_google_api_key"SmallestTools needs no extra SDK. It calls Smallest AI over httpx, which installs with Agno. You can also pass the Smallest AI key through the toolkit's api_key parameter.
How do you generate your first audio clip?
Add SmallestTools() to the agent, and set target_directory if you want the audio saved to disk. Tell the agent when to produce speech:
from agno.agent import Agent
from agno.models.google import Gemini
from agno.tools.smallest import SmallestTools
agent = Agent(
model=Gemini(id="gemini-pro-latest"),
tools=[
SmallestTools(
voice_id="kaitlyn",
language="en",
target_directory="tmp/audio",
)
],
instructions=["Use text_to_speech when the user requests spoken audio."],
)
response = agent.run(
"Generate spoken audio welcoming listeners to a podcast "
"about the history of aviation."
)
if response.audio:
print(f"Generated {len(response.audio[0].content)} bytes of audio")
else:
print("No audio was generated.")
print(response.content)With target_directory set, the toolkit creates the folder if it's missing and saves each clip under a unique filename, such as tmp/audio/3f2c….wav, so repeated calls never overwrite each other. Leave it off and the toolkit writes nothing to disk.
Either way, response.audio holds a list of audio objects. The first clip's bytes are in response.audio[0].content.
The example uses Gemini because it accepts audio input, so the model receives the clip the tool generated and can respond to it. With a model that can't take audio input, response.audio still gets the clip, but the model never hears it and Agno logs "Audio input is currently unsupported."
Which Smallest AI model should you use?
Lightning v3.1 Pro, which is what the toolkit configures when you don't set model. It draws on the premium voice pool and covers 31 languages. The toolkit accepts one other ID:
| Model | Status |
|---|---|
lightning_v3.1_pro (default) | The current model: premium voices, 31 languages |
lightning_v3.1 | Legacy. Smallest AI retires it on 30 November 2026 at 18:30 UTC |
New projects should leave the default alone. If you already pass model="lightning_v3.1", move to the Pro model and pick a voice from its pool before that date, since the toolkit sends the model name straight through to the API.
An unknown model name raises a ValueError when you create SmallestTools, before the agent runs. That includes the dated preview checkpoints Smallest AI publishes, such as lightning_v3.1_pro_10_26. The toolkit only accepts the two IDs above.
Language support depends on the model and the voice. The Lightning v3.1 and Lightning v3.1 Pro model cards list the codes each one supports.
How do you choose a voice and language?
The default voice is kaitlyn, and the default language is English ("en"). Set both when you create the toolkit:
SmallestTools(
model="lightning_v3.1_pro",
voice_id="kaitlyn",
language="en",
)Write the text in the language you set and pick a voice that speaks it. The language parameter sets the synthesis language. It doesn't translate anything.
The voice has to belong to the configured model's pool. Smallest AI rejects a mismatched pair, and the toolkit returns an error instead of audio.
If you don't know which voice to use, let the agent call get_voices. It returns the ID and name of every voice in the Lightning v3.1 Pro catalog, along with metadata such as gender and accent, and it lists the languages each voice supports. So you can ask for a British female voice and let the agent pick one. The toolkit reads that same Pro catalog whichever of the two models you configured, which is another reason to stay on the Pro model: against the legacy model, a voice the agent discovers there may not be one it can use.
The agent can also pass a different voice_id on any text_to_speech call, which means one agent can voice both sides of a dialogue.
If you don't need voice discovery, turn it off:
SmallestTools(
voice_id="kaitlyn",
enable_get_voices=False,
)This removes get_voices from the agent's tools. It doesn't lock the agent to kaitlyn, because text_to_speech still accepts a voice override.
Which audio format should you use?
The toolkit defaults to WAV and supports these formats:
| Format | Use it for |
|---|---|
wav (default) | Playback, editing, and further audio processing |
mp3 | Compact files for web or mobile playback and storage |
pcm | Audio pipelines that want raw samples |
ulaw, alaw | Phone calls: μ-law is the telephone standard in North America and Japan, and A-law in Europe and most other regions |
Set the format with output_format:
SmallestTools(output_format="mp3")Two more settings shape the sound:
sample_ratedefaults to24000. Smallest AI accepts8000,16000,24000, and44100Hz.speeddefaults to1.0and accepts values from0.5to2.0. Raise it for faster speech.
For phone audio, match the encoding and sample rate your telephony provider expects. μ-law at 8,000 Hz is a common choice:
SmallestTools(
output_format="ulaw",
sample_rate=8000,
)Each audio object carries its format and sample rate, so whatever reads response.audio knows how to play it. Your playback or telephony system still has to support the encoding you picked, plus any framing it requires.
What happens when speech generation fails?
A failed request doesn't crash the run. The toolkit catches HTTP errors and non-audio responses and returns a tool result that starts with Error:. The agent can use it to explain the failure or try again, but it won't retry on its own.
If SMALLEST_API_KEY isn't set, the toolkit logs an error when you create it, and generation fails with an authentication error.
Requests time out after 30 seconds by default. Raise the limit for long passages:
SmallestTools(timeout=60)Check response.audio before you read a clip. It's empty when generation failed or when the agent didn't call the speech tool.
What can you do with the generated audio?
text_to_speech returns a ToolResult that carries the audio, and Agno adds it to response.audio. From there it's ordinary product work: store the file and serve a URL, attach it to a message, play it in your app, or hand μ-law audio to your phone system.
For example, with the default WAV output:
from pathlib import Path
if response.audio:
Path("welcome.wav").write_bytes(response.audio[0].content)The toolkit returns each clip after Smallest AI finishes synthesizing it, so you can't play audio mid-generation.
If you're serving the agent inside your product, see How to build an agent for your product and serve it everywhere. It shows how one agent can serve REST clients, Slack users, Claude, and ChatGPT from the same code.
For a runnable version of this guide, see the Smallest AI cookbook. The Smallest AI toolkit reference lists every option.
Checked against Agno v3.2.0 and the Smallest AI documentation on 16 November 2026. Smallest AI updates its models and voices often, so check the linked model cards before you ship.
Sources
Frequently asked questions
Add SmallestTools() to an Agno agent, set SMALLEST_API_KEY, and set language to a supported language code. Write the text in that language and choose a voice that speaks it. The audio comes back on response.audio.
Yes. Create the clone in Smallest AI, then pass its ID as voice_id. Configure the model the voice was cloned onto. The toolkit cannot create a clone itself.
Set output_format to ulaw or alaw, and set sample_rate to the rate your telephony provider expects, usually 8,000 Hz. μ-law is the standard in North America and Japan, and A-law in Europe and most other regions.
No. enable_get_voices=False removes voice discovery, but the agent can still pass a different voice_id when it calls text_to_speech. The configured voice only sets the starting choice.
If you set target_directory, the toolkit saves each clip there under a unique filename. Otherwise it does not write a file. Either way, the bytes are on response.audio.
No. The toolkit makes speech generation available to the agent. Give it instructions or a prompt that asks for audio so it knows when to call text_to_speech.
Not for the toolkit itself, since it calls the Smallest AI API over httpx, which installs with Agno. Your model provider may need its own extra, such as agno[google] for Gemini. You also need a Smallest AI API key and a key for the model that runs the agent.
Lightning v3.1 Pro, with kaitlyn as the default voice. The older lightning_v3.1 model is still accepted but Smallest AI retires it on 30 November 2026, so new projects should stay on the Pro default.