Activefrontiermultimodal Proprietary

GPT-4o-audio-preview

by OpenAI· Released October 2024· Cutoff October 2023

GPT-4o-audio-preview is a multimodal model that extends GPT-4o with native audio input and output capabilities, enabling real-time voice interactions and audio processing. It is designed for applications requiring low-latency speech-to-speech or audio understanding, such as voice assistants and audio transcription with reasoning.

Official Site API Docs

Input cost

$5.00 per 1M tokens

Output cost

$15.00 per 1M tokens

Context window

128K tokens

Max output

4096 tokens

Modalities

textaudio

License

proprietary

Capabilities

Audio InputAudio OutputText InputText OutputFunction CallingStreamingJSON ModeSystem Messages

Best For

Real-time voice applications and audio-based interactions requiring low-latency multimodal understanding and generation.

Strengths

Native audio input and output for seamless voice interactions
Low-latency response suitable for real-time applications
Combines GPT-4o's reasoning with audio modality
Supports function calling and structured outputs

Limitations

Preview model with potential for changes and limited stability
Higher cost compared to text-only GPT-4o
Audio quality and accuracy may vary in noisy environments
Limited to English language support initially

Use Cases

Real-time voice assistants and chatbots

Audio transcription with summarization and analysis

Voice-controlled applications and hands-free interfaces

Language learning and pronunciation feedback

Accessibility tools for visually impaired users

Customer service voice bots

Audio content generation and editing

Improvements Over Previous Model

First OpenAI model with native audio input and output support
Enables real-time speech-to-speech interaction without separate ASR/TTS pipeline
Reduces latency compared to chaining separate audio and text models
Maintains GPT-4o's reasoning capabilities while adding audio modality

Back to all models