Activefrontiermultimodal Proprietary

computer-use-preview

by OpenAI· Released October 2024· Cutoff October 2023

computer-use-preview is a specialized model from OpenAI designed to control computer interfaces by interpreting screenshots and performing actions like clicking, typing, and scrolling. It is part of the GPT-4o family and enables AI agents to interact with software applications directly, bridging the gap between language models and GUI automation.

Official Site API Docs

Input cost

$3.00 per 1M tokens

Output cost

$12.00 per 1M tokens

Context window

128K tokens

Max output

4096 tokens

Modalities

textimage

License

proprietary

Capabilities

Computer UseVisionFunction CallingCode GenerationStreamingJSON ModeStructured Outputs

Best For

Automating GUI-based tasks and controlling computer interfaces via natural language instructions.

Strengths

Direct computer control via screenshots and actions
High accuracy in interpreting visual interfaces
Supports complex multi-step workflows
Integrated with OpenAI's safety and monitoring tools

Limitations

Limited to screenshot-based interaction (no real-time video)
May struggle with highly dynamic or non-standard UIs
Requires careful prompt engineering for reliable performance
Higher latency compared to text-only models

Use Cases

Automating data entry in legacy software

Testing web applications by simulating user interactions

Building AI assistants that can navigate desktop applications

Extracting information from non-API-accessible systems

Automating repetitive tasks in enterprise software

Creating demos and tutorials by controlling the interface

Assisting users with accessibility by performing UI actions

Improvements Over Previous Model

First model from OpenAI specifically designed for computer use
Enables direct GUI interaction without custom integrations
Built on GPT-4o architecture with enhanced visual grounding
Includes safety measures to prevent unintended actions

Back to all models