Sourcify
Effortlessly find and manage open-source dependencies for your projects.

Empowering NPCs with human-like conversation and intelligent spatial awareness in virtual worlds.

Convai is a specialized AI orchestration platform designed to provide real-time conversational intelligence for non-player characters (NPCs) and virtual assistants within 3D environments. By 2026, its architecture has evolved into a highly optimized pipeline that bridges Large Language Models (LLMs) with gaming engines like Unity, Unreal Engine 5, and NVIDIA Omniverse. The platform's technical core is built around a low-latency Speech-to-Speech (S2S) engine that handles STT (Speech-to-Text), LLM processing, and TTS (Text-to-Speech) in a unified stream, reducing perception-response time to sub-200ms levels. Beyond simple dialogue, Convai distinguishes itself through its Action API and Spatial Awareness features, which allow characters to perceive their environment and execute programmatic actions based on natural language commands. This enables a level of agency where an NPC can navigate a map, identify objects, and interact with the game world's physics or logic systems autonomously. Positioned as a mission-critical middleware for the Metaverse and high-fidelity training simulations, Convai provides the infrastructure for persistent, memory-capable digital entities that evolve through user interaction.
Convai is a specialized AI orchestration platform designed to provide real-time conversational intelligence for non-player characters (NPCs) and virtual assistants within 3D environments.
Explore all tools that specialize in rag querying. This domain focus ensures Convai delivers optimized results for this specific requirement.
A semantic mapping engine that converts user verbal requests into structured JSON triggers for game engine logic.
Enables NPCs to query the 3D bounding boxes and metadata of nearby objects in the game scene.
Vector database integration that stores past interactions, allowing characters to recall previous conversations across sessions.
Generates viseme and blend-shape data in real-time synchronized with the audio stream.
An interface that allows the AI to receive a text-based summary of the 3D environment layout.
Integration with ElevenLabs and proprietary models for custom character voices.
A unified character state managed on the cloud, accessible via REST or gRPC across different engines.
Sign up for a Convai developer account and obtain your API Key.
Create a new Character in the Convai Dashboard.
Define the character's 'Backstory' and 'Core Personality' to guide the LLM's response style.
Upload knowledge base documents (PDF/TXT) to the character for RAG-based domain expertise.
Define 'Actions' in the dashboard that the NPC can trigger (e.g., 'Open Door', 'Follow Player').
Download the Convai SDK for your specific engine (Unity, Unreal Engine, or Web).
Import the SDK and enter your Character ID and API Key in the engine's global settings.
Map the SDK's lip-sync and skeletal animation outputs to your 3D character model.
Configure the spatial awareness sensor on the NPC to detect nearby game objects.
Test interactions in-editor using the push-to-talk or text-input components.
All Set
Ready to go
Verified feedback from other users.
"Users praise the extremely low latency and the power of the Action API, though some note a steep learning curve for complex spatial integration."
Post questions, share tips, and help other users.
Effortlessly find and manage open-source dependencies for your projects.

End-to-end typesafe APIs made easy.

Page speed monitoring with Lighthouse, focusing on user experience metrics and data visualization.

Topcoder is a pioneer in crowdsourcing, connecting businesses with a global talent network to solve technical challenges.

Explore millions of Discord Bots and Discord Apps.

Build internal tools 10x faster with an open-source low-code platform.

Open-source RAG evaluation tool for assessing accuracy, context quality, and latency of RAG systems.

AI-powered synthetic data generation for software and AI development, ensuring compliance and accelerating engineering velocity.