Sourcify
Effortlessly find and manage open-source dependencies for your projects.

Automated testing that finds & fixes AI risk in development.

Promptfoo is an open-source CLI and library for evaluating and red-teaming LLM applications. It facilitates test-driven LLM development by enabling users to build reliable prompts, models, and RAGs with benchmarks tailored to their use-case. The tool supports automated red teaming and pentesting to secure apps and speeds up evaluations through caching, concurrency, and live reloading. Users can score outputs automatically by defining metrics. It integrates with various LLM APIs including OpenAI, Anthropic, Azure, Google, HuggingFace, and open-source models like Llama. Promptfoo provides continuous monitoring, centralized security dashboards, and actionable remediation steps to close the loop between finding and fixing vulnerabilities. Its architecture supports integration into CI/CD pipelines, GitHub, GitLab, and Jenkins and can be deployed on-premise or in the cloud.
Promptfoo is an open-source CLI and library for evaluating and red-teaming LLM applications.
Explore all tools that specialize in vulnerability scanning. This domain focus ensures Promptfoo delivers optimized results for this specific requirement.
Simulates real users to uncover application-specific vulnerabilities using custom attacks generated for the target, including prompt injections, jailbreaks, and data leaks.
Leverages a community of 300k+ users for early warnings on new attack vectors, which are automatically deployed, providing up-to-date security.
Monitors LLM security status across all projects with real-time alerts and automated evaluations, ensuring ongoing compliance and security.
Tracks remediation progress and provides suggested steps for each issue, streamlining the process of fixing vulnerabilities.
Allows the creation of customizable plugin collections, scan configurations, and target settings that can be shared among colleagues, ensuring tailored security measures.
Install Promptfoo CLI: `npx promptfoo@latest init`
Connect to your AI applications, agents, and workflows.
Define test cases by identifying core use cases and failure modes.
Configure evaluation by specifying prompts, test cases, and API providers.
Run evaluation using the command-line tool or library.
Analyze results in a structured format or web UI.
Implement remediation steps based on security findings and track fixes across teams.
Continuously monitor for new vulnerabilities and maintain compliance.
All Set
Ready to go
Verified feedback from other users.
"Promptfoo is highly regarded for its ease of use, comprehensive feature set, and ability to integrate seamlessly into existing development workflows."
Post questions, share tips, and help other users.
Effortlessly find and manage open-source dependencies for your projects.

End-to-end typesafe APIs made easy.

Page speed monitoring with Lighthouse, focusing on user experience metrics and data visualization.

Topcoder is a pioneer in crowdsourcing, connecting businesses with a global talent network to solve technical challenges.

Explore millions of Discord Bots and Discord Apps.

Build internal tools 10x faster with an open-source low-code platform.

Open-source RAG evaluation tool for assessing accuracy, context quality, and latency of RAG systems.

AI-powered synthetic data generation for software and AI development, ensuring compliance and accelerating engineering velocity.