Open-source testing platform for AI agents. Run simulations, catch regressions, and ship autonomous agents with confidence. Built for developers who treat AI like software. Agent simulations are the new unit tests
Subscribe
Sign in
Open-source testing platform for AI agents. Run simulations, catch regressions, and ship autonomous agents with confidence. Built for developers who treat AI like software. Agent simulations are the new unit tests
Launched on December 19th, 2024
How do you validate an AI agent that could reply in unpredictable ways?
My team and I have released Agentic Flow Testing an open-source framework where one AI agent autonomously tests another through natural language conversations.
Every day I speak with AI teams building with LLM-powered applications and something is changing.
I see a new role is quietly forming:
The AI Quality lead as the quality owner.
Awesome to see more products launch in this field. One of the biggest challenges for companies to create AI solutions is having the right platform in place as a starting point. Can't wait to explore this product even more for my own team!
I highly recommend LangWatch to anyone looking to elevate their AI-generated content with precision and effectiveness.
I recently spun up LangWatch, Langfuse, Langsmith and Opik for a real comparison in our production environment and LangWatch was just such a pleasure to use. It just hits the mark on everything that I want a monitoring platform to have. I'm looking forward to the video showcasing my experience.
LangWatch Agent Simulations
Hey Product Hunt! 👋
We're excited to be launching LangWatch Scenario the first and only testing platform that allows you to test agents in simulated realities, with confidence and alongside domain expertise.
The problem that we’ve found is that teams are building increasingly complex agents, but testing them is still manual, time-consuming, and unreliable. You tweak a prompt, manually chat with your agent, hope it works better... and repeat. It's like shipping software without unit tests.
Our solution: Agent simulations that automatically test your AI agents across multiple scenarios. Think of it as a test suite for agents — catch regressions before they hit production, simulate edge cases alongside domain experts in a collaborative fashion, and ship with confidence.
What makes us different:
🧠 Agent simulations that act as unit tests for AI agents
🧪 Simulate multi-turn, edge-case scenarios
🧑💻 Code-first, no lock-in, framework-agnostic
👩⚕️ Built for domain experts and not just devs
🔍 Catch failures before users see them
✅ Trust your agent in production, not just evals
🏗️ Works with any agent framework (LangGraph, CrewAI, etc.)
LangWatch scenarios is our latest breakthrough that will allow teams to ship agents with confidence, not crossed fingers.
Get started today:
⭐ GitHub: https://github.com/langwatch/scenario
📖 Docs: https://docs.langwatch.ai/
🎮 Try Agent Simulations: https://langwatch.ai/
If you're building and testing AI agents, we'd love to hear what you're working on and how we can help.
A big thanks to the PH community for all your feedback and support.
We're here all day and can't wait to hear your thoughts, questions, and feedback!
LangWatch Agent Simulations
Hello everyone! 👋
I'm Rogerio, founder of LangWatch, been developing software for 15+ years, and my career really changed once I started dominating unit tests, TDD and so on, not only delivering mission critical software with zero bugs but also having a much more pleasant experience in doing so.
So I couldn't be more excited for the Agent Simulations solution we are bringing today to the world, it feels like finally the missing piece in delivering agents, bringing much stronger craftsmanship to agent development.
I'll be your technical guide here, ask me anything!
LangWatch Agent Simulations
Alphadoc
LangWatch Agent Simulations
Thanks @jobrietbergen AI agents need even more than just evals, give it a try!