👋 Hi Product Hunt,
My name is Vince, and together with my friend Hugh we've been building LLMonitor, an open-source observability toolkit for builders of AI apps.
If you’re using OpenAI or LangChain, the integration takes seconds to start recording data.
Main Features:
⚡ Log full queries & errors
💸 Metrics into costs, tokens & latency
👨👩👧👦 Track users and their usage
💬 Record chat conversations & user feedback
🕹️ Prompt playground to iterate on prompts
🐾 Traces to easily debug agents
🧪 Evaluations of outputs quality
🧑💻 You can Self-host it on your own servers. Or use our cloud-hosted version (free up to 1k events /day.)
I started building LLMonitor as an internal tool at my previous startup. We had no insight into how users interacted with our chatbots and needed a way to monitor chats and LLM queries easily.
If you’re building a conversational app powered by AI, we’d love to hear what you think and how we can make it better!
Also, we'd be reaaally happy if you can support us with a ⭐ star on GitHub: https://github.com/llmonitor/llm...
Thankss 🙏
@vincelwt Hey Vince. Congrats on the launch. For sure AI apps need a lot of quality safeguarding. This tool seems to be a great aid to developers for that. Keep up the good work!
LLMonitor
New Year Resolutions AI
LLMonitor
Portkey
LLMonitor