Subscribe
Sign in
Langtail 1.0 — The low-code platform for testing AI apps
LLM testing made easy with a spreadsheet-like interface. Score tests with natural language, pattern matching, or code. Optimize LLM apps by experimenting with models, parameters, and prompts. Gain insights from test results and analytics.
56
Replies
Best
@yigit For integration, we have SDK or OpenAPI. You can also use us as a Proxy - all data goes through us, or asynchronously - you send data from your codebase to Langtail. Additionally, you can use us just for development if you want to develop prompts and test them in the UI environment.
Looks super interesting! The logs are a nice touch, but what got me truly hooked is the cost analytics to keep LLM cost under control. Well done!
@matt_roskovec Thanks Matt! Glad those metrics stuck out to you 📊 That's just the beginning of what you can do in Langtail though! Check out the tests to make sure that your prompts respond accurately too!
I have used quite a few options out there, and this is probably the nicest UI I've seen.
Now, it does lack (maybe I didn't see it) a couple of things I'd like to see:
- Can I do bulk updates from API?
- How would I unify version control with the prompts in my system.
- Prompts in actual systems are usually composed (i.e. dynamically created) so the evaluation should ideally pick up from that moment on.
- More preset evaluations, particularly for RAG evals
Really great work @petrbrzek, happy to chat if you want to brainstorm some options!
Thanks for the kind words Jorge! I'm definitely happy to chat. Now to your questions:
Yeah, we don't currently have a bulk update from the API, but we're very much flexible and can prioritize stuff which our users want to use.
In terms of version control, it depends how you currently store your prompts. You can even store them in Langtail if you want, and there's also version history. That's usually good if non-developers would like to have access to the prompts and change them as well. But on the other hand, if you store them in your code, there's currently no good way to automatically share it. I'm definitely open for ideas on how to do it nicely.
Regarding dynamically created prompts:
- There's a concept of variables which works in a way that you can send the dynamic data
- It's also possible to use function calling and function handler to get some external data, so that could be useful for testing
- We have definitely ideas for having dynamic variables so you could even use JavaScript functions for getting dynamic data for the prompt
But yeah, as I said, I'm super super happy to chat about it - we want to make the best possible experience for our testing!
👋 Hi Product Hunt community! I’m Petr, co-founder and CEO of Langtail.
We launched Langtail into public beta six months ago, and today we're excited to introduce Langtail 1.0 — a major step forward for building and testing AI apps.
Taking an LLM app from an exciting demo to a dependable product is hard work. Over the past year, we’ve heard a lot about the challenges: LLMs are unpredictable, prompt iteration is a mess, and traditional testing tools fall short.
Langtail 1.0 is our answer to these problems. It’s all about control, consistency, and confidence.
Here’s what’s new:
🧪 Spreadsheet-Like Testing Interface: Drop your test cases in with ease, get instant feedback, and iterate quickly—like using Google Sheets, but for LLMs.
🔧 Hosted Tools: Langtail now handles function execution directly within the app, making prototyping easier by running tools without needing to mock responses or set up external infrastructure.
🛠️ Test Configurations: Compare different models side-by-side with just a few clicks, making it easy to see which one works best for your app.
🔗 Shareable AI Apps: Create shareable links to let anyone interact with your LLM—no sign-up required. Perfect for getting feedback or showcasing to colleagues.
🤖 Assistants: Introducing stateful assistants that handle memory and conversation history automatically, reducing the overhead of prompt management. These assistants can be tested, deployed as APIs, and even integrated across models with ease.
✨ Magic Buttons: We've added Magic Buttons to streamline workflows—automatically generate new test cases, adjust prompts, or implement improvements with a click.
🔥 AI Firewall: Real-time protection for your app. Stop prompt injections, denial of service attacks, and data leaks before they happen.
🌟 New Redesign & Light Theme: A complete redesign to improve usability, including a new light theme for those who prefer it.
🚀 Self-Hosting Available: Full control over your data, entirely on your infrastructure. Perfect for larger teams and enterprises.
See more here https://langtail.com/blog/introd....
We’d love for you to give it a try and let us know what you think!
@petrbrzek The feature set looks very impressive, Petr! I shared the product in some communities that may find it useful. :)
Hey! I'm Tomas, co-founder and CTO of Langtail. 🚀
Quick update on our LLM provider support!
During beta launch, we only supported OpenAI. Now with Langtail 1.0, we've added all major providers:
- Anthropic Claude
- Google Gemini (includes video support!)
- Open Source models like LLama on Groq
- And more! 🤖
This means more flexibility for you to test and compare different models in one place.
Looking forward to hearing your thoughts! 💬
I was always testing LLMs quite manually, but this can be a game changer. Congrats on the launch, I will definitely try this out!
Awesome product you've built with Langtail, @petrbrzek and Team! I think this has great potential as building apps based on LLMs is not that trivial. I hope this launch boosts you!
@samuel_bissegger It's deceptively tricky to build a production-grade app using LLMs. Thank you so much for your support, Samuel!
We’ve been using Langtail for a while at Deepnote. It’s helped us improve the reliability of our text-to-SQL system, and let us quickly prototype potential new behaviors for Deepnote AI. The Langtail team’s been super responsive to our feedback, which is great. Excited for what comes next!
I’m one of the engineers on this project! Very excited to launch 1.0 after months of working on it.
Our team has thought deeply (and talked to other companies much larger than us) about testing LLM-based apps. AMA.
Maker
Yo, I'm the Product Manager behind Langtail. Thanks for the support so far. We've put a lot into this, really curious what you think of the new serverless function handling and AI assistant support. Check it out and let us know what you think! 🚀
That's true, best practices are missing and the whole community is figuring out how to do it properly.
Big big congratulations on the 1.0 launch! 🎉 Testing prompts and AI generally can be pretty cumbersome, and it looks like you guys got a really good take on it. The testing feature with multiple models and assertions seems useful and very well thought out 🚀
The proxy feature for gradually adopting existing codebases is also a great idea!
It'd be awesome to craft responseFormat directly in the Langtail interface (specifically in zod or another schema library). However, for the time being (or maybe I just missed the feature?) — it’s great that we can invoke the deployed prompt with the response format option or use the proxy as a workaround.
Thanks @susickypavel. Good idea with the responseFormat. We could definitely support it when you're only interested in text output. It gets a bit tricky with function calling. Overall, Vercel AI SDK probably has the best DX (developer experience).
Saurav from Apify here, congratulations on the launch. Looking forward to see the growth <3
@troubletony Thanks for your support, Tom! Are you building an LLM-powered app?
Huge congrats to the Langtail team on the 1.0 launch! I love how you've simplified LLM testing with a spreadsheet-like interface - who wouldn't want to tame AI complexity with a familiar feel? Quick question: Can you share an example of a surprising insight or optimization that a beta tester achieved using Langtail's analytics?
@ihuzaifashoukat Hi! Thanks for support. There are several ways to enhance the consistency of LLM responses through testing, especially when you're evaluating multiple models to optimize for performance and cost. Not all models deliver consistent results for specific requirements. You can explore our case study with our customer, Deepnote, to see how they improved their AI features using Langtail. Check it out here: https://langtail.com/customers/d.... Let me know if you have any further questions. Martin
Great to see Langtail evolving into a full-fledged testing suite for AI apps! The spreadsheet-style interface is exactly what I've been looking for - been struggling with messy prompt iterations in my recent projects. Love that you've added hosted tools and that AI firewall feature (seriously, prompt injection has been keeping me up at night 😅). The self-hosting option is a huge plus for enterprise teams who need to keep everything in-house. Feels like you guys really listened to the community pain points and delivered. Definitely giving this a spin on my next LLM project! 👍
@_ivan1 Woohoo! Glad to hear that. When you start your next project, hit us up in the chat and we'd be happy to walk you through the setup process and show you around.
Vela Terminal