Hey everyone, excited to do our second launch on Product Hunt! It's been really exciting to see the massive interest in AI this year! There's a lot of cool things people are building with foundation models like ChatGPT, Claude, and Stable Diffusion. However, it still feels like we're pretty early in figuring out how to use AI in the right way - over the last few months, we've talked to a lot of people in the space, and a theme that we noticed is that the work of an AI developer is now about building a product *around* an ML model to create a compelling user experience. Anecdotally: I heard that Github Copilot took many rounds of iterating on how the interface should be constructed and how the workflow should look before it "clicked" and became a really addictive (and useful) tool to use, but most of this work took place on the Copilot product and not on the GPT model that powered it. In web development, there's a suite of product analytics tools that helps us refine product-market fit. These tools operate on user interactions in visual interfaces: tracking where users clicked, how they moved from one view to another, and constructing graphs / funnels / queries to dig into these events. Using these tools, we can analyze user behavior, understand usage patterns and patterns of engagement / dropoff, and figure out what changes to make to improve the product. In LLM apps, users can work iteratively with the software via a natural language interface, generating user inputs and model responses consisting of unstructured text. Traditional product analytics techniques don't deal well with large amounts of unstructured text. It's hard to summarize, it's hard to aggregate, and it's hard to effectively sample. AI developers resort to digging through a pile of hundreds to hundreds of millions of datapoints of unstructured text to understand how users interact with their product. That's why we built Tidepool! Tidepool does product analytics for AI text interfaces. It uses neural network embeddings to help product teams find patterns in their user text interactions and make better product decisions for their AI apps. After you upload user text interaction events, Tidepool will: - Automatically group your data by similarity. Tidepool runs embedding clustering on your users’ text interactions to surface interesting attributes: things like prompt topics, prompt languages, and common usage patterns that can be turned into shortcuts. - Summarize common attributes in your data, using LLMs to determine what each cluster “contains.” For example, understanding that the most common topics that users discuss are business, education, and art. - Track attributes in production traffic, allowing you to uncover how a specific attribute might be correlated to good / bad product outcomes. We utilize lightweight models running on foundation model embeddings to scalably extract these attributes from hundreds of millions of production interaction events. Tidepool then integrates into existing product analytics stacks so it’s easy for teams to get started: - Public API. Developers can easily import / export user activity data via a REST API. - CDP Support. We also support common CDPs (e.g. Segment) for production scale integration. - Integration with existing tooling. Whether you use Segment, Hightouch, Census, or want to send events directly from your app, Tidepool works with your data and the tools you already use. Over the last few years, we've worked a lot with computer vision companies to help them curate labeled datasets and improve their fine-tuned models using our core embedding technology. Our mission has always been to make it easier for people to build and improve production ML systems that solve real-world problems. When we saw the Cambrian explosion of LLM apps earlier this year, we realized that our core embedding technology and expertise was very useful for getting these new apps to product-market fit even faster! We put this product together over the last few months after working with a few select design partners. We're now at the point where Tidepool been useful for helping teams like Mutiny figure out what to do next to make their AI products better. Now we're releasing this to the world with the hope that this is a useful tool in your toolbox, and we're excited to do our part to support this wave of world-changing AI applications!

Peter Gao

Tidepool by Aquarium

Maker

📌

2yr ago

Cyril Gupta

CloudFunnels AI

Looks interesting! gonna try it out

@cyriljeet Glad to hear it! Let us know how it goes :)

Tom Bachant

Unthread

This looks great Peter! We're building AI features with unstructured data and similarly have low visibility into what's happening under the hood. Excited to check it out!

Tidepool by Aquarium

Product analytics for AI applications

Do you use Tidepool by Aquarium?

Review Tidepool by Aquarium?