OpenAI launches new tools for building agents
On their livestream today, OpenAI just released a bunch of new tools for reliably building and using AI agents. From what I can tell, this is what's new-
New APIs:
Responses API - a new multi-modal API that builds on chat completions to allow for the next-generation of tool calling, starting with the new tools announced today.
New tools:
Web search tool - Gives the API access to leverage web search and cite sources
Computer use tool - Gives the API access to use a remote machine, this is huge
File search tool - Upload a batch of files to a vector store and then query them via an endpoint
I read this as we can now give API calls the ability to search the web and the return object can contain citations. This is interesting to me because it's the first time (to my knowledge) that OpenAI has surfaced it's citations via API. Amongst many other things, this would enable AISEO optimization companies and anyone gathering "AI SEO-esque ranking" data to build datasets and reverse engineer how to become cited by AI. This also seems like a bad day for @exa.ai though they are still much further along than Open AI. I have to imagine AI rank tracking tools must be coming soon in products like @Ahrefs and @semrush.
Web search tool pricing: $30 and $25 per 1K queries for GPT‑4o search and 4o-mini search.
This enables developers to use the API to to control remote environments. This is massive to be able to do this programmatically and unlocks a ton of new use cases for automation.
Computer use tool pricing: $3 per 1M input tokens and $12/1M output tokens.
The new file search tool allows you to create vector stores and upload files to them and then give the model access to these knowledge bases. Prior to this tool, you'd need to have set up a vector store and uploaded files to it. This cuts out that step. In other words, this is RAG in the easiest possible way to get there. Projects are limited to a total size of 100GB for all files but each file can be up to 512MB.
File search tool pricing: $2.50 per 1K queries and file storage at $0.10/GB/day. First GB is free.
What did I miss? What do you think? What can we build now with these tools that we couldn't yesterday? How does this impact other startups?
Replies
Impressive knowledge in the description so far, tools for search, locate and use for operational activities. Great.
My-legacy.ai
These new tools open up incredible possibilities for automation and AI-driven workflows. The Web Search tool with citations is a game-changer for AI SEO and knowledge retrieval, while the Computer Use tool could redefine how we think about AI-powered operations. Excited to see how startups and enterprises integrate these capabilities. Definitely a space to watch!
@mylegacy Agree. How are you going to use them?
Oof, the SEO reverse engineering scenario is a bummer, but it makes sense.
@leah_madden a bummer because it's possible that we'll end up with the same SEO spam in AI as with web?
Currently looking into implementing RAG into my saas so this is great news! Will have to look into how the pricing compares to other implementations though.
@steveb is the forum or the launch the better place to discuss these topics?