OpenAI launches new tools for building agents
On their livestream today, OpenAI just released a bunch of new tools for reliably building and using AI agents. From what I can tell, this is what's new-
New APIs:
Responses API - a new multi-modal API that builds on chat completions to allow for the next-generation of tool calling, starting with the new tools announced today.
New tools:
Web search tool - Gives the API access to leverage web search and cite sources
Computer use tool - Gives the API access to use a remote machine, this is huge
File search tool - Upload a batch of files to a vector store and then query them via an endpoint
I read this as we can now give API calls the ability to search the web and the return object can contain citations. This is interesting to me because it's the first time (to my knowledge) that OpenAI has surfaced it's citations via API. Amongst many other things, this would enable AISEO optimization companies and anyone gathering "AI SEO-esque ranking" data to build datasets and reverse engineer how to become cited by AI. This also seems like a bad day for @exa.ai though they are still much further along than Open AI. I have to imagine AI rank tracking tools must be coming soon in products like @Ahrefs and @semrush.
Web search tool pricing: $30 and $25 per 1K queries for GPT‑4o search and 4o-mini search.
This enables developers to use the API to to control remote environments. This is massive to be able to do this programmatically and unlocks a ton of new use cases for automation.
Computer use tool pricing: $3 per 1M input tokens and $12/1M output tokens.
The new file search tool allows you to create vector stores and upload files to them and then give the model access to these knowledge bases. Prior to this tool, you'd need to have set up a vector store and uploaded files to it. This cuts out that step. In other words, this is RAG in the easiest possible way to get there. Projects are limited to a total size of 100GB for all files but each file can be up to 512MB.
File search tool pricing: $2.50 per 1K queries and file storage at $0.10/GB/day. First GB is free.
What did I miss? What do you think? What can we build now with these tools that we couldn't yesterday? How does this impact other startups?
