OpenAI-compatible endpoint. Single API, routes to the cheapest and fastest provider for each model. Works with closed and open LLMs. Real-time benchmarks (price, latency, load) run in the background. Usable direclty now on Roo and Cline forks
Subscribe
Sign in
MakeHub.ai
Really exciting! The real-time arbitrage feature sounds like a game-changer for optimizing cost and performance. How does it handle model compatibility across different providers, especially with closed LLMs?
MakeHub.ai
@evgenii_zaitsev1 It does very well on those subject, we spent a lot of time uniformising everything from tool call to prompt caching, so overall it works as if you had only a single API key. Closed LLMs were particularly difficult because most of them have their own framework, and act like they handle openai compatible endpoints but it doesn't work on all their features, so we have to create proxys to do that. Hoping that I answered your question!