We provide low cost API to our hosted models, enabling businesses to prioritize delivering products that cater to their customers' specific needs. With our support, you can craft exception products while we take care of intricate details!

Congrats with the launch! I have one question though: can you provide more details on how the pricing structure works for accessing the hosted flan-t5-small model through the API? Is it based on usage duration, the number of API calls, or some other metric, and are there any specific pricing tiers or options available?

Congratulations on the launch. This is very good timing, there are a lot of startups which are based on using LLMs in different scenarios. This will help people focus on the product.

Hi Everyone, Maruti.io provides hosting solution for Large Language Models, and reduces development hours. So that you can focus on providing maximum value to your customers. We are working on features such as fine-tuning, faster inference times, analytics, model evaluations, and retrieval augmented generation. Why am I doing this? The idea for this project came from me wanting to host a Open Source Large Language Model. Hosting a model such as T5, or any other model can get expensive really fast, along with all the analytics that need to be put in place. Especially if you don't have access to a performant GPU. Solution: To solve this problem, and allow makers, founders and creators access to a Large Language Model with high performance, low costs, and using lower compute power so that you can focus on building a product that can stand out! Features: Get access to a hosted flan-t5-small model in no time. You can access it via an easy to use API. You only pay for what you use, so you don't have to worry about constantly running a Virtual Machine and paying absurd GPU costs. We will always fine-tune and update our model, so you get a model that is always improving. How do we do this? We use a smaller model that can run efficiently on a CPU. We train it further to be more performant, and get closer to existing Large Language Models in terms for performance.

@sagarpatel10 congrats for your launch! Your hosting solution for LLMs sounds promising. I'm curious about a few details: 1. Can you provide more information about the cost structure for your hosting service? What are the pricing options available to users? 2. How do you ensure the security and privacy of the data and models hosted on your platform? 3. Could you elaborate on the fine-tuning process you mentioned? How frequently do you update and fine-tune the models, and can users have input into this process? 4. What programming languages and frameworks are supported for making API calls to access the hosted model?

@michelepomposo Of cource Michele 1. So there are two plans. There is a free tier which gives you 1 project API key, and 1,000 API calls per month. 2. We use API keys to authenticate requests. On the dashboard side, we make sure the customer is logged in, and only return project data for customer that have created those projects. The analytics dashboard shows all the input traffic for a given project. So that they know what is going into their systems. 3. So the fine-tuning process is still being worked on, but it allows users to fine-tune models on their data using this service. We then return the model back to the user in a file. We also fine-tune all our models to increase their performance. Users can also request a model to be hosted on this service. 4. Since the API calls will be made over a network, rather than locally users can use any programming language that they might be using to make a POST request to our url. There is documentation on how to do that with python. I will expand it for more languages in future.

@sagarpatel10 thank you for the detailed answers! Don't you mind to share the python docs?

@michelepomposo Of course! Here are the docs! This is like a quick start guide, so it has everything from creating a project, to then running the python code. https://maruti.io/docs

Well done, Sagar! Wish you good luck with the launch! The idea is amazing!

@ayhan_dzhemalov Thank you Ayhan! I appreciate the kind words! :)

Congratulations on the launch. This is very good timing, there are a lot of startups which are based on using LLMs in different scenarios. This will help people focus on the product.

@leon_novacki Thank you Leon! Your support is much appreciated!

Maruti.io

API for open source Large Language Models

Do you use Maruti.io?