Sometimes there’s a problem you feel called to solve — or one that you get deep enough into that, being stubborn, you keep working on it until either you break or the problem does. Teaching LLMs new facts has been that problem for me. I started working on it when I was doing client work last year, and I went All In on it at the start of this year. After 7 months of nonstop work, research, iteration, training, dataset generation, blood, sweat, and tears, it’s finally complete: Augmentoolkit 3.0 is out. It’s on GitHub right now. But what even is Augmentoolkit? Augmentoolkit is a production-ready way to train AI subject matter experts. It lets you update an LLM's knowledge cutoff and put new facts into its brain, without any retrival needed. You can then do reinforcement learning to improve its performance in any task you can imagine. And you can do all this locally with open-source models! It includes: - Factual finetuning: A massive data pipeline which, given some documents, will automatically generate training data that teaches an LLM the facts inside. Augmentoolkit will then automatically train an AI on those documents for you, download it, and prepare it for inference on your computer. - Data generation model: A custom dataset generation LLM built for running Augmentoolkit pipelines, allowing at-scale dataset generation on your own hardware. - Individual Alignment: an experimental GRPO training pipeline where you have the option of making an LLM be your reward model. Write a prompt to grade an LLM's output against any criteria you can think of -- by grading better responses higher, your LLM will be trained to respond more like that in the future. Finally, alignment can be done on an individual level, rather than a one-size-doesn't-fit-all approach. - Automatic RAG dataset generation: in case you still want grounding, Augmentoolkit will repurpose its generated questions and answers at the end of a data generation run into a dataset ready for powering a RAG system. It can also automatically run a RAG-powered inferene API for you to use. - Production scale: even if you generate gigabytes of data with it, Augmentoolkit's code won't break or become painfully slow. The dataset generation model, whose dataset was about 2 gigabytes large at the end, had its dataset made using an Augmentoolkit pipeline. - Easy use: making data is easy, intuitive, and fast. Augmentoolkit's start scripts mean all you need to do to get started is to run a single command. A custom-built interface allows full functionality without touching a command line or code editor. - Tools to build your own data: a whole bunch of reusable code, templates, conventions, examples, and abstractions are at your disposal for when you want to make your own dataset generation pipelines. When you want to make a custom LLM that does something that no other model does, Augmentoolkit is the place to start. - Classifier training: Augmentoolkit can bootstrap a binary classification datasets and models. -- Why this is useful -- Training an LLM on facts, rather than relying on including these facts in-context, comes with many benefits. Besides faster generation times and lower costs, an expert AI that is trained on a domain gains a "big-picture" understanding of the subject that a generalist just won't have. It's the difference between giving a new student a class's full textbook and asking them to write an exam, versus asking a graduate student in that subject to write the exam. The new student probably won't even know where in that book to look for the information it needs, and even if it sees the correct context, there's no guarantee that it understands what it means or how it fits into the bigger picture. Augmentoolkit proves that an LLM can learn even the facts of an entirely new domain through training. While the method excels at improving a model's understanding, giving it a big-picture view of a subject, the approach naturally prioritizes information that appears frequently across the training materials. Details mentioned only once or twice in a large corpus may require additional reinforcement — and Augmentoolkit gives you the tools to do this, such as by increasing the number of times data is generated from specific documents, or by grounding edge-cases with RAG. Indeed, Augmentoolkit does not necessarily compete with RAG, but can instead improve on it: LLMs trained with Augmentoolkit are trained to, if there is retrieved context, use that first -- and then, if retrieval fails, they fall back to their memorized information to try and answer questions, providing a "second line of defence" with their parametric memory. You can be confident in getting high-quality specialist models when you use Augmentoolkit. -- Why this is meaningful -- Trying to build AI apps based on closed-source LLMs released by big labs, sucks: - The lack of stable checkpoints under the control of the person running the model, makes the tech unstable and unpredictable to build on. - Capabilities change without warning and models are frequently made worse. - People building with AI have to work around the LLMs they are using (a moving target), rather than make the LLMs they are using fit into their system - Censorship and refusals force people deploying models to dance around the stuck-up morality of these models while developing. - Closed-source labs charge obscene prices, doing monopolistic rent collecting and impacting the margins of their customers. - Using closed-source labs is a privacy nightmare, especially now that API providers may be required by law to save and log formerly-private API requests. - Different companies have to all work with the same set of models, which have the same knowledge, the same capabilities, the same opinions, and they all sound more or less the same. But current open-source models either suffer from a severe lack of capability, or are massive enough that they might as well be closed-source for most of the people trying to run them. The solution? Small, efficient, powerful models that achieve superior performance on the things they are being used for (and sacrifice performance in the areas they aren't being used for) which are trained and controlled by the companies that use them. With Augmentoolkit: - Companies train their models, decide when those models update, and have full transparency over what went into them. - Capabilities change only when the company wants, and no one is forcing them to make their models worse. - People working with AI can customize the model they are using to function as part of the system they are designing, rather than having to twist their system to match a model. - Since you control the data it is built on, the model is only as censored as you want it to be. - 7 billion parameter models (the standard size Augmentoolkit trains) are so cheap to run it is absurd. They can run on a laptop, even. - Because you control your model, you control your inference, and you control your customers' data. - With your model's capabilities being fully customizable, your AI sounds like your AI, and has the opinions and capabilities that you want it to have. Now, using Augmentoolkit's factual finetuning ability, you can control what facts your AI knows, and — since opinions are just subjective facts — you decide what it believes. With the GRPO pipeline and open code, if you want to go further, then you can control every aspect of your model's capaibilities. Open-source LLMs had the promise of customization, but people and organizations needed to invest absurd time and money to even get started, with no guarantee of success. No longer. Augmentoolkit's production-ready factual finetuning is the best open-source dataset generation pipeline. It has evolved from the experience of multiple successful consulting projects. Demo models are available now for you to see some example results. Try it yourself!

Augmentoolkit 3.0

Augmentoolkit 3.0: create custom datasets and LLMs.