A Founder’s Guide to AI Fine-Tuning

Introduction

1. What is fine-tuning?
2. Why fine-tune? What problems does it solve?
3. When is the right time to think about fine-tuning?
4. What tools and expertise do I need to be successful?

What is AI Fine-Tuning?

Why Fine-Tune?

1. Unreliable adherence to instructions. For example, if you ask an LLM to create a summary that is under 50 words, it might still produce a 60+ word summary 20% of the time. This becomes even more true with instructions that are nuanced and subjective.
2. High costs for operations at scale. Current frontier model costs are low enough that they usually can be justified for realtime user interactions. But for large operations like sifting through millions of documents or scraping large parts of the web, they can quickly become cost prohibitive.
3. Latency that is too high for certain applications. If your app requires multiple rounds of LLM calls in eg. an agentic workflow, or near-realtime responses for a text-to-voice stream, frontier models may not respond fast enough provide a good experience.

1. A good dataset leads to far greater reliability because the model is strongly conditioned to create outputs that correspond to your needs.
2. Fine-tuned models are generally much smaller than generalist models. This leads to much lower inference costs (often a 10-100x improvement).
3. Since fine-tuned models are much smaller, they also can have much lower latency than large generalist models.

When Should You Fine-Tune?

1. Fine-tuning specializes a model on a specific shape of input. But if you’re just starting off, you probably don’t know exactly what your input shape will look like! You need actual users using your system to see what inputs they will bring, so you can specialize on that.
2. Fine-tuning does take some time and money (even if it’s minimal, as we’ll see below!). So it doesn’t make sense to reach for it until you’ve exhausted the limits of easier approaches like prompt engineering.

1. Prototype Quickly with GPT-4: When exploring an idea or seeking product-market fit, using an off-the-shelf model like GPT-4 makes sense. You can iterate rapidly without worrying about custom infrastructure.
2. Scale with Fine-Tuning: Once you’ve identified your core use case and are ready to scale, fine-tuning becomes valuable. A fine-tuned model can be:

How to Fine-Tune

• Inner Loop: Rapid, iterative evaluations using a “golden dataset” or having an LLM act as a judge.
• Outer Loop: Business metrics—how well is the fine-tuned model performing in terms of user satisfaction and task success?

Is Fine-Tuning Right for You?