Module 11 of 13 in the AI Track, a free, self-directed curriculum for building AI that moves revenue.
The idea
Most of senior engineering is knowing what not to build. Here’s the call. Prompting is enough far more often than people think. RAG wins when you need fresh facts or private ones. Fine-tuning only pays off for a consistent style or format, or a narrow task you run at huge volume, and it comes with a real cost-and-maintenance bill. You also want a feel for distillation and small models as the cheap, fast workers for high-volume steps.
The trap worth keeping people out of is reflexive fine-tuning. It sounds impressive in a meeting and it’s usually the most expensive way to get a worse result.
Why it matters
Clients ask for the fancy thing. The person who says “you don’t need to fine-tune, here’s the cheaper approach that works better” earns more trust than the one who says yes to everything. Saying no is the service.
Build this
A one-page decision tree, “prompt vs. RAG vs. fine-tune,” that someone non-technical can actually use. It doubles as a blog post. It takes a call you’d make in your head and turns it into something you can hand a client.
How to work through it
- The Anthropic and OpenAI fine-tuning docs
- The relevant chapters of Chip Huyen’s AI Engineering
- Draft the decision tree with real cost notes
Resources
- Anthropic and OpenAI fine-tuning docs
- AI Engineering by Chip Huyen