Fine-tuning sounds like ownership. Prompting sounds like a hack. In production, the opposite is often true: a versioned prompt plus retrieval is easier to audit, cheaper to change, and good enough for most ERP and SaaS tasks. Fine-tuning shines for narrow extraction and tightly scoped classifiers—not for “make the model understand our company.”
Three layers, one strategy
- Prompting & tools: instructions, function calling, structured outputs.
- RAG: fresh facts from systems of record.
- Fine-tuning / adapters: style, schema extraction, or domain jargon when examples are plentiful and stable.
Pick the cheapest layer that hits quality
| Need | Try first | Fine-tune when | Avoid |
|---|---|---|---|
| Answer from changing policies | RAG | Never for the facts themselves | Baking SOPs into weights |
| Extract invoice fields | Structured prompt | High volume, stable layout mix | One-off vendor formats |
| Brand voice in emails | Style prompt + examples | Thousands of approved emails | Customer-specific legal tone |
| Classify ticket intent | Small classifier or FT | 20+ labels, noisy language | Giant general chat model |
Hidden costs of fine-tuning
You inherit a data pipeline, evaluation harness, rollback plan, and a reason to retrain when the ERP form changes. If your labeled set is 400 examples maintained by one analyst on Fridays, you do not have a model—you have a weekend project.
A sequence that keeps options open
Ship prompting + RAG. Log failures. Cluster those failures. Only then fine-tune on the stubborn cluster (for example, packing-slip line extraction). Keep the base assistant generic so a model swap does not require rewriting the business.
Conclusion
Choose prompting and retrieval for knowledge and policy. Reserve fine-tuning for stable, high-volume patterns you can evaluate. That sequence protects budget, auditability, and your ability to ride the next model release—which, in 2026, still arrives faster than most ERP upgrades.