What Is Fine-Tuning? Teaching an AI Model a Narrower Job
Fine-tuning takes a model that has already learned from huge data and trains it a little more on a smaller, focused set of examples. Here are the methods, uses and risks.
In this article
Fine-tuning means taking an AI model that has already been trained on a very large body of data and training it further on a smaller, focused set of examples, so that it does one job better. IBM describes it as a form of transfer learning, which is reusing what a model learned on one task for another. It is generally easier and cheaper than building a model from scratch. This article explains the main types and the trade-offs.
Starting from a pretrained model
Training a large model from nothing needs vast data and computing power. A pretrained model has already done that work and holds general patterns in its internal numbers, called weights. Wikipedia describes fine-tuning as applying additional training to those pretrained parameters, often on new data, to adapt the model from the original (upstream) task to a narrower (downstream) task. Sometimes a new output layer is added and trained from scratch while the rest starts from the pretrained values.
Why people fine-tune
IBM lists several common reasons:
- Matching a style or tone, such as a brand voice
- Specialising a general model for coding or chat
- Adding knowledge from fields like law, finance or medicine
- Handling edge cases the original training probably missed
- Using an organisation's own data
Full fine-tuning
In full fine-tuning, every weight in the network can change. Wikipedia says this often gives better results but costs more computing. IBM notes it usually uses a smaller learning rate, meaning smaller adjustment steps, so the model's earlier knowledge is not wrecked.
Cheaper ways: partial and parameter-efficient tuning
Several methods change only a small part of the model:
- Partial tuning: freeze most layers and update only some. In image networks the early layers, which capture simple features, are often frozen
- Adapters: small modules added to the network; only their weights are trained
- LoRA (low-rank adaptation): trains small matrices that represent the weight changes while the original weights stay frozen
- Prompt tuning: learns "soft prompts", which are numbers rather than words, instead of changing weights
Together, methods like these are called parameter-efficient fine-tuning. Wikipedia notes that a model with billions of parameters may be tuned by training only millions. IBM adds that LoRA updates can be swapped in and out for different tasks.
Instruction tuning and human feedback
Chat assistants are usually fine-tuned in more than one way. In supervised fine-tuning, the model learns from labelled pairs of prompts and good responses. IBM calls the version that teaches a model to follow instructions instruction tuning. Reinforcement learning from human feedback (RLHF) goes further: people rate outputs, a reward model learns those ratings, and the main model is adjusted toward higher-rated answers. Wikipedia cites it as part of how ChatGPT and Sparrow were produced.
Risks and limits
- Catastrophic forgetting: further training can erode what the model knew before; smaller learning rates and parameter-efficient methods reduce this, according to IBM
- Overfitting: a large model trained on few examples may do well on them and poorly on new inputs
- Cost: full fine-tuning of big models can be very expensive
- Robustness: Wikipedia says fine-tuning can reduce a model's robustness when real-world data differs from the tuning data
- Iteration: good results often need repeated adjustment of the dataset and settings such as batch size
Fine-tuning compared with prompting
Fine-tuning changes the model itself, while a prompt changes only what the model is asked. If a task can be handled by giving the model clear instructions and a few examples in the prompt, no training is needed, and the model stays exactly as it was. Fine-tuning is considered when a prompt alone cannot produce the style or accuracy wanted, or when the same instructions would otherwise have to be repeated in every request. IBM does not give a direct comparison with retrieval-based approaches, which fetch documents at question time, so this article does not rank them.
Another practical point is that a fine-tuned model is a separate copy or add-on that has to be stored, tested and updated. With LoRA-style methods, the added piece is small compared with the full model, which is part of why those methods are popular. The sources do not say how much data a given task needs, and results depend on the model, the quality of the examples and the settings chosen.
Fine-tuning is one option among several for shaping a model. Writing a better prompt changes the input without changing weights at all, which is why IBM describes prompt tuning as sitting between the two. This is a general explanation of the field.
Sources
Share
Keep reading
Related articles
How Neural Networks Learn: Layers, Weights, Training
A neural network is a stack of simple calculating units whose connection strengths are adjusted from examples. Here is how the layers work and how training fixes mistakes.
What Is Training Data? How AI Learns From Examples
Training data is the set of examples a machine-learning model learns from. Learn about labels, supervised and unsupervised learning, and why test data is kept apart.
What Is Model Quantization in AI? Smaller, Faster Models
Quantization stores an AI model's numbers in fewer bits so it needs less memory and can run faster. Here is how it works and what it costs in accuracy.