Aprendizado por transferência
Definição
Transfer learning reutiliza conhecimento de uma tarefa ou domínio de origem para melhorar o aprendizado em uma tarefa-alvo com dados limitados. Modelos pré-treinados (por ex. ImageNet, BERT) are fine-tuned on downstream tasks.
É standard in NLP (por ex. BERT, GPT) and vision (por ex. ImageNet backbones). When the target has little labeled data, starting from a source model and fine-tuning on target data is much more data-efficient than training from scratch. See few-shot and zero-shot for the extreme of very few or no target examples.
Como funciona
Obtain a source model (pré-treinado em a large dataset, por ex. ImageNet or web text). Take target data (your task’s labeled examples) and fine-tune: update the model (all parameters or only a subset, por ex. adapter, head) to minimize loss on the target task. O resultado é um target model. Full fine-tuning updates all weights; adapter or prompt tuning updates a small number of parameters to save compute and preserve source knowledge. Works best when source and target share useful representations (por ex. same modality, related domains).
Casos de uso
Transfer learning is standard when you have limited target data and a related pretrained model to adapt.
- Fine-tuning BERT or GPT on domain-specific NLP tasks
- Adapting ImageNet-pretrained models to medical or satellite imagery
- Reusing pretrained representations when target data is limited