Processamento de linguagem natural (NLP)
Definição
O NLP abrange tarefas sobre texto: classificação, NER, QA, resumo, tradução e geração. Modern NLP is dominated by pretrained transformers (BERT, GPT, etc.) and LLMs.
Inputs are discrete (tokens); models learn from large corpora and are then adapted via fine-tuning or prompting. RAG and agents add recuperação and tools on top of NLP models for grounded QA and task completion.
Como funciona
Texto é tokenizado (dividido em subpalavras ou palavras) e opcionalmente normalizado. O modelo (por ex. BERT, GPT) processes token IDs through embeddings and transformer layers to produce contextual representations. A task output head (por ex. classifier, span predictor, or next-token decoder) maps those to the final prediction. Models são pré-treinados em large corpora (masked LM or previsão do próximo token), then fine-tuned or prompted for downstream tasks. Pipelines often combine tokenization, embedding, and task-specific heads; LLMs can do many tasks with a single model and the right prompt.
Casos de uso
NLP applies to any product or pipeline that needs to understand or generate text at scale.
- Machine translation, summarization, and question answering
- Named entity recognition, sentiment analysis, and text classification
- Chatbots, code generation, and document understanding