What You Will Learn

The goal of this book is to enable you to build your own language applications. To that end, it focuses on practical use cases, and delves into theory only where necessary. The style of the book is hands-on, and we highly recommend you experiment by running the code examples yourself.

The book covers all the major applications of transformers in NLP by having each chapter (with a few exceptions) dedicated to one task, combined with a realistic use case and dataset. Each chapter also introduces some additional concepts. Here’s a high-level overview of the tasks and topics we’ll cover:

- Chapter 1, Hello Transformers, introduces transformers and puts them into context. It also provides an introduction to the Hugging Face ecosystem.

- Chapter 2, Text Classification, focuses on the task of sentiment analysis (a common text classification problem) and introduces the Trainer API.

- Chapter 3, Transformer Anatomy, dives into the Transformer architecture in more depth, to prepare you for the chapters that follow.

- Chapter 4, Multilingual Named Entity Recognition, focuses on the task of identifying entities in texts in multiple languages (a token classification problem).

- Chapter 5, Text Generation, explores the ability of transformer models to generate text, and introduces decoding strategies and metrics.

- Chapter 6, Summarization, digs into the complex sequence-to-sequence task of text summarization and explores the metrics used for this task.

- Chapter 7, Question Answering, focuses on building a review-based question answering system and introduces retrieval with Haystack.

- Chapter 8, Making Transformers Efficient in Production, focuses on model performance. We’ll look at the task of intent detection (a type of sequence classification problem) and explore techniques such a knowledge distillation, quantization, and pruning.

- Chapter 9, Dealing with Few to No Labels, looks at ways to improve model performance in the absence of large amounts of labeled data. We’ll build a GitHub issues tagger and explore techniques such as zero-shot classification and data augmentation.

- Chapter 10, Training Transformers from Scratch, shows you how to build and train a model for autocompleting Python source code from scratch. We’ll look at dataset streaming and large-scale training, and build our own tokenizer.

- Chapter 11, Future Directions, explores the challenges transformers face and some of the exciting new directions that research in this area is going into.