Skip to content
Tutorials Center
My learning

Topics

Hugging Face Transformers Tutorial

Learn Hugging Face Transformers step by step: 38 lessons in 9 chapters with examples, quizzes, diagrams and projects.

Start learning →

of lessons completed

Getting Started

  1. 1 What Is Hugging Face Transformers? Meet the Hugging Face ecosystem: the Transformers library, the Hub, Datasets and Tokenizers, and learn what you can build and how this course teaches it.
  2. 2 Installation and Environment Setup Install transformers, PyTorch and friends in a clean virtual environment, pick the right device, understand the cache folder and log in to the Hub safely.
  3. 3 The Transformer Architecture in One Lesson Understand how transformers work: embeddings, self-attention, encoder, decoder and encoder-decoder models, with a runnable NumPy attention example.
  4. 4 Your First Pipeline Use pipeline() to run a pretrained model in two lines, learn what happens inside, choose models and devices, and build a runnable stand-in of the same flow.
  5. 5 Model Families and Tasks Map tasks to model families: BERT-style encoders, GPT-style decoders, T5 and BART, vision transformers and Whisper, and learn how to choose the right size.

Hub, Models and Tokenizers

  1. 6 The Model Hub: Repositories, Model Cards and Versions Learn how the Hugging Face Hub works: Git-based repositories, model cards, files, revisions, searching and downloading models safely with from_pretrained.
  2. 7 Tokenization Basics Learn why models need tokenizers, how text becomes token ids, and the trade-offs between word, character and subword tokenization, with runnable examples.
  3. 8 Subword Tokenizers: BPE and WordPiece Learn how Byte-Pair Encoding learns merges from a corpus and how WordPiece splits words greedily, with runnable Python implementations you can edit.
  4. 9 AutoTokenizer in Practice Use AutoTokenizer for batches: padding, truncation, attention masks, token type ids, offsets, pairs of texts and chat templates, with runnable stand-ins.
  5. 10 AutoModel, Configs and Task Heads Understand AutoConfig, AutoModel and the AutoModelFor classes: body versus head, logits, softmax, label mappings, and how from_pretrained builds a model.

Pipelines and Inference

  1. 11 Pipelines for Text Tasks Use text pipelines for classification, zero-shot labels, NER, question answering, fill-mask, summarization and translation, and learn to read their outputs.
  2. 12 Pipelines for Vision and Audio Run image classification, object detection, speech recognition and audio classification pipelines, and see how images become patches and audio becomes frames.
  3. 13 Text Generation and Decoding Strategies Understand greedy search, beam search, temperature, top-k and top-p sampling, repetition penalties and stopping, with runnable NumPy demos and generate() settings.
  4. 14 Batching, Devices and Inference Speed Speed up inference with batching, sorting by length, half precision, GPU placement, KV caching and memory arithmetic, with runnable estimators.

Datasets and Preprocessing

  1. 15 Loading Datasets with the datasets Library Load datasets from the Hub and local files with load_dataset, understand Dataset and DatasetDict, Arrow memory mapping, features, indexing, filter and select.
  2. 16 Preprocessing with Dataset.map Tokenize datasets with Dataset.map, use batched processing, remove columns, handle long texts with overflowing chunks and align token labels.
  3. 17 Splits, Streaming and Custom Data Create reliable train, validation and test splits, avoid data leakage, stream huge datasets, build a Dataset from your own records and push it to the Hub.

Fine-tuning with Trainer

  1. 18 The Trainer and TrainingArguments Learn how the Hugging Face Trainer runs the training loop for you: TrainingArguments, steps and epochs, evaluation, logging, learning-rate schedules, with runnable stand-ins.
  2. 19 Data Collators and Dynamic Padding Understand how data collators batch tokenized examples: dynamic padding, attention masks, label padding, masked language modeling and length grouping, with runnable stand-ins.
  3. 20 Custom Training Loops with Accelerate Write your own fine-tuning loop when Trainer is not enough: gradient accumulation, clipping, schedulers and Hugging Face Accelerate for multi-GPU, with runnable stand-ins.
  4. 21 Fine-tuning for Text Classification Fine-tune a pretrained transformer for text classification: classification head, label mappings, logits and softmax, freezing layers and a complete Trainer script, with stand-ins.
  5. 22 Checkpoints, Callbacks and Early Stopping Save, resume and manage training checkpoints, use Trainer callbacks and EarlyStoppingCallback, keep the best model and avoid overfitting, with runnable stand-ins.

Efficient Fine-tuning and Quantization

  1. 23 Parameter-Efficient Fine-Tuning (PEFT) Overview Why full fine-tuning gets expensive and how parameter-efficient methods (adapters, LoRA, prefix tuning, prompt tuning, BitFit) cut trainable parameters, with runnable counting stand-ins.
  2. 24 LoRA Explained: Low-Rank Adaptation Understand LoRA step by step: the low-rank update W + BA, rank and alpha, target modules, zero initialization and merging, with numpy demos and the peft code you would use.
  3. 25 Quantization Fundamentals Learn how quantization shrinks models: bits and number formats, absmax and zero-point schemes, per-tensor versus per-channel scales, outliers and error, with runnable numpy demos.
  4. 26 QLoRA and bitsandbytes: 4-bit Fine-tuning Fine-tune large language models on one GPU with QLoRA: 4-bit NF4 quantization, double quantization, paged optimizers and the BitsAndBytesConfig code, with numpy demos of the ideas.
  5. 27 Memory Math and Mixed Precision Training Estimate GPU memory for training and inference, understand fp16 versus bf16, loss scaling, gradient checkpointing and batch size trade-offs, with runnable calculators.

Evaluation and Metrics

  1. 28 The Evaluate Library and Metric Basics Compute metrics with Hugging Face Evaluate: load, compute, add_batch, combine, compute_metrics for Trainer, and how a metric works inside, with a runnable mini version.
  2. 29 Classification Metrics: Precision, Recall, F1 Evaluate classifiers properly: confusion matrix, accuracy pitfalls, precision, recall, F1, macro versus micro averaging, thresholds and a Trainer compute_metrics function.
  3. 30 Text Generation Metrics: BLEU, ROUGE, Perplexity Learn how BLEU, ROUGE, perplexity, exact match and token F1 are computed, what they miss, and how to evaluate generated text, with runnable pure-Python implementations.
  4. 31 Error Analysis, Calibration and Benchmarks Go beyond one accuracy number: slice errors, read confusion matrices, measure calibration with ECE, and learn how benchmark scores for language models are produced and misread.

Deployment and Spaces

  1. 32 Saving, Pushing and Documenting Models Save a fine-tuned model with save_pretrained, understand the safetensors file format, push to the Hugging Face Hub, and write a model card that makes your model usable and safe.
  2. 33 Serving a Model with FastAPI and Batching Wrap a Hugging Face pipeline in a FastAPI service, load it once, validate input, and use dynamic micro-batching to raise throughput. Includes runnable latency simulations.
  3. 34 Gradio Demos and Hugging Face Spaces Turn a Transformers pipeline into a shareable web demo with Gradio and host it free on Hugging Face Spaces: app files, requirements, secrets, hardware, caching and limits.
  4. 35 Optimized Inference: ONNX, Optimum and Compiled Models Make Transformers inference cheaper: export to ONNX, use Optimum and ONNX Runtime, fuse operators, quantize weights, compile with torch.compile, and benchmark honestly.

Projects and Practice

  1. 36 Project: A Support-Ticket Classifier, Data to Demo Build a complete text-classification project: data splits, tokenization, a baseline, fine-tuning with Trainer, error analysis, saving and a demo, with a runnable numpy baseline.
  2. 37 Project: Semantic Search with Embeddings Build a semantic search engine: chunk documents, embed with mean pooling, rank by cosine similarity, evaluate recall@k, and scale up. Real code plus a runnable numpy stand-in.
  3. 38 Troubleshooting and Practice A field guide to common Transformers problems (shape errors, NaN loss, out of memory, wrong labels, slow training), a debugging routine, a loss-curve doctor, and practice tasks.