Natural Language Processing Tutorial
Learn Natural Language Processing step by step: 63 lessons in 11 chapters with examples, quizzes, diagrams and projects.
Start learning →
of lessons completed
Getting Started with NLP
- What Is Natural Language Processing? Meet natural language processing: the main tasks, how the field moved from hand-written rules to statistics to neural networks, and why human language is so hard for computers.
- Levels of Language: Characters to Meaning See language the way an NLP system does: characters and encodings, morphology, syntax, semantics and pragmatics, with Unicode normalization examples you can run.
- Working with Text in Python for NLP The Python text toolkit every NLP project uses: string methods, regular expression essentials, Counter for word frequencies, sorting by count and f-string reports.
- The NLP Toolkit: NLTK, spaCy, scikit-learn and Transformers A practical map of the Python NLP ecosystem: what NLTK, spaCy, scikit-learn, Gensim and Hugging Face Transformers do, when to choose each, and how to install them.
Cleaning and Tokenization
- Cleaning and Normalizing Text Clean raw text for NLP with Python: lowercasing, accents, punctuation, whitespace, HTML, URLs and emoji, a regex pipeline, and the pitfalls of over-cleaning.
- Tokenization: Splitting Text into Words Learn word tokenization from first principles: whitespace and regex splitting, punctuation, contractions, hyphens, numbers, and a hand-written Python tokenizer.
- Sentence Segmentation Split text into sentences in Python. See why a period is not enough, build a rule-based splitter that handles abbreviations, and measure it with precision and recall.
- Stop Words and When to Keep Them Learn what stop words are, how to remove them with a list or a frequency cutoff, and why blind removal breaks negation, phrases and modern neural models.
- Stemming and Lemmatization Reduce words to a common form with a hand-written Porter-style stemmer and a dictionary lemmatizer, then compare speed, accuracy and the mistakes each one makes.
- Subword Tokenization: BPE and WordPiece Understand why modern language models split words into pieces. Train a byte-pair encoding tokenizer from scratch in Python and compare it with WordPiece and SentencePiece.
- Building a Reusable Preprocessing Pipeline Combine cleaning, tokenization and filtering into one reusable NLP preprocessing pipeline with composable functions, a saved configuration, a fingerprint and tests.
Counting and Classic Representations
- Word Frequencies and Zipf's Law Count words with Counter, plot a rank-frequency curve with matplotlib, see Zipf's law in real text, and measure how vocabulary grows with Heaps' law.
- N-grams Learn what n-grams are, build word and character n-grams in Python, and use pointwise mutual information (PMI) to find collocations like New York.
- Bag of Words Turn text into numbers with the bag-of-words model. Build a document-term matrix by hand, then use CountVectorizer, n-grams and sparse matrices in Python.
- TF-IDF Learn how TF-IDF weights words by how informative they are. Compute it by hand, understand smoothing and normalization, then use TfidfVectorizer in scikit-learn.
- Feature Hashing and Large Vocabularies Handle huge vocabularies in NLP. Learn how HashingVectorizer maps words to buckets, how collisions work, and how min_df, max_df and max_features shrink features.
- Document Similarity and Cosine Distance Measure how similar two texts are. Compare cosine similarity, Jaccard and Euclidean distance on word vectors, and find the nearest documents with TF-IDF.
- Building a Small Search Engine Build a tiny search engine from scratch in Python. Create an inverted index, answer AND and OR queries, rank results with TF-IDF and handle empty queries.
Language Models Before Neural Nets
- N-gram Language Models Learn how n-gram language models assign probabilities to sentences using the chain rule and the Markov assumption, then build one and a text generator.
- Smoothing and Perplexity Fix zero probabilities with add-one smoothing, interpolation and backoff, then measure language models with perplexity on held-out text in Python.
- Spelling Correction Build a spelling corrector from scratch: Levenshtein edit distance, candidate generation and the noisy-channel model that ranks fixes by likelihood.
- Rule-Based Information Extraction Extract emails, phone numbers, dates and prices from text with Python regular expressions, and learn the pitfalls: greediness, boundaries and validation.
Text Classification
- Text Classification Overview Get the big picture of text classification: common tasks, how labels work, train/test splits, baselines and data leakage, with runnable scikit-learn code.
- Naive Bayes Text Classifier Learn Bayes' rule for text, build a multinomial Naive Bayes classifier from scratch with Laplace smoothing and log probabilities, then use scikit-learn.
- Logistic Regression and Linear SVMs for Text Train logistic regression and linear SVM classifiers on TF-IDF features, tune regularization with C, and read the top weights to see what the model learned.
- scikit-learn Pipelines for Text Chain a vectorizer and a classifier with Pipeline, combine several columns with ColumnTransformer, and tune everything with GridSearchCV and cross-validation.
- Evaluating Classifiers Learn why accuracy misleads on imbalanced data, then compute precision, recall, F1 and the confusion matrix, compare macro and micro averages, and tune thresholds.
- Sentiment Analysis Build a lexicon-based sentiment scorer with negation and intensifiers, train a machine-learning classifier, and see where both fail, from sarcasm to tiny data.
- Multi-class and Multi-label Classification Understand multi-class and multi-label text classification, build one-vs-rest models with MultiLabelBinarizer, choose thresholds and score them with the right metrics.
- Topic Modeling with LDA and NMF Discover hidden themes in unlabeled text with NMF and Latent Dirichlet Allocation in scikit-learn, read topics from their top words and choose the number of topics.
- Clustering Documents Group documents without labels. Run k-means on TF-IDF vectors, choose k with inertia and silhouette scores, and learn how to read clusters in scikit-learn.
- Keyword and Summary Extraction Pull keywords and key sentences out of a document. Rank words with TF-IDF, write TextRank by hand, and build an extractive summarizer step by step.
Sequences and Structure
- Part-of-Speech Tagging Learn POS tagging in Python: tag sets, a most-frequent-tag baseline, how a hidden Markov model resolves ambiguity, and Viterbi decoding written from scratch.
- Named Entity Recognition Learn named entity recognition: entity types, a gazetteer and regex baseline, BIO tagging, and entity-level precision, recall and F1, with runnable Python.
- Syntactic Parsing Understand syntactic parsing: constituency vs dependency trees, context-free grammars, a CKY parser written from scratch, and reading dependency trees in Python.
- Coreference Resolution and Word Sense Disambiguation Learn how NLP resolves what words refer to and which meaning they carry: a simplified Lesk algorithm and rule-based pronoun linking written from scratch.
- Dependency Parsing and Semantic Role Labeling Build an arc-standard transition parser from scratch, train a tiny scikit-learn action classifier, and map dependency trees to semantic role frames.
- Sequence Labeling with CRFs and Features Build a feature-based sequence labeler with scikit-learn: token features, context windows, BIO decoding constraints, and how CRFs add label-to-label structure.
Word Meaning and Embeddings
- Distributional Semantics Learn how a word's meaning can be read from the company it keeps. Build co-occurrence matrices, apply PPMI weighting and compare word vectors with cosine.
- Word Embeddings and Word2Vec Understand dense word vectors and the skip-gram idea behind Word2Vec. Train a tiny embedding model in NumPy and try word analogies like king minus man.
- Embeddings from SVD and Using Pre-trained Vectors Compress a PPMI matrix into dense word vectors with SVD, load GloVe and fastText files, and see how embeddings absorb bias from the text they learn from.
- Sentence and Document Embeddings Turn sentences and documents into single vectors. Average word vectors, weight them with IDF, see what averaging misses, and build a tiny semantic search.
Neural NLP and Transformers
- Neural Networks for Text Build a small neural text classifier from scratch in NumPy: embedding layer, mean pooling, hidden layer, softmax, backpropagation and a training loop.
- Recurrent Networks and LSTMs Learn how recurrent networks read text word by word, why gradients vanish, and how LSTM gates keep memory. Includes runnable NumPy forward passes.
- Attention and the Transformer Understand scaled dot-product attention, multi-head attention and positional encoding by building them in NumPy, and see how they form the Transformer.
- Pre-trained Language Models: BERT, GPT and Beyond Learn how pretraining and fine-tuning work, how encoder models like BERT differ from decoder models like GPT, and what model size means for memory and cost.
- Large Language Models and Prompting See how large language models generate text token by token, how temperature, top-k and top-p sampling work, and which prompt patterns make answers reliable.
- Retrieval and RAG Learn how retrieval-augmented generation works: chunk documents, retrieve the best passages, assemble a grounded prompt and measure retrieval quality in Python.
- Machine Translation in Depth From word alignment to Transformers: learn how machine translation works, with IBM Model 1 by EM, corpus-level BLEU and a beam search demo in plain Python.
- Translation, Question Answering and Summarization Learn the sequence-to-sequence idea behind translation and summarization, extractive question answering, and compute BLEU and ROUGE scores from scratch.
Practice and Production
- Evaluation, Bias and Responsible NLP Learn how to build trustworthy NLP systems: split data safely, measure annotator agreement with kappa, audit bias by group, redact private data and analyze errors.
- NLP in Production Ship an NLP model safely: measure latency and model size, version your artifacts, detect data drift, test text pipelines and monitor a live system.
- Speech Recognition and Text-to-Speech Learn how speech becomes text and text becomes speech: framing, spectrograms and mel features in NumPy, word error rate with edit distance, and the CTC idea.
- OCR and Text from Documents Learn the OCR-to-NLP pipeline: error types, character error rate, cleaning hyphenated scans, and correcting OCR noise with confusion-weighted edit distance.
- Chatbots and Dialogue Systems Learn how dialogue systems work: intents, slots, dialogue state and policy, with a runnable intent classifier, slot filler and booking state machine.
Projects
- Project: Spam Filter Build a spam filter end to end in Python: a labeled dataset, a Naive Bayes baseline, extra features, honest evaluation, error analysis and a safe threshold.
- Project: Movie Review Sentiment Analyzer Build a movie review sentiment analyzer in Python: a labeled dataset, lexicon and TF-IDF baselines, grid search, negation handling and model interpretation.
- Project: Document Search with Ranking Build a document search engine in Python: an inverted index, TF-IDF and BM25 ranking written by hand, and an honest evaluation with precision and MRR.
- Project: Extractive Text Summarizer Build an extractive text summarizer in Python: split sentences, score them by word frequency and TextRank, and check quality with a ROUGE-style metric.
- Project: Rule and Retrieval Chatbot Build a customer-support chatbot in Python without a language model: regex intents with slots, TF-IDF retrieval over an FAQ, a safe fallback and tests.
Practice and Review
- Practice Exercises with Solutions 15 graded NLP exercises with runnable solutions: tokenizing, normalizing, n-grams, TF-IDF, cosine similarity, edit distance, Naive Bayes and metrics.
- NLP Interview Questions 28 common NLP interview questions with concise answers: tokenization, TF-IDF, embeddings, metrics, transformers, LLMs and RAG, plus runnable code questions.
- NLP Cheat Sheet A one-page NLP cheat sheet with the text pipeline, key formulas, scikit-learn snippets, a metrics table and a guide to choosing the right model.