Skip to content
Tutorials Center
My learning

Topics

Lesson 53 of 53 ยท Practice and Review

Generative AI Cheat Sheet

A one-page Generative AI cheat sheet: key terms, sampling parameters, prompt patterns, RAG and evaluation checklists and common LLM failure modes.

This page packs the course into one place: the terms, settings, patterns and checklists you will use every day when you build with language models. Use it as a refresher before writing a prompt, while debugging a bad answer, or right before an interview.

Each section links to the lesson that explains the topic in depth. Three small editors let you try the most useful snippets: a prompt skeleton, a token and cost estimator, and the sampling filters. For practice, see the exercises; for spoken explanations, the interview questions.

The pipeline at a glance

Tokenize Text becomes token ids.
Predict The model scores every next token.
Sample Temperature, top-p and friends pick one.
Repeat Until a stop token or a limit.

Everything else is built on this loop.

1 / 2

Key terms

Term Meaning
Token The unit a model reads and writes: a word, word piece or character. See Tokens.
Vocabulary The fixed set of tokens a tokenizer knows (often 30,000 to 250,000).
Embedding A vector of numbers that represents a token or a piece of text. See Embeddings.
Logits The raw scores for every possible next token, before softmax.
Softmax Turns logits into probabilities that sum to 1.
Context window The most tokens (prompt plus answer) a model can handle at once.
Attention The mechanism that lets each token weigh every earlier token. See Attention.
KV cache Stored keys and values of earlier tokens, so generation does not recompute them.
Pretraining Learning to predict the next token on a huge corpus.
Fine-tuning Further training on a smaller, targeted dataset.
RLHF / DPO Tuning with human preference data so answers are more helpful and safe.
LoRA Fine-tuning by training small low-rank matrices instead of all weights.
Quantization Storing weights with fewer bits to save memory.
RAG Retrieving documents and adding them to the prompt before answering.
Agent A loop where a model picks tools, sees results and decides what to do next.
Hallucination A fluent answer that is not supported by facts or sources.
Prompt injection Untrusted text that tricks a model into following instructions.
Perplexity exp of the average negative log-probability per token; lower means less surprised.

Rules of thumb for size and cost

  • One token is about 4 characters, or three quarters of an English word. Other languages and code often use more tokens per word.
  • You pay for input and output tokens separately, and output usually costs several times more per token.
  • Every request must fit prompt + system + retrieved context + answer <= context window. Reserve room for the answer.
  • KV cache memory grows linearly with context length and with the number of simultaneous conversations.
  • Attention compute grows with the square of the prompt length.
  • Count tokens with the provider's tokenizer before you trust an estimate. See Counting Tokens and Estimating Cost.
Python Quick token and cost estimator

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Sampling parameters

All of these reshape the next-token distribution before one token is drawn. See Logits, Softmax and Temperature, Top-k, Top-p and Min-p and Beam Search and Penalties.

Parameter Effect Typical use
temperature Divides logits; low is sharp and predictable, high is flat and varied 0 to 0.3 for facts, extraction and code; 0.7 to 1 for writing
top_p Keeps the smallest set of tokens whose probabilities add up to p 0.9 to 1.0; change this or temperature, not both
top_k Keeps only the k most likely tokens Where offered, 20 to 100
min_p Drops tokens below a fraction of the top token's probability Adaptive alternative to top-k
max_tokens Hard limit on the length of the answer Always set it; it caps cost
stop Strings that end generation Delimiters, end of a list item
frequency_penalty Subtracts more for every repeat of a token Reduce repetition
presence_penalty Subtracts once a token has appeared Encourage new topics
seed Makes sampling repeatable (best effort) Debugging and tests
logit_bias Adds or subtracts a value for chosen tokens Force or ban specific words
Python Temperature, top-k and top-p in one function

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Prompt patterns

See The Anatomy of a Good Prompt, Few-Shot Prompting, Reasoning, Structured Output and Prompt Templates.

Pattern Use it when Sketch
Role and task Almost always "You are a support agent. Answer the customer's question."
Constraints You need limits "Use at most 3 sentences. Do not mention pricing."
Output format Code consumes the reply "Reply with JSON: {"label": ..., "confidence": ...}"
Delimiters Input is untrusted or long <document>...</document> around the data
Few-shot examples The format or style is hard to describe 2 to 5 balanced input and output pairs
Step by step Multi-step reasoning "Think through the steps, then give the final answer on its own line."
Self-check Accuracy matters "Check each claim against the context and remove unsupported ones."
Allow refusal Facts may be missing "If the context does not contain the answer, say you don't know."
Cite sources Answers must be verifiable "After each claim add the source number, like [2]."
Python A reusable prompt skeleton

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

RAG checklist

See RAG: Giving Models Your Data and the lessons after it.

  • Documents are cleaned (no navigation, boilerplate or duplicates) and carry metadata such as source, title and date.
  • Chunks follow natural boundaries, have modest overlap and a size you tested (Chunking).
  • The same embedding model is used for documents and queries, and its name is stored with the index.
  • Search is hybrid (keywords plus vectors), with optional reranking of the top results (Hybrid Search).
  • Filters (tenant, language, date) are applied before the model sees anything, so users never get other users' documents.
  • The prompt says to answer only from the context, to cite sources and to admit when the answer is missing (Citations).
  • The context budget is enforced: fewer, better chunks beat many mediocre ones.
  • Index updates are planned: re-embedding changed documents, deleting removed ones.
  • Retrieval and answers are evaluated separately.

Evaluation checklist

See Evaluating LLM Applications, LLM-as-Judge and Metrics.

  • A test set of realistic inputs exists, including hard, rare and adversarial cases.
  • Each case has an expected answer or a clear scoring rule.
  • Checks that can be exact (regex, JSON schema, exact match) are exact; judges are used only where needed.
  • Judge prompts use a rubric, randomize answer order and are compared with human labels.
  • Retrieval is measured with recall@k and MRR; generation with faithfulness and citation checks.
  • Scores come with a confidence interval, and test sets are big enough for the differences you care about.
  • The whole suite runs on every prompt, model or data change, and failing cases are read, not just counted.
  • Production traffic feeds new test cases back in.

Common failure modes

Symptom Likely cause First fix
Confident wrong facts Hallucination, missing knowledge Ground with retrieval, allow "I don't know" (Hallucinations)
Ignores part of the instructions Too many or conflicting instructions, buried in a long prompt Shorten, order by importance, split into steps (Debugging Prompts)
Broken JSON No schema, high temperature, prose around the JSON Schema plus validation plus retry
Repeats itself Low temperature, greedy decoding Add a frequency penalty, raise temperature a little
Misses facts in a long prompt Information in the middle of a long context Put key facts first or last, retrieve less
Bad retrieval Chunks too big or small, embeddings miss exact terms Retune chunking, add keyword search
Follows instructions found in documents Prompt injection Treat content as data, least privilege (Security)
Different answers each run Sampling randomness Lower temperature, set a seed, test with several runs
Costs grow unexpectedly Long prompts, big outputs, retries Budget tokens, cache, route to smaller models (Cost and Caching)
Agent loops or wanders No step limit, unclear tools Step and cost limits, better tool descriptions (Agent Loop)

Check your understanding

Quick check ยท 3 questions

of 3 answered You scored / 3 ยท perfect!

  1. 1. You need a model to extract fields from invoices reliably. Which setting is the best starting point?

    Extraction wants predictable output: low temperature, a clear format and code that validates the reply.
  2. 2. Which problem is best solved with retrieval rather than fine-tuning?

    Retrieval brings current, private knowledge in at question time and lets you cite it.
  3. 3. Roughly how many tokens is a 1,200-character English paragraph?

    Using about 4 characters per token, 1,200 characters is about 300 tokens.

Summary

  • A model turns text into tokens, scores the next token, samples one and repeats; every other idea builds on that loop.
  • Set sampling by task: low temperature for facts, higher for creative work, and always cap max_tokens.
  • Prompts work best with a role, a task, constraints, a format and delimiters around untrusted data.
  • RAG needs clean chunks, hybrid search, grounded prompts and separate retrieval and answer evaluation.
  • Evaluate with a real test set, honest statistics and a review of failing cases.
  • Most failures have a known cause and a first fix; check the table before changing models.

Back to the start: revisit What Is Generative AI? or build something with Project: A Chatbot with Memory.