Skip to content
Tutorials Center
My learning

Topics

Lesson 28 of 49 ยท Genomics Pipelines

Chapter 5 Review: Genomics Pipelines

Review the short-read genomics pipeline: files, tools, formulas and quality checks from FASTQ to annotated variants, with a recap table and a ten-question quiz.

You have now followed a sample from raw instrument output to a list of annotated variants. This review pulls the chapter into one picture, one table and one set of checks you can reuse on any project. If any row of the recap table feels shaky, follow its link back to the lesson before moving on to transcriptomics.

Four file cards FASTQ, BAM, VCF and Report joined by arrows map, call and annotate, with typical tools below and quality checks along the bottom.
Caption: each arrow is a program, each box is a file format, and every stage has its own quality check.

The chapter in one table

Stage Input Output Typical tools Key numbers to remember
Quality control and trimming (lesson) FASTQ cleaned FASTQ FastQC, MultiQC, fastp, cutadapt Q = -10 log10(P); Q30 = 1 error in 1,000; ASCII offset 33
Mapping (lesson) FASTQ + reference sorted, indexed BAM BWA-MEM2, Bowtie2, minimap2, samtools 11 SAM fields; FLAG 99/147; MAPQ 0 = ambiguous
Variant calling (lesson) BAM + reference VCF bcftools, GATK, FreeBayes, DeepVariant GT 0/0, 0/1, 1/1; QUAL; Ti/Tv near 2
Annotation and filtering VCF annotated table VEP, SnpEff, ANNOVAR gnomAD frequency, consequence, ClinVar

Ideas that connect the stages

Information is carried forward, and so are errors. A base's Phred quality travels from the FASTQ into the mapper's alignment scores and into the caller's likelihoods. Garbage that enters at the beginning is expensive to find at the end. That is why the cheapest place to catch a problem is the first QC report.

Every stage is a probabilistic guess, written in the same log scale. Base quality, mapping quality and variant quality are all Phred-scaled: -10 * log10(probability of being wrong). Learn this one scale and three different columns become readable: QUAL in FASTQ, MAPQ in SAM, QUAL in VCF. They answer different questions (is this base right, is this placement right, is this site really variant), so a high value in one does not rescue a low value in another.

Depth is a budget. Mean depth N * L / G tells you how much evidence you have on average. Reads fall at random, so a 30x genome still has positions at 5x and positions at 60x. Callers can be confident at 30x and unsure at 5x. When you see a "missing" variant, ask first whether the position was covered.

Formats are contracts. FASTQ is four lines per read; SAM positions are 1-based; VCF REF and ALT anchor indels. Many bugs come from violating a contract: sorting a FASTQ with a text tool, mixing 0-based BED and 1-based VCF coordinates, or running a caller against a different reference build than the mapper used.

A reproducible checklist

Before you trust a call set, work through this list:

  1. FASTQ: Per-base quality stays high for most of the read; the adapter content is handled; no strange GC peaks.
  2. BAM: Percent mapped is high, pairs are proper, insert size has one tight peak, duplicate rate is reasonable, the sorted file has an index.
  3. Coverage: Mean depth meets the study's plan; the fraction of the target at the minimum depth is acceptable.
  4. VCF: Ti/Tv near 2 for whole-genome SNVs; heterozygous allele balance centred near 0.5; no pile of calls in a few repetitive hotspots.
  5. Provenance: Reference build, tool versions and commands are written down, ideally in a Snakemake or Nextflow workflow so anyone can rerun them.

Worked example: a puzzling result

Imagine a colleague says: "My pipeline found only 40,000 variants in a human genome, and half of them are heterozygous at around 15% alternate reads. Is that a new biological finding?" Use the chapter to reason it through.

First, 40,000 is about a hundred times fewer than the four to five million differences a typical human genome has against the reference. Either the sample is not human, the genome is mostly uncovered, or the caller's filters are far too strict. Check the mean depth and the fraction of the genome at 10x or more. Suppose the depth is a healthy 30x. Then the filters are the next suspect.

Second, heterozygous calls at 15% alternate reads do not fit a normal germline sample, where heterozygotes sit near 50%. Allele balance this low points at contamination (another person's DNA mixed in), at a mixed sample such as a tumor with normal cells, or at systematic mapping artifacts in repeats. Look at MAPQ and strand bias for those calls, and run a contamination estimate.

Third, check the provenance. Was the BAM mapped to the same reference build the VCF header claims? A mismatch in contigs (chr1 versus 1) can silently drop whole chromosomes.

The conclusion is not "a new biological finding". It is "a data-quality problem until proven otherwise". That attitude, checking depth, balance, mapping and provenance before believing a surprising result, is the single most useful habit this chapter can give you.

Questions to ask whenever someone hands you a VCF

A VCF often arrives from a collaborator or a sequencing core with the message "here are the variants". Before you analyze it, ask a short list of questions, each of which draws on a lesson in this chapter.

Which reference build and annotation release were used? Without this, coordinates are ambiguous and you cannot merge the file with other data. Which mapper and caller, with which versions and filters? Different callers disagree on indels and on low-coverage sites, and filtering thresholds change the counts by large factors. What was the mean depth, and what fraction of the target reached the minimum depth? A variant list is only meaningful alongside a statement of where the data had enough evidence to call anything at all. Was the file joint-genotyped, and does it contain reference calls? A site absent from a single-sample VCF may be homozygous reference or may be uncovered, and the file alone cannot tell you which.

Were duplicates marked and base qualities handled? This affects the evidence behind every call. How was sample identity checked? A sex check, a comparison of relatedness with the known pedigree, or a fingerprint panel can catch swaps before they cause trouble. Collaborators who can answer these questions quickly usually run careful pipelines; vagueness is itself a data-quality signal.

Where this connects

The same mapping step appears again in RNA-seq, where the mapper must also cope with reads that jump across introns. The same Phred idea returns in chapter 8, when you judge machine-learning predictions with probabilities and thresholds. And the habit of keeping every command in a script pays off in the projects chapter, where you will rebuild a miniature version of this pipeline yourself.

Try it

First, a small QC checker that turns a handful of run metrics into pass or warn flags. The thresholds are examples, not universal rules. Then a lookup widget that reminds you what each tool consumes and produces.

Python A tiny QC checker

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Which tool, which file?

Check your understanding

Quick check ยท 9 questions

of 9 answered You scored / 9 ยท perfect!

  1. 1. Which pair correctly matches a file format to its content?

    FASTQ holds raw reads, BAM alignments and VCF variants and genotypes.
  2. 2. A base with Q = 30 has an error probability of:

    10^(-30/10) = 10^(-3) = 0.001.
  3. 3. A properly paired forward first mate has which FLAG?

    99 = 1 + 2 + 32 + 64.
  4. 4. Mean depth of a 5-megabase bacterial genome sequenced with 100 million bases is:

    100,000,000 / 5,000,000 = 20.
  5. 5. A CIGAR string 8S42M has how many reference bases aligned?

    Soft-clipped bases do not align to the reference; only the 42 M bases do.
  6. 6. Why should duplicates be marked before variant calling?

    Counting copies of the same molecule as independent reads inflates confidence in a random error.
  7. 7. A whole-genome SNV call set has Ti/Tv of 0.6. What is the most likely explanation?

    Random errors produce roughly twice as many transversions as transitions, so a low ratio signals noise.
  8. 8. MAPQ 0 means:

    MAPQ is Phred-scaled confidence in the placement, and 0 means essentially no confidence.
  9. 9. Which coordinate convention do VCF and SAM use?

    SAM and VCF are 1-based; BED files are 0-based.

Summary

  • The pipeline is FASTQ, then BAM, then VCF, then annotation; each arrow is a program, each box a contract.
  • Phred scaling (-10 log10 P) is shared by base quality, mapping quality and variant quality.
  • Depth is N * L / G; coverage is random, so low-depth positions need cautious interpretation.
  • Check each stage: Q30 share and adapters, percent mapped and insert size, depth and uniformity, Ti/Tv and allele balance.
  • Write down the reference build, tool versions and commands, ideally inside a workflow manager.

Next: Chapter 6 Overview: Transcriptomics and Expression