This is the second half of practical 10, done here instead of in Galaxy. Two people, a mother and her child, each sequenced from genomic DNA enriched for mitochondria by long-range PCR — so the files still carry a good deal of nuclear DNA, which in this experiment is contamination. The job is to map what is there against the mitochondrial genome, throw out the reads that landed badly, call what differs, and then work out what fraction of the copies in each person carries it.
FastQC is Java and cannot run in a tab, so the read check and the trimming are fastp. bwa is not compiled for the browser yet, so the mapping is minimap2 with its short-read preset. FreeBayes is not either, so the calling is bcftools. Nothing here is a house-made lookalike wearing somebody else's name, and every number below says which tool produced it.
The practical's own slides for this dataset: what heteroplasmy is, and where the reads came from — pages 13-14 of MBT5011_DP_P10_260627.pdf, from your own copy. Nothing is fetched for you.
A page becomes a picture when you insert it, so this notebook needs no PDF reader to show it. It stops following the deck at that moment, which is why each one records the file and page it came from.
First the reference. GRCh38 is 3.26 GB and a browser cannot hold it, but it does not need to: the FASTA has a .fai beside it, the index turns a locus into a byte offset, and the host answers a range request. And chrM is 16,569 bases, so this is not a window onto a genome — it is the whole mitochondrial genome, fetched entire, for about seventeen kilobytes.
This genome is 3.26 GB. Nothing downloads it: the index beside it says which bytes hold your locus, and only those bytes are asked for.
samtools faidx https://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa chrM:1-16569 > chrM.fa
Reading chrM.fa…
@read_0042Name, always starts with @+A separator lineIIIIFFFF,,,,##!One quality score per baseNow the reads, which are the practical's own. Download the four files from https://zenodo.org/record/1251112/files — raw_child-ds-1.fq, raw_child-ds-2.fq, raw_mother-ds-1.fq and raw_mother-ds-2.fq, about 14 MB each — and bring them onto the page with the cells below.
Zenodo serves those files without an Access-Control-Allow-Origin header, so no page in any browser may fetch them for you — not this one, and not Galaxy's uploader running in your tab either. Downloading and bringing the file in is the honest route, and it is the same first step the practical gives you. Nothing leaves your machine: the file goes into this page's workspace and no further.
Reading raw_child-ds-1.fq…
Reading raw_child-ds-2.fq…
Reading raw_mother-ds-1.fq…
Reading raw_mother-ds-2.fq…
Look at what came off the sequencer before mapping any of it. Quality falls away at the ends of Illumina reads, and a tail of bad bases is how a mapper is made to place a read somewhere it does not belong — which matters more than usual here, because a variant present in three copies out of a hundred has to be told apart from an error present in three reads out of a hundred.
No reads on this page yet. Bring a FASTQ in through a file cell, and it will appear here.
fastp -i raw_child-ds-1.fq -I raw_child-ds-2.fq -O raw_child-ds-2.trimmed.fq --detect_adapter_for_pe -o raw_child-ds-1.trimmed.fq -j raw_child-ds-1.fastp.json -h raw_child-ds-1.fastp.html -q 20 -l 50 --cut_tail
raw_child-ds-1.fq and raw_child-ds-2.fq through fastp (q 20, length 50, tails cut), into raw_child-ds-1.trimmed.fq
Before fifty thousand reads, one. The cell below pulls the first record out of what fastp just wrote, and the one after it puts that single read beside the reference and names every base where the two differ. Nothing in the practical gives you this look, and it is the one that makes a CIGAR string mean something.
Nothing to align yet. A trace cell or a reference cell above this one can write its result into the page, and it will appear here.
minimap2 -a -x sr chrM.fa one-read.fq > one-read.sam
If that read did not place at all, that is a result and not a failure: these samples are full of nuclear DNA by design, and a read from chromosome 4 has no business on chrM. Change RECORD in the cell above to look at another one, or map the lot below and see what fraction of them lands.
Now the mapping and the filtering, as commands. The practical does these as four Galaxy forms — BWA-MEM, merge, MarkDuplicates, Filter BAM — and the two that carry the argument are the mapping and the filter, so those are the ones written out here. Paste them into the terminal below, one line at a time.
Every tool cell on this page shows the command it ran and exports it, and anything a cell does you can type here instead. The child's sample goes through the instruments and the mother's goes through this terminal on purpose: they are the same pipeline, and neither one is the real version.
Map the child's trimmed pair against the mitochondrial genome. BWA-MEM in the class; minimap2's short-read preset here, under its own name.
minimap2 -ax sr chrM.fa raw_child-ds-1.trimmed.fq raw_child-ds-2.trimmed.fq > child.sam
The practical's Filter BAM step, as the two numbers it really is: mapping quality 20 or better, and properly paired only. Then sort what survived into position order, and ask samtools how much of it there is.
samtools view -h -q 20 -f 2 child.sam > child.filtered.sam samtools sort child.filtered.sam -o child.sorted.sam samtools flagstat child.sorted.sam
Now the mother's sample, the same way, from the files you brought in.
fastp -i raw_mother-ds-1.fq -I raw_mother-ds-2.fq -o m1.fq -O m2.fq -q 20 -l 50 --cut_tail minimap2 -ax sr chrM.fa m1.fq m2.fq > mother.sam samtools view -h -q 20 -f 2 mother.sam > mother.filtered.sam
And call hers. FreeBayes in the class; bcftools here. This is exactly what the caller cell below does to the child's sample, which is the point of doing it by hand once.
samtools sort mother.filtered.sam -o mother.sorted.bam samtools index mother.sorted.bam bcftools mpileup -Q 20 -f chrM.fa mother.sorted.bam > mother.pileup.vcf bcftools call -mv mother.pileup.vcf > mother.vcf
The class's mapping steps: BWA-MEM, read groups, and merging the two samples — pages 15-17 of MBT5011_DP_P10_260627.pdf, from your own copy. Nothing is fetched for you.
A page becomes a picture when you insert it, so this notebook needs no PDF reader to show it. It stops following the deck at that moment, which is why each one records the file and page it came from.
Type a command, or click the suggested one. ↑/↓ recall history, Tab completes file names. New to this? Hit Explain this command to see it in plain English.
The Filter BAM step as the class sets it: mapping quality 20, proper pairs only — pages 21-22 of MBT5011_DP_P10_260627.pdf, from your own copy. Nothing is fetched for you.
A page becomes a picture when you insert it, so this notebook needs no PDF reader to show it. It stops following the deck at that moment, which is why each one records the file and page it came from.
-q 20 keeps reads whose mapping quality says there is a one-in-a-hundred chance or less that they are in the wrong place; nuclear DNA that half-matches a mitochondrial gene fails it. -f 2 keeps only properly paired reads, which drops the singletons and the pairs that landed in impossible places. The practical sets both in a form. They are the same two numbers.
How much of it mapped, how deep it is, how much is duplicate. With mitochondria-enriched DNA the mapped fraction is the interesting number: it says how well the enrichment worked, and it is the first thing anybody will ask you about this dataset.
No alignment on this page yet. An alignment cell above this one writes a SAM into the page, and it will appear here.
samtools stats child.sorted.sam > child.sorted.stats
A table saying there is a difference at a position is a claim; the reads stacked under it, with a minority carrying another base, is the evidence. Set the region to whatever position the caller flags below and come back to look at it.
No alignment on this page yet. An alignment cell above this one writes a SAM into the page, and it will appear here.
samtools tview -d T child.sorted.sam chrM.fa
Now let the caller decide. bcftools mpileup walks the alignment and reports what it saw at each position; bcftools call decides which of those are variants. The cell sorts and indexes first, because no caller will look at an unsorted alignment.
No alignment on this page yet. An alignment cell above this one writes a SAM into the page, and it will appear here.
samtools sort child.sorted.sam -o child.sorted.bamsamtools index child.sorted.bamsamtools faidx chrM.fabcftools mpileup -Q 20 -f chrM.fa child.sorted.bam > child.pileup.vcfbcftools call -mv child.pileup.vcf > child.vcf
Reading child.vcf…
The class's VCFfilter step, with SPR, SAP, EPP, QUAL and DP defined — pages 30-33 of MBT5011_DP_P10_260627.pdf, from your own copy. Nothing is fetched for you.
A page becomes a picture when you insert it, so this notebook needs no PDF reader to show it. It stops following the deck at that moment, which is why each one records the file and page it came from.
QUAL is the caller's confidence the site is real at all. DP is how many reads it saw; a heteroplasmy read off twenty reads is a rumour. Strand bias is the one that matters most here: a real variant appears on reads from both strands in roughly the proportion the reference allele does, and an artefact of the chemistry or the mapping often appears on one strand only.
The counting step the class ends on: the VCF as a table, and the reads carrying each allele — pages 35-36 of MBT5011_DP_P10_260627.pdf, from your own copy. Nothing is fetched for you.
A page becomes a picture when you insert it, so this notebook needs no PDF reader to show it. It stops following the deck at that moment, which is why each one records the file and page it came from.
This is the practical's last step, and where it is worth going one further. Counting how many reads carried each allele gives a percentage; what it does not give is how well that percentage is known. Ninety reads out of a hundred and nine out of ten are both ninety per cent, and only one of them would survive somebody asking. So the cell below counts, and then puts an interval round the count, and where both samples were called it asks whether the two really differ.
Has anybody seen these positions before? dbSNP is 29.5 GB and the answer costs a few hundred kilobytes, because its tabix index turns a locus into a byte range: the database stays at NCBI and the question goes to it. The mitochondrial genome is one of the best described stretches of DNA there is, so most of what a healthy person carries here already has a name.
No called variants on this page yet. A variant cell above this one writes a VCF into the page, and it will appear here.
bcftools annotate -a https://ftp.ncbi.nlm.nih.gov/snp/latest_release/VCF/GCF_000001405.40.gz -c ID child.vcf
It says the position has been described. It does not say the variant is causal, or pathogenic, or the reason for anything in front of you. And a variant with no rs number is not thereby novel — it may be a position dbSNP has not reached, or a mistake in your alignment.
The practical ends this section by installing IGV and opening the files in it, and its first half ends by opening JBrowse. This is IGV — igv.js, the same code base, under its own name — over the reference, the reads and the calls already on this page. The cell sorts and indexes the alignment first, and shows you those commands and an IGV batch script that reproduces the same view on the desktop.
No alignment on this page yet. An alignment cell or a terminal line above this one writes a SAM or BAM into the page, and it will appear here.
samtools sort -o child.sorted.bam child.sorted.sam
samtools index child.sorted.bam
cat > igv.batch <<'BATCH'
new
genome chrM.fa
load child.sorted.bam
load child.vcf
goto all
BATCH
# igv -b igv.batch
One step past where the practical stops. Thirteen of the mitochondrial genome's genes are proteins, all of them parts of the respiratory chain, and a base that changes one of them changes a residue in a machine you can look at. The cell below folds MT-CO1, cytochrome c oxidase subunit 1, by its UniProt accession; change the accession to whichever gene your own call fell in.
url=$(curl -s 'https://alphafold.ebi.ac.uk/api/prediction/P00395' | python3 -c 'import json,sys; print(json.load(sys.stdin)[0]["pdbUrl"])')
curl -sL "$url" -o MT-CO1.cif
# Open MT-CO1.cif in PyMOL, ChimeraX, or https://molstar.org/viewer/
AlphaFold's model of P00395, kept as MT-CO1.cif
Everything above is compute, and compute is the cheap half. None of these tools decided whether the enrichment worked well enough to trust a three per cent allele, or whether a difference between mother and child is a real germ-line bottleneck or the two libraries having been made on different days. That judgement is the whole job, and it is the part the form in Galaxy hides by having a button for everything else.
The reads are downsampled, so the depths here are not the depths of the original experiment. Duplicates are not marked and indels are not left-aligned; both are Galaxy steps with no compiled equivalent in the tab yet, and both matter more for indels than for the substitutions this page ends up looking at. The practical's first half — Staphylococcus aureus, wildtype against mutant, Snippy and JBrowse — is the same chain over a bacterial genome, and every tool in it except Snippy itself already runs here.