QIIME 2 paired-end 16S amplicons skill

Processes paired-end 16S amplicon reads into QIIME 2 ASVs and taxonomy with retained artifact provenance.

by K-Dense-AI·MIT license·★ 45,977 Stars on the repo·GitHub ↗

Use now

Files of QIIME 2 paired-end 16S amplicons

K-Dense-AI/main1 file shown
SKILL.md
Show the full text133 lines

QIIME 2 paired-end 16S amplicons

Use for a demultiplexed, paired-end 16S assay with known primers, quality encoding, expected insert length, and sample metadata. This bounded workflow imports reads, removes 5′ primers, denoises with DADA2, classifies ASVs using an explicitly supplied classifier, and retains .qza/.qzv provenance. Do not treat read counts as absolute cell counts or taxonomy assignments as strain identification.

Establish the assay before running

  • Confirm Phred+33, read orientation, primer sequences as sequenced in forward/reverse reads, and whether primers have already been removed. The bundled runner requires primers still present at the 5′ ends; it anchors Cutadapt matching and discards untrimmed pairs. Already trimmed reads need a direct import → DADA2 workflow with that omission recorded in provenance.
  • Choose truncation positions from actual per-base quality and error profiles. trunc-f/r are positions after primer removal. The expected maximum insert length also excludes primers. Require trunc_f + trunc_r - maximum_insert_length >= 12; use a margin for length variation. The check predicts geometrical overlap, not successful biological merging.
  • Choose a classifier whose reference database, taxonomic coverage, orientation and training approach fit the assay. Full-length classifiers are supported; primer-region-specific training is not mandatory. Match its scikit-learn version exactly to the installed environment (the official 2026.7 distribution pins 1.7.1). Record source URL, database version and checksum. QIIME's current data-resources page links externally hosted classifiers for 2026.4 and later; its older downloads are not automatically compatible. Do not automatically fetch an arbitrary “latest” classifier or reuse an incompatible serialized sklearn model. Load sklearn classifier artifacts only from trusted sources: QZA format validation does not make an untrusted serialized model safe.
  • Include extraction blanks, PCR negatives, and a mock community where available. The runner rejects empty FASTQs; preserve empty-control IDs separately and report them rather than silently deleting control evidence. Assess contamination before ecological interpretation.

Input files

Manifest is a tab-separated PairedEndFastqManifestPhred33V2 file with exactly these headers:

sample-id	forward-absolute-filepath	reverse-absolute-filepath
sample1	/data/sample1_R1.fastq.gz	/data/sample1_R2.fastq.gz

This is the helper's deliberately narrow manifest profile. Use actual tab characters, literal absolute paths visible to the runtime (expand environment variables before calling this helper), and one row per sample; no comment/directive rows or additional columns in this manifest. Do not reverse-complement R2 files. Sample metadata is a tab-separated file with first column sample-id, unique IDs matching the manifest, and optional #q2:types annotation. Include covariates and biological replicate IDs needed downstream. The helper checks metadata IDs and row structure; QIIME performs full metadata typing/directive validation during execution. Metadata used by actions persists in artifact provenance, so use de-identified biological replicate IDs.

Execute

The helper lives at scripts/amplicon_workflow.py. Commands below assume the skill directory is the working directory. First validate without QIIME. These example primers and lengths are illustrative, not universal assay settings:

python scripts/amplicon_workflow.py validate \
  --manifest manifest.tsv --metadata sample-metadata.tsv \
  --primer-f GTGYCAGCMGCCGCGGTAA --primer-r GGACTACNVGGGTWTCTAAT \
  --trunc-f 220 --trunc-r 200 --amplicon-max 300

Then run in the QIIME 2 2026.7 environment with a compatible classifier. This study-specific invocation is illustrative; choose lengths using a preceding quality inspection or pilot:

python scripts/amplicon_workflow.py run \
  --manifest manifest.tsv --metadata sample-metadata.tsv \
  --primer-f GTGYCAGCMGCCGCGGTAA --primer-r GGACTACNVGGGTWTCTAAT \
  --trunc-f 220 --trunc-r 200 --amplicon-max 300 \
  --classifier compatible-classifier.qza --threads 4 --output run01

run executes immediately, writes only to a fresh output directory, and stops on a failing QIIME command. It streams through every paired FASTQ record to catch mismatched IDs/order/counts, checks sequence/quality consistency and sample alignment, and profiles the first 1,000 read pairs for exact IUPAC primer matches. Low exact-match rates are warnings: Cutadapt allows mismatches, but a low rate can also indicate incorrect orientation, adapters, or already-trimmed data. At least one complete pair per sample must meet nominal post-primer truncation lengths. These length estimates subtract the stated primer lengths; they do not simulate Cutadapt indels or quality filtering, and do not establish that any pair will actually merge.

The runner uses --output-dir for plugin methods with evolving output sets, preserving Cutadapt statistics and DADA2 base-transition artifacts when supplied by the release. The 2026.7 table summary also produces feature-frequencies.qza and sample-frequencies.qza beside table.qzv. It records qiime-info.txt, commands.json, workflow.log, input QC, classifier checksum and output artifact checksums. It runs maximum-level QIIME artifact validation before reporting completion. See references/runtime-and-interpretation.md for the release-pinned runtime, actual validation scope, restart handling and scientific interpretation.

Inspect results before analysis

Open trimmed.qzv, table.qzv, and taxa.qzv in a local QIIME visualization environment or QIIME 2 View as appropriate for the data. Examine quality/length profiles, per-sample depth and dominant taxa. Retain original artifacts rather than replacing them with CSV/BIOM exports: exports do not retain the original provenance graph.

retention-qc.json compares raw pairs with DADA2 input and non-chimeric reads, so trimming losses remain visible. A <50% retained fraction is a review heuristic, not a universal rejection rule. Inspect the individual stages in stats/stats.tsv: filtering loss suggests quality/expected-error settings; loss after forward denoising includes reverse-denoising and merging failures; chimera loss warrants reviewing library quality and parameters. Investigate missing/zero samples and control behavior before rarefaction, diversity, or differential abundance. Those downstream analyses need a separate design decision; this skill does not choose a rarefaction depth automatically. The standalone retention stats.tsv subcommand knows only DADA2 input counts: it reports raw_pairs: null and names that denominator explicitly. It supports merged-only paired DADA2 statistics, as produced by this runner; it rejects retained-unmerged/concatenated-read statistics.

Primary references

The rolling documentation may describe a development release. Inspect qiime info and action --help in the exact installed environment before adapting the pinned runner to a later release.

1---
2name: qiime2-amplicon
3description: Processes paired-end 16S amplicon reads into QIIME 2 ASVs and taxonomy with retained artifact provenance. Checks paired FASTQ manifests, primer orientation diagnostics, predicted post-trimming overlap, sample IDs, runtime versions and read retention, and guides selection of compatible taxonomic classifiers.
4license: MIT
5compatibility: Python 3.10+ for the standard-library validation helper; QIIME 2 2026.7 distribution with cutadapt, dada2, demux, feature-table, feature-classifier, taxa and types plugins for execution. Install through the official conda/container distribution, not PyPI. Requires a compatible trusted classifier and network for installation/reference downloads.
6metadata:
7 version: "1.1"
8 skill-author: K-Dense Inc.
9 upstream-version: "2026.7"
10 last-reviewed: "2026-10-01"
11---
12 
13# QIIME 2 paired-end 16S amplicons
14 
15Use for a demultiplexed, paired-end 16S assay with known primers, quality encoding, expected insert
16length, and sample metadata. This bounded workflow imports reads, removes 5′ primers, denoises with
17DADA2, classifies ASVs using an explicitly supplied classifier, and retains `.qza`/`.qzv` provenance.
18Do not treat read counts as absolute cell counts or taxonomy assignments as strain identification.
19 
20## Establish the assay before running
21 
22- Confirm Phred+33, read orientation, primer sequences **as sequenced** in forward/reverse reads,
23 and whether primers have already been removed. The bundled runner requires primers still
24 present at the 5′ ends; it anchors Cutadapt matching and discards untrimmed pairs. Already
25 trimmed reads need a direct import → DADA2 workflow with that omission recorded in provenance.
26- Choose truncation positions from actual per-base quality and error profiles. `trunc-f/r` are
27 positions **after primer removal**. The expected maximum insert length also excludes primers.
28 Require `trunc_f + trunc_r - maximum_insert_length >= 12`; use a margin for length variation.
29 The check predicts geometrical overlap, not successful biological merging.
30- Choose a classifier whose reference database, taxonomic coverage, orientation and training
31 approach fit the assay. Full-length classifiers are supported; primer-region-specific training
32 is not mandatory. Match its scikit-learn version exactly to the installed environment
33 (the official 2026.7 distribution pins 1.7.1). Record source URL, database version and checksum.
34 QIIME's current data-resources page links externally hosted classifiers for 2026.4 and later;
35 its older downloads are not automatically compatible. Do not automatically fetch
36 an arbitrary “latest” classifier or reuse an incompatible serialized sklearn model. Load sklearn
37 classifier artifacts only from trusted sources: QZA format validation does not make an
38 untrusted serialized model safe.
39- Include extraction blanks, PCR negatives, and a mock community where available. The runner
40 rejects empty FASTQs; preserve empty-control IDs separately and report them rather than silently
41 deleting control evidence. Assess contamination before ecological interpretation.
42 
43## Input files
44 
45Manifest is a **tab-separated** `PairedEndFastqManifestPhred33V2` file with exactly these headers:
46 
47```text
48sample-id forward-absolute-filepath reverse-absolute-filepath
49sample1 /data/sample1_R1.fastq.gz /data/sample1_R2.fastq.gz
50```
51 
52This is the helper's deliberately narrow manifest profile. Use actual tab characters, literal
53absolute paths visible to the runtime (expand environment variables before calling this helper),
54and one row per sample; no comment/directive rows or additional columns in this manifest.
55Do not reverse-complement R2 files. Sample metadata is a tab-separated file with first column
56`sample-id`, unique IDs matching the manifest, and optional `#q2:types` annotation. Include
57covariates and biological replicate IDs needed downstream.
58The helper checks metadata IDs and row structure; QIIME performs full metadata typing/directive
59validation during execution. Metadata used by actions persists in artifact provenance, so use
60de-identified biological replicate IDs.
61 
62## Execute
63 
64The helper lives at [scripts/amplicon_workflow.py](scripts/amplicon_workflow.py). Commands below
65assume the skill directory is the working directory. First validate
66without QIIME. These example primers and lengths are **illustrative**, not universal assay settings:
67 
68```bash
69python scripts/amplicon_workflow.py validate \
70 --manifest manifest.tsv --metadata sample-metadata.tsv \
71 --primer-f GTGYCAGCMGCCGCGGTAA --primer-r GGACTACNVGGGTWTCTAAT \
72 --trunc-f 220 --trunc-r 200 --amplicon-max 300
73```
74 
75Then run in the QIIME 2 2026.7 environment with a compatible classifier. This study-specific
76invocation is illustrative; choose lengths using a preceding quality inspection or pilot:
77 
78```bash
79python scripts/amplicon_workflow.py run \
80 --manifest manifest.tsv --metadata sample-metadata.tsv \
81 --primer-f GTGYCAGCMGCCGCGGTAA --primer-r GGACTACNVGGGTWTCTAAT \
82 --trunc-f 220 --trunc-r 200 --amplicon-max 300 \
83 --classifier compatible-classifier.qza --threads 4 --output run01
84```
85 
86`run` executes immediately, writes only to a fresh output directory, and stops on a failing QIIME
87command. It streams through every paired FASTQ record to catch mismatched IDs/order/counts,
88checks sequence/quality consistency and sample alignment, and profiles the first 1,000 read pairs
89for exact IUPAC primer matches. Low exact-match rates are warnings: Cutadapt allows mismatches,
90but a low rate can also indicate incorrect orientation, adapters, or already-trimmed data.
91At least one complete pair per sample must meet nominal post-primer truncation lengths.
92These length estimates subtract the stated primer lengths; they do not simulate Cutadapt indels
93or quality filtering, and do not establish that any pair will actually merge.
94 
95The runner uses `--output-dir` for plugin methods with evolving output sets, preserving Cutadapt
96statistics and DADA2 base-transition artifacts when supplied by the release. The 2026.7 table
97summary also produces `feature-frequencies.qza` and `sample-frequencies.qza` beside `table.qzv`.
98It records
99`qiime-info.txt`, `commands.json`, `workflow.log`, input QC, classifier checksum and output artifact
100checksums. It runs maximum-level QIIME artifact validation before reporting completion.
101See [references/runtime-and-interpretation.md](references/runtime-and-interpretation.md) for the
102release-pinned runtime, actual validation scope, restart handling and scientific interpretation.
103 
104## Inspect results before analysis
105 
106Open `trimmed.qzv`, `table.qzv`, and `taxa.qzv` in a local QIIME visualization environment or
107[QIIME 2 View](https://view.qiime2.org/) as appropriate for the data. Examine quality/length profiles,
108per-sample depth and dominant taxa. Retain original artifacts rather than replacing them with
109CSV/BIOM exports: exports do not retain the original provenance graph.
110 
111`retention-qc.json` compares raw pairs with DADA2 input and non-chimeric reads, so trimming losses
112remain visible. A <50% retained fraction is a review heuristic, not a universal rejection rule.
113Inspect the individual stages in `stats/stats.tsv`: filtering loss suggests quality/expected-error
114settings; loss after forward denoising includes reverse-denoising and merging failures; chimera loss warrants reviewing library
115quality and parameters. Investigate missing/zero samples and control behavior before rarefaction,
116diversity, or differential abundance. Those downstream analyses need a separate design decision;
117this skill does not choose a rarefaction depth automatically.
118The standalone `retention stats.tsv` subcommand knows only DADA2 input counts: it reports
119`raw_pairs: null` and names that denominator explicitly. It supports merged-only paired DADA2
120statistics, as produced by this runner; it rejects retained-unmerged/concatenated-read statistics.
121 
122## Primary references
123 
124- [Current installation entry point](https://library.qiime2.org/quickstart/qiime2) and
125 [amplicon documentation](https://amplicon-docs.qiime2.org/en/latest/).
126- [Import formats](https://amplicon-docs.qiime2.org/en/latest/how-to-guides/how-to-import/).
127- [Cutadapt actions](https://amplicon-docs.qiime2.org/en/latest/references/plugins/cutadapt/) and
128 [DADA2 actions](https://amplicon-docs.qiime2.org/en/latest/references/plugins/dada2/).
129- [Classifier data resources](https://library.qiime2.org/data-resources).
130 
131The rolling documentation may describe a development release. Inspect `qiime info` and action
132`--help` in the exact installed environment before adapting the pinned runner to a later release.
133 

Discussion