ncbi.nlm.nih.gov

fastp: an ultra-fast all-in-one FASTQ preprocessor - PubMed

  • ️Mon Jan 01 2018

fastp: an ultra-fast all-in-one FASTQ preprocessor

Shifu Chen et al. Bioinformatics. 2018.

Abstract

Motivation: Quality control and preprocessing of FASTQ files are essential to providing clean data for downstream analysis. Traditionally, a different tool is used for each operation, such as quality control, adapter trimming and quality filtering. These tools are often insufficiently fast as most are developed using high-level programming languages (e.g. Python and Java) and provide limited multi-threading support. Reading and loading data multiple times also renders preprocessing slow and I/O inefficient.

Results: We developed fastp as an ultra-fast FASTQ preprocessor with useful quality control and data-filtering features. It can perform quality control, adapter trimming, quality filtering, per-read quality pruning and many other operations with a single scan of the FASTQ data. This tool is developed in C++ and has multi-threading support. Based on our evaluation, fastp is 2-5 times faster than other FASTQ preprocessing tools such as Trimmomatic or Cutadapt despite performing far more operations than similar tools.

Availability and implementation: The open-source code and corresponding instructions are available at https://github.com/OpenGene/fastp.

PubMed Disclaimer

Figures

Fig. 1.
Fig. 1.

Workflow of fastp. (a) Main workflow of paired-end data processing, and (b) paired-end preprocessor of one read pair. In the main workflow, a pair of FASTQ files is loaded and packed, after which each read pair is processed individually in the paired-end preprocessor, described in (b)

Fig. 2.
Fig. 2.

A demonstration of extending an adapter seed in both forward and backward directions. The found adapter is GCAAATCGATCGACT, with the first two bases (GC) as the upstream sequence, the central ten bases as the adapter seed, and the last three bases (ACT) as the downstream sequence

Fig. 3.
Fig. 3.

The base content ratio curves generated by fastp for one Illumina NextSeq FASTQ file. (a) Before fastp preprocessing, and (b) after fastp preprocessing. As depicted in (a), the G curve is abnormal and the G/C curves are separated. In (b), the G/C separation problem is eliminated

Fig. 4.
Fig. 4.

Duplication estimation. The read percentages and mean GC ratios of different duplication levels. The mean GC ratio curve is truncated since the reads with higher duplication level are too few to compute a stable mean value

Fig. 5.
Fig. 5.

Overrepresented sequence analysis results. The right column shows the histogram of occurrence among all sequencing cycles

Fig. 6.
Fig. 6.

Result of adapter trimming performance evaluation. The X-axis is the number of allowed mismatches when searching for suspected adapter sequences, and the Y-axis is the count of suspected adapter sequences

Similar articles

Cited by

References

    1. Andrews S. (2010) A quality control tool for high throughput sequence data. http://www.bioinformatics.babraham.ac.uk/projects/fastqc/.
    1. Bianchi D.W., et al. (2015) Noninvasive prenatal testing and incidental detection of occult maternal malignancies. JAMA, 314, 162–169. - PubMed
    1. Bolger A.M., et al. (2014) Trimmomatic: a flexible trimmer for Illumina sequence data. Bioinformatics, 30, 2114–2120. - PMC - PubMed
    1. Brad Chapman R.K., et al. (2018) Validated, Scalable, Community Developed Variant Calling, RNA-Seq and Small RNA Analysis, https://github.com/chapmanb/bcbio-nextgen.
    1. Chen S., et al. (2017) AfterQC: automatic filtering, trimming, error removing and quality control for fastq data. BMC Bioinformatics, 18, 80–91100. - PMC - PubMed

Publication types

MeSH terms