Rapid point of care test for detecting urogenital Chlamydia trachomatis infection in nonpregnant women and men at reproductive age

Types of studies

We included diagnostic accuracy studies. Participants in included studies should have been enrolled prospectively and consecutively or through random sampling. Only studies reporting that all participants received the index test and the reference standard and presenting 2 x 2 data were eligible for inclusion. We excluded diagnostic case‐control studies.

Participants

We included symptomatic or asymptomatic nonpregnant women and men of reproductive age, recruited at primary or secondary care facilities, without previous diagnostic testing, co‐infections or complications. Co‐infection with both C. trachomatis and Neisseria gonorrhoeae or any other bacteria, may affect assay performance.

Index tests

Rapid tests at point of care from urine, urethral, or endocervical specimens, regardless of the type: solid‐phase EIA, solid‐phase optical immunoassay, or any other technology.

Target conditions

Urogenital C trachomatis (serovars D, Da, E, F, G, Ga, H, I, Ia, J, and K) infection.

Reference standards

NAATs of endocervical, vaginal, urethral, or urine samples. NAAT is considered the gold standard over culture for confirming the diagnosis of C trachomatis infection because of its superior diagnostic accuracy and the advantages described previously (Geisler 2011).

Electronic searches

We contacted the Cochrane Sexually Transmitted Infections' Information Specialist in order to implement a comprehensive search strategy to identify as many relevant diagnostic accuracy studies as possible in electronic databases. We used a combination of controlled vocabulary (Medical Subject Headings (MeSH), Emtree terms, DeCS, including exploded terms) and free‐text terms (considering spelling variants, synonyms, acronyms and truncation) and performed the search in the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE, Embase, and Latin American Caribbean Health Sciences Literature (LILACS) from inception up to November 2019, using search terminology for the index tests (point‐of‐care test, point of care, point of care testing, point of care devices, point of care diagnostic, point of care laboratory, POC, POCT, rapid test, rapid test device, self testing, self test, patient self testing) and the target condition (Chlamydia Infection, Chlamydia infections, Chlamydia trachomatis, sexually transmitted disease, pelvic infection, reproductive tract infection, urogenital infection, PID). In addition, we searched Web of Science (2001 to current) for conference proceedings, dissertations, and theses and the database of the World Health Organization International Clinical Trials Registry Platform (ICTRP), Health Services Research Projects in Progress (HSRProj), and the Database of Abstracts of Reviews of Effect (DARE) for additional articles. Detailed search strategies can be found in Appendix 1, Appendix 2, Appendix 3, Appendix 4, Appendix 5, Appendix 6, Appendix 7, and Appendix 8. We did not apply any language restrictions (Lefebvre 2019).

Searching other resources

We conducted handsearching of conference proceeding abstracts of the following events.

• International Society for Sexually Transmitted Diseases Research (ISSTDR; www.isstdr.org): 2007, 2009, 2011, 2013 and 2015

• British Association for Sexual Health and HIV (BASHH; www.bashh.org): from 2004 to 2018

• International Congress on Infectious Diseases (ICID; isid.org): 2010 and 2012

• International Union Against Sexually Transmitted Infections (IUSTI; www.iusti.org): 2011, 2012 and 2019

• International Society for Infectious Diseases (ISID; isid.org): 2011

• International Meeting on Emerging Diseases and Surveillance (IMED; www.isid.org): 2007, 2009, and 2011

• International Federation of Gynecology and Obstetrics (FIGO; www.figo.org): 2009, 2012 and 2015

Finally, we searched the citation lists from reviewed articles.

Selection of studies

Two review authors (CFG‐A, MT) independently applied inclusion and exclusion criteria in selecting potential titles and abstracts of studies retrieved as a result of the search. We resolved disagreements through consensus or, if required, by consultation with a third review author (HG). We retrieved the full text of a study if we had doubts about whether the study should be included or excluded.

Data extraction and management

We designed a data extraction form, which we pilot tested, to extract data from the included studies. For eligible studies, two review authors (CFG‐A, MT) extracted data independently using the form. We resolved discrepancies through consensus or, if required, by consultation with a third review author (HG). The data extraction form included the following.

• Methods

  • Country of the study. Setting

  • Basic study design

  • Power calculation

  • Number of participants and sampling of patients

  • Ethical issues

• Participants

  • Inclusion and exclusion criteria

  • Baseline information on participants: presentation at recruitment and characteristics (e.g. symptoms, presence of risk factors, sociodemographic characteristics)

• Index test

  • Point‐of‐care test specimen: urine, urethral, or endocervical

  • Point‐of‐care test technology: solid‐phase EIA, solid‐phase optical immunoassay, or any other technology.

• Outcomes

  • True positives, false positives, false negatives, true negatives

We collated and presented this information in 'Characteristics of included studies' tables. We added the data to Review Manager 5.3 (Review Manager 2014), and two review authors (CFG‐A, MT) independently assessed the accuracy of the data. We resolved differences through consensus or by evaluation by a third review author (HG). When information regarding any of the above was unclear, we contacted authors of the original reports to request further details.

Assessment of methodological quality

We assessed the quality of included articles by using the Quality Assessment of Diagnostic Accuracy Studies‐2 (QUADAS‐2) tool (Whiting 2011). Two review authors (CFG‐A, MT) independently performed the quality assessment using the four key domains to assess the risk of bias and concerns regarding applicability to the research question (participant selection, index test, reference standard, and flow and timing domains; Appendix 11). In case of disagreement, we resolved differences through consensus or by consultation with a third review author (HG).

To summarize the quality of the evidence, we defined low‐quality studies as those ranked with high risk of bias for the domains of patient sampling, index test, reference standard or flow and timing. We explored the impact of the level of bias by means of sensitivity analyses (see Sensitivity analyses).

Statistical analysis and data synthesis

We summarized diagnostic test accuracy by creating a 2 x 2 table for each study based on information retrieved directly from the papers. When there were two or more tests evaluated in the same cohort, we included them as separate data sets, since the unit of analysis was the test result, not the participant. Each table contained false‐positive, false‐negative, true‐positive, and true‐negative rates. Two review authors (CFG‐A, MT) independently entered the data into Review Manager 2014. We resolved discrepancies by consensus or, if required, by consultation with a third review author (HG).

In the first instance, we analysed in a descriptive way all data retrieved from the included studies. For this purpose and given that results of the rapid POC tests are reported qualitatively (positive or negative), we presented the results by plotting their sensitivity and specificity (and their 95% confidence intervals) both in forest plots and in a scatter plot in receiver operating characteristic (ROC) space. For the meta‐analysis of diagnostic accuracy measures, we used the bivariate model (Reitsma 2005). For studies with a common threshold, this model takes into account within‐study variation and between‐study variation and focuses on estimating a summary operating point (i.e. a summary value for sensitivity and specificity). In addition, we estimated the 95% confidence region and the 95% prediction region around the summary operating point. We performed these analyses using STATA (Stata 2017), according to the licenses available.

Investigations of heterogeneity

We explored heterogeneity initially by performing visual inspection of forest plots of sensitivities and specificities and visual examination of the prediction region. We formally assessed the source of heterogeneity by examining differences in diagnostic accuracy between subgroups of studies. Again we used the bivariate method to analyse how the summary estimate of sensitivity and specificity varies according to study‐level covariates. For this purpose, we created a factor variable with N categories and will generate an N‐1 dummy that was entered into the bivariate model to test the effects of covariates on both sensitivity and specificity (Macaskill 2010).

We defined sources of heterogeneity a priori and we included the following factors: POC specimen (endocervical versus urine or vaginal), reports of urogenital symptoms among participants (symptomatic versus asymptomatic participants), and study setting (low/middle‐income versus high‐income countries).

Sensitivity analyses

We performed sensitivity analyses taking into account the risk of bias associated with the quality of included studies based on overall 'Risk of bias' assessment according to QUADAS‐2 domains (participant sampling, index test, reference standard, and flow and timing).

Assessment of reporting bias

We assessed publication bias because we included more than ten studies in this systematic review. We initially assessed reporting bias using funnel plot visual asymmetry, plotting a measure of effect size against a measure of study precision. We then performed a formal assessment using Deeks' test and diagnostic odds ratio (DOR) as a measure of test accuracy (Van Enst 2014).

Participant selection

We assessed all studies as low risk of bias for this domain. Retrieved studies reported a consecutive sampling of participants in primary or secondary care without previous test results, making this bias unlikely.

Index test

We rated seven studies (eight study arms) as unclear risk of bias (Abbai‐Shaik 2016; Bandea 2009; Huang 2013; Hurly 2014 Acon; Hurly 2014 CRT; Rani 2002; Widjaja 1999; Yin 2006), because they did not report whether they had conducted the index test without knowledge of the results of the reference standard, or whether they had completed the index test before the reference standard was known. The other included studies described the index test appropriately, and how they conducted and interpreted it, making risk of bias unlikely.

Target condition and reference standard

Five studies did not report appropriately how they had conducted and interpreted the reference standard (Abbai‐Shaik 2016; Bandea 2009; Huang 2013; Lauderdale 1999; Rani 2002), making the risk of bias unclear for this domain. Trained operators conducted the reference standard in the remaining studies, and it was interpreted in a blind manner without knowledge of the index test results. We appraised these studies as low risk of bias, because NAAT is considered the current gold standard for the diagnosis of C trachomatis infection. For this reason, we assessed the included studies as being at unlikely risk for misclassification bias.

Flow and timing

All the included studies collected the index test and reference standard on the same participants at the same time, the whole study group received confirmation of the diagnosis by the reference standard and the study group received the reference standard irrespective of the index test results. However, we considered risk of bias to be unclear in eight studies (10 study arms), which did not provide a flow diagram of participants, did not explain withdrawals or exclusions, and were unclear as to whether the numbers recruited matched those in the analysis (Bandea 2009; Lauderdale 1999; Mahilum‐Tapay 2007; Michel 2009; Pate 1998; Rani 2002; Van Dommelen 2010 Biorapid; Van Dommelen 2010 Handilab‐C; Van Dommelen 2010 QuickVue; Widjaja 1999). For the remaining studies, it was clear what happened to all participants who entered the study and, therefore, we assessed them as low risk of bias.

Applicability

We assigned high concern in participant selection applicability to seven studies (10 study arms) because they recruited predominantly high‐risk populations (e.g. female sex workers; Michel 2009; Saison 2007 Clearview; Saison 2007 CRT), or symptomatic populations (e.g. discharge, dysuria, or abdominal pain; Huang 2013; Nadala 2009; Nuñez‐Forero 2016 Acon; Nuñez‐Forero 2016 Acon Duo; Nuñez‐Forero 2016 QuickVue; Pate 1998; Yin 2006). Five studies (seven study arms) demonstrated unclear concern inasmuch as they did not provide enough information to judge whether the participants included matched the review question in terms of severity, demographic features, presence of differential diagnosis or comorbidity, or study setting (Hesse 2011; Rani 2002; Swain 2004; Van Dommelen 2010 Biorapid; Van Dommelen 2010 Handilab‐C; Van Dommelen 2010 QuickVue; Widjaja 1999). The remaining studies posed low concern for participant selection applicability.

Regarding index test applicability, we considered five studies (seven study arms) as unclear concern because their reports had insufficient information to allow us to judge if the index test, its performance, or its interpretation differed from the review question (Abbai‐Shaik 2016; Bandea 2009; Nuñez‐Forero 2016 Acon; Nuñez‐Forero 2016 Acon Duo; Nuñez‐Forero 2016 QuickVue; Rani 2002; Widjaja 1999). We assessed one study as high concern given that, during the recruiting phase, there were multiple variations in terms of test technology (Hesse 2011). Consequently, we considered that variations in the test technology, its performance and interpretation may affect the index test's applicability. We rated the other included studies as low concern in terms of index test applicability.
 Regarding the gold standard test applicability, we judged two studies as unclear concern, given that their reports did not provide enough information to allow us to make a judgment (Abbai‐Shaik 2016; Rani 2002). We considered the other studies low concern, because the target condition (defined by the reference standard) matched the review question.

1.0 Primary objective

1.2 Secondary objectives

Heterogeneity assessment

Initially we explored heterogeneity by visual inspection of sensitivity and specificity forest plots as well as of the prediction region. Later, we formally assessed the source of heterogeneity by examining differences in diagnostic accuracy between subgroups, using the bivariate method to analyze how the summary estimate of sensitivity and specificity varied according to specimen (endocervical versus urine or vaginal), reported urogenital symptoms among participants (symptomatic versus asymptomatic participants), and study setting (low/middle‐income versus high‐income countries). The results observed as a consequence of the formal evaluation of heterogeneity sources are shown below (Table 3).

According to specimen source (endocervical or urine and vaginal)

Eleven studies (14 study arms) assessed the accuracy of POC testing of endocervical samples (Bandea 2009; Hesse 2011; Lauderdale 1999; Nuñez‐Forero 2016 Acon; Nuñez‐Forero 2016 Acon Duo; Nuñez‐Forero 2016 QuickVue; Pate 1998; Rani 2002; Sabidó 2009; Saison 2007 Clearview; Saison 2007 CRT; Swain 2004; Widjaja 1999; Yin 2006) and eight studies (13 study arms) assessed other specimen types (Abbai‐Shaik 2016; Huang 2013; Hurly 2014 Acon; Hurly 2014 CRT; Mahilum‐Tapay 2007; Michel 2009; Nadala 2009; Saison 2007 Clearview; Saison 2007 CRT; Van der Helm 2012; Van Dommelen 2010 Biorapid; Van Dommelen 2010 Handilab‐C; Nuñez‐Forero 2016 QuickVue).
 Sensitivity for endocervical specimens ranged from 0.23 to 0.64 while specificity ranged from 0.51 to 1.00. The mean sensitivity and specificity of included studies were 0.48 (95% CI 0.41 to 0.56) and 0.99 (95% CI 0.97 to 0.99), respectively. Findings were quite similar to those that were documented for POC testing performed in other specimen types (urine or vaginal). Sensitivities ranged from 0.17 to 0.93 and specificities from 0.88 to 1.00. The mean sensitivity and specificity were 0.49 (95% CI 0.32 to 0.67) and 0.97 (95% CI 0.94 to 0.98), respectively. Regardless of the specimen type, the visual inspection of forest plots for sensitivities, demonstrated a substantial degree of heterogeneity between studies and visual inspection suggested similar performance for both samples. The likelihood ratio test was not significantly different in terms of sensitivity or specificity when heterogeneity source was explored by specimen (P = 0.27; endocervical versus urine or vaginal). For this reason, this covariate did not appear to explain the variability in diagnostic accuracy.

According to the presence of symptoms of the participant (symptomatic versus asymptomatic participants)

The included studies did not provide enough information to carry out this analysis.

According to study setting (low/middle‐income versus high‐income countries)

Eight studies (12 study arms) assessed the accuracy of POC testing in low/middle income countries (Abbai‐Shaik 2016; Hurly 2014 Acon; Hurly 2014 CRT; Michel 2009; Nuñez‐Forero 2016 Acon; Nuñez‐Forero 2016 Acon Duo; Nuñez‐Forero 2016 QuickVue; Sabidó 2009; Saison 2007 Clearview; Saison 2007 CRT; Van der Helm 2012; Widjaja 1999) and 11 studies (13 study arms) in high‐income countries (Bandea 2009; Hesse 2011; Huang 2013; Lauderdale 1999; Mahilum‐Tapay 2007; Nadala 2009; Pate 1998; Rani 2002; Swain 2004; Van Dommelen 2010 Biorapid; Van Dommelen 2010 Handilab‐C; Van Dommelen 2010 QuickVue; Yin 2006).
 Sensitivities for POC testing in low/middle‐income countries ranged between 0.19 and 0.71 while specificities ranged between 0.88 and 1.00. The mean sensitivity and specificity were 0.42 (95% CI 0.32 to 0.53) and 0.98 (95% CI 0.97 to 0.99), respectively. Findings, again, quite similar to those that were documented for high‐income countries. Sensitivities ranged between 0.17 and 0.93 and specificities between 0.51 to 1.00. The mean sensitivity and specificity were 0.54 (95% CI 0.40 to 0.68) and 0.98 (95% CI 0.95 to 0.99), respectively. Regardless of the specimen type, the visual inspection of forest plots for sensitivities and specificities revealed a substantial degree of heterogeneity between studies, and visual inspection suggested similar performance for both settings. The likelihood ratio test was not significantly different in terms of sensitivity or specificity when heterogeneity source was explored by study setting (P = 0.28 low/middle versus high‐income countries). For this reason, this covariate did not appear to explain the variability in diagnostic accuracy.

Sensitivity analyses

We planned to conduct sensitivity analyses to assess the impact of the methodological quality of included studies on the results of the meta‐analysis. We defined low‐quality studies as those with high risk of bias for the domains of participant sampling, index test, reference standard, or flow and timing. However, we could not carry out these analyses because we did not rate any of the included studies as high risk of bias.

Publication bias

We assessed publication bias because this review included more than 10 studies. Initially we assessed reporting bias using funnel plot visual asymmetry, plotting a measure of effect size against a measure of study precision. This inspection did not suggest an asymmetrical distribution of the studies around the regression line. We performed a formal evaluation using Deeks’ test to investigate the asymmetry (Van Enst 2014; Figure 6). The statistically nonsignificant P value (0.74) for the slope coefficient suggests symmetry in the data and a low likelihood of publication bias. However, the test is known to have low power (Deeks 2005).

Completeness of evidence

We conducted a comprehensive search of the literature, including full‐text publications, without language restrictions or use of filters in the search strategy. Although we included studies between 2001 and November 2019, it is unlikely that any may have been missed, given that publications of POC test studies in sexually transmitted infections began to appear in 2001, and considering that a 'snowball' search was conducted as well. However, we may have missed some studies as a result of the lack of a totally reliable search strategy for finding diagnostic test accuracy studies (Beynon 2013); moreover, we were unable to find some unpublished studies through our search strategy.

Accuracy of the reference standard used

Currently, the NAAT test is considered the gold standard for C trachomatis infections. The NAAT test has a better performance than culture because it is easier to handle and has a higher sensitivity. NAAT sensitivity is 92% to 95% and specificity is 99% (Black 1997), while culture has a sensitivity of 35% to 85% and a specificity of 100% (Olshen 2005)). On the other hand, we did not detect any performance bias because the NAAT samples were taken without knowledge of the POC results, and the tests were performed using trained staff and standardized procedures. It was clear that 12 studies carried out the POC tests before the results of the standard test were known; it is unlikely that the POC tests were done after the NAAT tests, and the result of the POC tests should have been obtained within the next hour after the sample was taken.

Quality of evidence of the included studies

This systematic review follows the standard methodology suggested by Cochrane, with two independent review authors assessing the inclusion criteria for the studies, extracting data and assessing the risk of bias of the included studies, thus reducing the risk of performance bias in the review and data extraction errors. Through the quality assessment, we classified all the included studies as having low risk of bias for the QUADAS‐2 domains of participant sampling, and flow and timing. The participant selection domain was the major concern; three studies recruited predominantly high‐risk or symptomatic participants. On the other hand, we found high heterogeneity among study results, which could not be explained by the differences in sample source or the setting where the tests were done (low/middle‐income countries versus high‐income countries).
 The few published studies on seven of the nine brands did not allow us to estimate the median sensitivity and specificity or to build the summary ROC curves for comparison. Regarding the other subgroup analyses, we were able to compare the source of the sample, the performance of the tests and the setting where the POC tests were evaluated.

Comparison with other systematic reviews

We found three systematic reviews on POC testing in C trachomatis infections. Our results show a lower performance for POC tests than the systematic review by Hislop 2010, which included 13 studies. They found a higher sensitivity of 89% (95% CI 73% to 85%) for vaginal specimens and 77% (95% CI 59% to 89%) for urine specimens, with similar specificity, and only included the CRT, with the comparator reference tests being different NAATs. They provided the pooled sensitivity and specificity and assessed heterogeneity using the I² statistic.
 Herbst de Cortina 2016 found eight studies that assessed the accuracy of POC testing for C trachomatis. They evaluated 10 rapid tests ranging from Gramm stain to rapid NAATs, and all tests were compared against NAATs. They did not provide any grouped estimator of the assessed tests but the authors said that POC tests based on antigen detection showed a sensitivity and specificity lower than 75% or 50%.

Finally, in 2017, a systematic review (Kelly 2017) included 11 studies, nine of which assessed antigen detection rapid tests and two assessed rapid NAATs. Kelly 2017 obtained similar results to our review, with a pooled sensitivity of 53% for endocervical swabs, 37% for vaginal sample and 63% for male urine, and a specificity higher than 97%. Our review included a higher number of studies, without language restriction, and took into account the studies included by those systematic reviews. Two of those systematic reviews report that rapid NAATs for POC use have a sensitivity higher than 90% and a specificity of more than 95%. However, costs could restrict their use in settings with constrained resources.

Based on the results of this systematic review, the point‐of‐care (POC) test based on antigen detection has suboptimal sensitivity but good specificity. Performance of this test translates, on average, to a 52% chance of mistakenly indicating absence of infection and a 2% chance of mistakenly pointing to the presence of this condition. Because of its deleterious consequences for reproductive health, and considering the current availability of safe and effective interventions to treat Chlamydia trachomatis (C trachomatis) infection, the POC screening strategy should not be based on a rapid diagnostic test for antigen detection. Different technologies should be addressed in future research on this topic.

There is a need for well‐conducted diagnostic accuracy studies of rapid POC tests performed in a general patient population. Given that some of the included studies recruited high‐risk or symptomatic participants, future studies that evaluate the performance of the same tests could change our conclusions.
 There are uncertainties that need to be addressed as part of research in this area, including the development of tests with higher sensitivity. Given the consequences of the disease, and considering the availability of safe and effective therapeutic interventions, it is imperative to have tests that detect this infection, especially in asymptomatic populations, in order to be able to provide timely treatment. Future evaluations should include the use of different technologies (e.g. rapid NAAT tests) in different contexts including general, male and asymptomatic populations.

Abbai‐Shaik 2016.

Bandea 2009.

Hesse 2011.

Huang 2013.

Hurly 2014 Acon.

Hurly 2014 CRT.

Lauderdale 1999.

Mahilum‐Tapay 2007.

Michel 2009.

Nadala 2009.

Nuñez‐Forero 2016 Acon.

Nuñez‐Forero 2016 Acon Duo.

Nuñez‐Forero 2016 QuickVue.

Pate 1998.

Rani 2002.

Sabidó 2009.

Saison 2007 Clearview.

Saison 2007 CRT.

Swain 2004.

Van der Helm 2012.

Van Dommelen 2010 Biorapid.

Van Dommelen 2010 Handilab‐C.

Van Dommelen 2010 QuickVue.

Widjaja 1999.

Yin 2006.

Abbai‐Shaik 2016 {published data only}

Bandea 2009 {published data only}

Hesse 2011 {published data only}

Huang 2013 {published data only}

Hurly 2014 Acon {published data only}

Hurly 2014 CRT {published data only}

Lauderdale 1999 {published data only}

Mahilum‐Tapay 2007 {published data only}

Michel 2009 {published data only}

Nadala 2009 {published data only}

Nuñez‐Forero 2016 Acon {published data only}

Nuñez‐Forero 2016 Acon Duo {published data only}

Nuñez‐Forero 2016 QuickVue {published data only}

Pate 1998 {published data only}

Rani 2002 {published data only}

Sabidó 2009 {published data only}

Saison 2007 Clearview {published data only}

Saison 2007 CRT {published data only}

Swain 2004 {published data only}

Van der Helm 2012 {published data only}

Van Dommelen 2010 Biorapid {published data only}

Van Dommelen 2010 Handilab‐C {published data only}

Van Dommelen 2010 QuickVue {published data only}

Widjaja 1999 {published data only}

Yin 2006 {published data only}

Althaus 2012

Angel‐Müller 2012

Beynon 2013

Black 1997

Bossuyt 2013

Carder 2006

CDC 2002

CDC 2009

Chernesky 2005

Cook 2005

Coppus 2008

Cárcamo 2012

De Codes 2006

Deeks 2005

Domeika 2009

Gaitán‐Duarte 2013

Geisler 2011

Gottlieb 2013

Herbst de Cortina 2016

Hislop 2010

Hsu 2011

Kelly 2017

Kimani 1996

Lefebvre 2019

Low 2016

López‐Castro 2012

Macaskill 2010

Morré 2000

Navarro 2002

Olshen 2005

Papp 2014

Peeling 2006

PHAC 2010

RCOG 2008

Reitsma 2005

Review Manager 2014 [Computer program]

Romero 2017

Schwebke 1997

Soto 2007

Stata 2017 [Computer program]

Sánchez 2006

Van Enst 2014

Vickerman 2003

Whiting 2011

Workowski 2010

Grillo‐Ardila 2015