A Performance Evaluation of an Optoelectronic Cervical Screening Device in Comparison to Cytology and HPV DNA Testing

Objective : An optoelectronic screening device (OESD) is evaluated for the detection of cervical intra-epithelial neoplasia (CIN) 2+ lesions in comparison to Liquid Based Cytology (LBC) and high-risk HPV DNA (hrHPV) testing. Methods : In total 506 consecutive women referred because of abnormal cervical cytology or a positive high-risk HPV test, had an examination using OESD, LBC, and hrHPV testing. They were screened in 4 colposcopy clinics in New South Wales, Australia. In a retrospective audit, results were compared to the gold standard of colposcopy and biopsies if required. Sensitivity, specificity, area under the receiver operating characteristic (ROC)-curves, and differences using McNemar tests were calculated. All results were available for comparison on 474 patients. Results : The sensitivity to detect CIN II+ lesions by OESD, LBC and hrHPV-testing was 0.72, 0.81, and 0.88, and the specificity was 0.71, 0.95, and 0.76 respectively. The age- and previous-treatment adjusted area under the ROC curve for OESD was 0.83, for LBC 0.94, and for hrHPV testing 0.89. McNemar’s tests showed no significant difference in sensitivity between OESD and LBC ( p = 0.26), and no significant difference in specificity between OESD and hrHPV-testing ( p = 1.0) amongst patients without previous treatment. Conclusions : The optoelectronic screening device demonstrated comparable sensitivity to high quality cytology conducted in a hospital clinical setting. Specificity was comparable to hrHPV-testing in an approximate primary screening setting. OESD has the advantage of producing an immediate result and being easy to use without need of laboratory equipment. This device can potentially become an important tool in the prevention of cervical cancer, particularly in developing countries and resource-limited settings.


Introduction
Cervical cancer is the third most common cancer in females worldwide, with an estimated 570,000 new cancer cases annually. In developing countries, where more than 85% of the new cervical cancer cases occur, it is the second most common cancer among women after breast cancer [1][2][3][4]. A persistent infection with high risk human papillomavirus (hrHPV) in the uterine cervix has been established as the primary cause of cervical cancer [5][6][7] . Cervical cancer has detectable premalignant stages, which offer major opportunities for screening, early treatment of precancer and cancer and a consequent reduction in cancer incidence and mortality [8,9] . In developed countries, introduction of organised exfoliative cytology based screening programs have led to marked reductions in the incidence of cervical cancer [10][11][12] . In addition, improved screening programs using hrHPV testing and the introduction of HPV vaccination programmes leading to reduced incidence of high-grade precancerous cervical lesions suggest that cervical cancer incidence rates will fall even further [13] .
In low-and middle-income countries (LMI), successful implementation of organised cervical screening often fails due to lack of financial support and properly trained cytologists, and poor laboratory and program support services [14,15] . Until a prophylactic HPV vaccine covering all hrHPV types becomes available, there will continue to be a need to screen and treat women for cervical premalignant lesions. Cervical screening alternatives which are simpler to implement, acceptable to women and cost-effective are still being researched. One of these alternatives is a realtime optoelectronic device. These devices are handheld and use electrical and optical signals to classify cervical tissue into normal and abnormal. They provide an immediate result without the need for laboratory facilities or qualified cytologists. Earlier studies reported real-time optoelectronic devices to be safe and feasible [16][17][18].
In this retrospective audit, we evaluated the performance of the optoelectronic cervical screening device Tr-uScreen as a single screening method to detect cervical intraepithelial neoplasia (CIN 2+). A comparison was made to the performance of liquid-based cytology (LBC) and high-risk HPV DNA testing in the same women in a research setting with colposcopy and histology of colposcopically directed biopsies as the gold standard. Sensitivity and specificity were assessed, and any adverse effects of the optoelectronic screening device were recorded and evaluated.

Materials and Methods
From June until December 2017, consecutive women with an abnormal pap smear referred to the colposcopy clinic at the Royal Hospital for Women (RHW) in Sydney, to the Orange Aboriginal Medical Service (OAMS) in Orange, to Tottenham Multipurpose Hospital (TMH) in Tottenham, or to Pius X Aboriginal Medical Service (PXAMS) in Moree, all in New South Wales Australia, consented to be screened with the optoelectronic cervical screening device TruScreen as part of an assessment of its clinical performance. This screening was in addition to the standard hr-HPV and LBC screening, and colposcopic examination.
Women over the age of 18 years, who had agreed to the additional procedure were eligible. Exclusion criteria were current menstrual period, current or recent pregnancy (within 4 months post-delivery), Pap smear within 6 weeks, surgical treatment to the cervix within the past 3 months, previous pelvic radiation, chemotherapy in the previous 5 weeks, clinically apparent acute or subacute cervical infection, photosensitizing disease, and previous hysterectomy.
All women were first screened using the TruScreen Handheld Device, following which a sample was taken for LBC and hrHPV testing. All women then had a colposcopic examination performed by an experienced colposcopist, and abnormal areas were biopsied.
The results for the TruScreen device were immediately available and categorized as: normal (normal squamous epithelium, columnar epithelium, physiologic metaplasia, or latent HPV-related changes) or abnormal (CIN I-III, invasive cancer). All liquid based cytologic and histologic specimens were processed by the department of Anatomical Pathology, South-Eastern Area Laboratory Service (SEAL) in Prince of Wales Hospital, Sydney. LBC samples were classified according to the Australian Modified Bethesda System [19]. Results of biopsies were categorized as normal, CIN I, CIN II, CIN III, AIS, and invasive carcinoma (squamous, adenosquamous, adenocarcinoma).
The samples were tested for high risk HPV DNA using Cobas HPV Assay (Roche Cobas 4800) and were categorized as: HPV negative, HPV 16 positive, HPV 18 positive, HPV positive "other". The results of the TruScreen test were only known to the team present at the clinical examination. All complications or adverse outcomes were recorded.

Description of TruScreen
The optoelectronic TruScreen device measures physical properties of tissue. By comparing characteristics of the tissue of interest with the behaviour of known tissue types, the device can categorise tissue. In this way, it can detect pre-cancerous and cancerous changes in the cervix. Two types of physical measurements are used, optical and electrical.
The device is composed of a hand-held probe coupled with a wireless electromagnetic induction Qi charging cradle. The instrument is approximately 37 cm in length from base to tip (Figs. 1,2,3). The section of the probe that enters the vagina is 120 mm in length with a tip diameter of approximately 5 mm. The system incorporates a single-use sensor (SUS) -encompassed in a sheath which covers the probe of the handpiece, increasing the tip diameter to approximately 6.5 mm.   The tip of the probe interrogates the tissue by repetitively pulsing it with low levels of optical and electrical energy. Real-time interpretation of the cervical tissue response is achieved by automatic comparison with a digitally stored catalogue of tissue signatures. The device measures the directly reflected light, backscattered light and electrical decay curves of the cervical tissue. It assesses the response of surface epithelial cells, but also identifies changes in the epithelial basal layer and stromal cells. These changes include enlarged cell nuclei, increased cytoplasmic density, increased blood circulation and variations in blood vessels, and changes that occur with neoplastic lesions.
The device is powered by a lithium ion battery with approved patient protection and delivers several electrical pulses of millisecond duration. Twenty-seven pulses are delivered per "observation", and fourteen observations are made per second. These very low energy pulses are below normal sensation thresholds.
The optical measurements operate within the visible and near infrared spectrum. The light emitting diodes (LEDs) have a power output range of 20-200 microwatts. The light intensity is far below that of the colposcope. Four LEDs are used to emit light at three discrete wavelengths: green 520 ± 10 nm, near and distant red 660 ± 10 nm, and infra-red 936 ± 15 nm. Per observation, the LEDs operate for approximately one hundredth of a second, and approximately 14 observations are made per second.
The tissue classification algorithm is a patternmatching expert system. The final output is a result of algorithmic operation on three levels -the observation level, the "spot" level, and the overall patient screening result. With each tissue "observation" approximately 70 optical and electrical parameters are measured.
A "spot" is defined as a series of observations made with the tip of the TruScreen device kept at one location on the cervix. Under prompting from the device LCD screen, the operator moves the tip of the TruScreen probe around the ectocervix, the everted portion of the endocervix and the distal endocervical canal (Figs. 2, 3). To cover the whole squamocolumnar junction a minimum of 15 spots is required for each patient. A maximum number of 32 spot measurements can be taken by the device during each examination. Only after the minimum 15 spot measurements have been performed will the operator be able to conclude the examination and allow for the data to be processed via the patient-level algorithm. The TruScreen device is an expert system and, as for all expert systems, the recognition algorithm had to be "trained". The optical and electrical data were related to an independent "gold standard" reference diagnosis for the training data set. For algorithm training during TruScreen's development, extensive reference data were obtained, including colposcopic and histologic information.

Statistical Analysis
Statistical analysis was performed using SAS software (SAS version 9.4, SAS Inc, Cary, NC, USA). Sensitivity and specificity results were calculated, and classification accuracy was quantified by ROC curves for all three screening modalities. McNemar's test was used to derive the associated p values for the significance of the differences in sensitivity and specificity.

Results
In total, 506 women were recruited. Of these patients, 498 were successfully screened using the optoelectronic device, and no adverse effects occurred. In 8 patients, the optoelectronic examination failed due to rebooting of the device in the first screening attempt. A total of 23 patients were excluded for the following reasons: 9 patients were screened twice, 10 patients were screened within three months of treatment/punch biopsy, 3 patients were screened less than four months postpartum and 1 patient was found to have an acute and severe cervicitis.
Of the final 475 patients, 246 were referred with their first abnormal Pap smear and 229 were having a followup colposcopic examination due to previous treatment for a CIN lesion. In total, 393 patients were from the RHW, 44 from PXAMS, 28 from OAMS, and 2 from TMH. The mean age was 37.9, range 19-82 years, standard deviation 11.5 years. For the optoelectronic device and for hrHPV testing, 475 results were available for comparison with the gold standard whilst for LBC, 474 results were available.

Comparison between the Performance of the Optoelectronic Device, HPV Testing, and LBC Screening
Using histology of colposcopically directed biopsies as the gold standard, the overall sensitivity for detection of CIN II + lesions for the optoelectronic device, LBC and HPV testing was 0.72, 0.81, and 0.88, respectively; the specificity was 0.71, 0.95, and 0.76, respectively (Table 1). Results for patients without previous treatment, which more closely approximates primary screening sensitivity, were 0.71 for TruScreen, 0.82 for LBC and 0.88 for HPV, and specificities were 0.72, 0.93, and 0.70, respectively.
For follow up screening, the sensitivity was 0.80 for TruScreen, 0.70 for LBC and 0.90 for HPV testing, and the specificity 0.70, 0.97, and 0.82, respectively.
Location of the screening, the person performing the screening, and the experience of the person performing the screening did not significantly influence the result of the TruScreen test (results not shown).
The unadjusted area under the ROC curves was 0.71 for TruScreen, 0.82 for HPV testing, and 0.88 for LBC; the age-adjusted area under the ROC curves was 0.74, 0.85, and 0.91, respectively. The age-and previous-treatment adjusted area under the ROC curve was 0.83, 0.89 and 0.94, respectively. See Fig. 5 for the age-and previous-treatment adjusted ROC curves.
McNemar's test of difference did not show a statistically significant difference between the sensitivity for detection of CIN II+ lesions of TruScreen versus LBC, p = 0.26, but did show LBC to be significantly more specific. These results were consistent in the overall group, those without previous treatment, and for those with previous treatment. HPV was more sensitive than TruScreen overall, and in those without previous treatment, and were equally specific in those that hadn't had previous treatment (p = 1.0). For details see Table 2.

Discussion
In this study, we evaluated the performance of an optoelectronic device, TruScreen, in a research setting for detection of CIN 2+ cervical lesions using the gold standard of histology of colposcopically-directed biopsies. TruScreen was compared to the performance of LBC and high-risk HPV DNA testing. All patients consented to the use of the optoelectronic screening device. The screening was well tolerated, and no adverse effects were reported. Patients responded positively regarding the immediate result.
The sensitivity for detection of CIN 2+ lesions by Tr-uScreen in the overall group was found to be 0.71, which is comparable to sensitivity results described in earlier optoelectronic device studies by Singer et al. [18] 0.70, and by Lee et al. [20] 0.77. A systematic review and metaanalysis of nine Chinese studies, that included 2730 patients with 567 cases of cervical neoplasia, reported a pooled sensitivity of an optoelectronic device (TruScreen) of 0.76 [21] . An Indonesian study reported their sensitivity to be 0.76 [22]. Both these studies included sensitivities for all CIN lesions and didn't categorise their results into detection of CIN 2+ lesions. Ozgu et al. [23] and Pruski et al. [24] have reported sensitivities as high as 0.86 and 0.90 with the TruScreen optoelectonic device.
The sensitivity of optoelectronic screening might be influenced by failed detection of very small lesions or exclusively endocervical lesions. To increase sensitivity, optoelectronic devices might be used in combination with cytology, as described by Rahmadhany [22] and Singer [18] . In these studies, the sensitivity increased significantly to rates as high as 92.8%. However, combining optoelectronic screening with cytology undermines the major advantages of low cost and the immediate availability of the result.
In our study, the overall specificity for TruScreen was 0.71, comparable to a pooled specificity of 0.69 described by Yang et al. [21] . The specificity of TruScreen was also comparable to the 0.76 found for hrHPV testing in this study, but both were less than the 0.94 obtained by the liquid-based cytology. The implication for lower specificity is a higher false positive rate and consequently a higher referral rate for further cytology and/or colposcopy + biopsy.
In this study, LBC performed very well. Unfortunately, in many LMIC settings, use of LBC and successful implementation of organised cytology-based cervical cancer screening has failed due to high cost, lack of trained cytologists and poor laboratory and program infrastructure [14,15] . In contrast, the sensitivity for the TruScreen optoelectronic device was comparable to the sensitivity obtained by LBC in perfect clinical circumstances. The specificity of TruScreen was comparable to the specificity of hrHPV testing in the group without previous treatment (McNemar's test of difference: p = 1.0). This group approximates a primary screening setting. Given these characteristics, this device could potentially become an important tool in a primary screening setting, particularly in developing countries and resource-limited settings.
There is a risk of missing endocervical lesions due to the design of the TruScreen probe which mainly screens the visible cervical surface. In our study, the one endocervical lesion identified by liquid-based cytology, HPV DNA testing and colposcopy, was also identified by TruScreen, presumably because it was low in the canal and accessible to the probe. Future devices would benefit from the development of a probe that could be passed into the canal and emit electrical impulses and light waves horizontally.
A strength of this study is the high quality setting in which the screening with LBC, hr-HPV testing and the golden standard of colposcopy were performed. Their optimal performance, gives a reliable result when compared to the opto-electronic device. A limitation of the study is that the study was performed in a referred population, which serves the assessment of the performance of the optoelec-tronic device, but the results should also be confirmed in a population based setting.
In the search for more objective, reproducible cervical cancer screening, artifical intelligence is being used to classify cervical lesions on images from colposcopy [25]. Hu et al. [26] conducted an observational study in which they used an image analyzer that performed "automated visual evaluation" of the cervix as a primary screening method. In their cohort they found excellent sensitivity for detection of CIN 2+ [26]. The potential combined AI -TruScreen application lends itself well to the utilization of optoelectronic cervical screening for identifying the location of cervical lesions as an aid to colposcopy (as per Z-Scan). This requires spot-by-spot analysis and reporting of detected abnormalities with an indication of suspected severity. The only current application of the TruScreen device is screening for cervical neoplasia. To this end, the device gives a "negative" or "positive" result with respect to a screendetected abnormality, leading to an appropriate clinical response, specifically colposcopy and/or treatment. AI has limited application in this context: while this is a promising way to improve cervical cancer screening, the same limitation in screening of endocervical lesions applies.

Conclusions
The optoelectronic device TruScreen demonstrated comparable sensitivity to high quality cytology conducted in a teaching hospital setting, and specificity comparable to hrHPV testing in a setting which approximated primary screening. It has the potential to become an important tool in the prevention of cervical cancer, particularly in developing countries and resource-limited settings, due to the immediate availability of results, the objectivity and noninvasive character of the test and the relative ease of learning the technique. Further studies are required under field conditions to assess its performance and effectiveness in the primary screening setting.

Ethics Approval and Consent to Participate
Research and Ethics Governance Information System (REGIS) NSW Health project number 2021/ETH00445 was recommended to be a Quality Assurance or Quality Improvement (QA/QI) activity not requiring independent ethics review. All women consented to be screened with the optoelectronic cervical screening device TruScreen as part of an assessment of its clinical performance.