Natural Population Cohort Study of West China Hospital of Sichuan University, a prospective community-based trial of screening in rural areas such as Mianzhu (Site 1), Longquan (Site 2), Pidu, and Seda, Sichuan Province, China, aimed to build a cohort of 80,000 natural population. The lung cancer screening trial was approved by the ethics committee of West China Hospital of Sichuan University.

During January 2020 to June 2020, recruitment of physical examination population was carried out in the corresponding community. The cohort invited permanent residents who were over 20 years old. Each participant provided written informed consent and completed the questionnaire including demographic information, behavior habits such as smoking, disease history like chronic obstructive pulmonary disease (COPD), and family history of cancer. People over 40 years old were eligible for lung cancer screening. Exclusion criteria were as follows: previously diagnosed with lung cancer, pulmonary surgery, chest CT scans within the past 6 months, unwilling to receive mobile CT examination. The screening program in Site 1 started from Jul 2020 to Sep 2020, and the program in Site 2 started from Oct 2020 to Nov 2020. All data had passed the strict quality-control and were stored in Electronic Data Capture (EDC) system.

Chest CT images were acquired from mobile LDCT scanners (Neusoft Corporation). Scanning parameters were standardized as follows: tube voltage of 120 kV of, 512 $\times$ 512 of pixel matrix. Images were reconstructed to high-resolution images (window width 1800, level –400, slice thickness 1 mm), then sent to Picture Archiving and Communication Systems (PACS) to West China Hospital of Sichuan University through cloud network.

We built a two-stage deep learning model to identify high-risk patients from the population-based cohort (Fig. 1). Specifically, the first-stage model detected suspicious nodule candidates from LDCT images of the participants. In the second stage, then, the model input each nodule’s image patches and predicted the corresponding probability of malignancy. Accordingly, the malignancy risk for each patient was depended on the detected nodule which had the highest probability of being lung cancer.

Fig. 1.

Overall modeling framework. Each enrolled participant was required mobile CT scanning and demographic questionaries. The deep learning model inputs CT volume, analyzes volumetric ROIs, and outputs nodule localization and its malignancy. Abbreviations: LDCT, low-dose computed tomography; ROI, region of interest.

To detect lung nodules from LDCT images, we constructed the detection model and trained the framework followed the steps by our preceding work [20]. Given a LDCT scan, the detection pipeline first went through a series of pre-processing steps (containing lung segmentation, cropping the lung field, resizing, dividing into patches, and normalizing) to adapt to the input space of the model. At the end of the forward propagation, the region proposal network output predictions including the probability of the region being a lung nodule, the 3-dimensional coordinates, and the diameter. Then, non-maximum suppression algorithm was used to remove overlapping predictions. In order to derive a well detector, we iteratively optimized the parameters of the model under the guidance of a loss function on the training dataset. The detection loss function consisted of two parts including the cross-entropy loss (CEL) for classifying whether the region being a nodule and the smoothed L1-norm loss for regressing location and diameter of lung nodules. More details about the training setup could be referred to our previous studies [20]. In this study, we set smaller anchor sizes compared with the consideration that natural population tends to have smaller lung nodules, and their sizes were 5.0 mm, 10.0 mm, and 20.0 mm, respectively [20].

We further constructed a classification model, which predicted probability of each nodule candidate being malignant. The risk of each participant was determined by the most-likely cancerous nodule. For each nodule candidate (detected by the first stage model), the resolution was first resampled to 1.0 $\times$ 1.0 $\times$ 1.0 mm for each voxel, and a patch with the size of 32 $\times$ 64 $\times$ 64 located at the nodule’s center was cropped from the resampled images. These image patches were then inputted into the classification model, a 3D ResNet-18 network as the backbone, to forward compute the predicted probability of malignancy risk. We did not use deeper networks since the deep counterparts had more trainable parameters and were more likely to be overfitted when trained on small datasets. CEL was employed to evaluate the distance from predictions to ground-truth labels, so that to guide the model to update its parameters. Stochastic gradient descent algorithm was employed as the optimizer, with initial learning rate setting to 0.01, momentum setting to 0.9, and weight decay setting to 0.001. The model was trained with 200 epochs, and learning rate would decrease by multiplying 0.1 every 50 epochs. Moreover, data augmentation operations such as random flip, rotation, and center perturbation were employed to alleviate overfitting risk of the model. The model parameters obtaining the best accuracy on the validation set were selected, and then the results on the testing set were evaluated.

The performance of the proposed method was evaluated according to available targets from two aspects: (1) evaluating two models respectively by giving the nodule-wise location and the corresponding risk; (2) evaluating the overall performance of the framework by giving the patient-wise cancerous risk. We split the dataset into 7:1.5:1.5 for training,validation and testing regardless of which evaluating approach was used. It could ensure no prior knowledge of nodule location was available on the testing set when evaluates the performance of the proposed framework. In other word, the overall framework was only provided with LDCT images, and the detection model provided the location of nodule candidates for the classification model. All in all, the key index of our framework was depended on results of the latter evaluation methods since it imitated the real diagnostic procedure in clinical practice.

The questionnaires were strictly defined. A former or current smoker was defined as a person who had smoked more than one cigarette per day lasting more than 6 months. Occupational exposure history focused on exposure to radon,asbestos, arsenic, dust and oil fume at work for more than 12 months. Tumor history was expected lung cancer. Family included parents, brothers, sisters, grandparents, grandsons, and other immediate family members.

Remote reading of CT images was managed by experienced radiologists and nodules were classified into four existing Lung-RADS risk buckets [21]. Lung-RADS 1 was defined low-risk without nodules or with definitely benign nodules. Since Lung-RADS 2 and 3 have relatively mediate risk of malignancy, the above both were grouped in mid-risk bucket for this experiment. Lung-RADS 4A/B/X suspected or confirmed malignancy was defined as high-risk. Clinicians combined with results of deep learning model and imaging reports to judge whether the nodule was at high risk, and further recommend patients for treatment and follow-up. Patients diagnosed with lung cancer pathologically by surgery or biopsy would further receive standardized medical treatment according to lung cancer guideline of China [22] and NCCN guidelines [23].

The follow-up to December 2021 was conducted by the whole process management center of West China Hospital of Sichuan University. The clinical information of patients with lung cancer were retrieved from hospital information system (HIS) or telephone visit. Patients with low-risk nodules or non-abnormalities were recommend to screen annually.

Demographic information was analyzed utilizing Student’s t-test for continuous variables and chi-square test for categorical variables. Two-side p 0.05 was recognized statistically significant. Only an inference procedure was required for the population-based cohort. The experimental code and deep learning model were implemented on an Ubuntu 20.04 server equipped with Python (version 3.7.0, https://www.python.org/) and PyTorch (version 1.7.0, https://pytorch.org/). As the two-stage model can be split to two binary classification tasks, we used free-response receiver operating characteristic curve (FROC), accuracy (ACC), area under the receiver operating characteristic curve (AUC), F1 score and 95% confidence interval (CI) to evaluate the performance of the model.

After recruiting 19,281 people, a total of 12,360 participants were enrolled in this lung cancer screening cohort including 4730 people of Site 1 (Mianzhu) and 7630 people of Site 2 (Longquan) (Fig. 2, Table 1). The median age was 58 years old. 8169 (66.09%) participants were female, and 9784 (79.16%) participants were never smokers. The vast majority of people had no history of occupational exposure (11,075, 89.60%). A few people had a history of COPD (85, 0.69%), previous tumor history (129, 1.04%), family history of cancer (1169, 9.46%), or family history of lung cancer (329, 2.66%). Baseline characteristics differed significantly between the two sites such as age (p 0.001), sex (p 0.001), smoking history (p 0.001), occupational exposure history (p 0.001), family history of cancer (p 0.001), family history of lung cancer (p 0.001), except for COPD (p = 0.14) and tumor history (p = 0.87). 9511 (76.95%) patients were detected with pulmonary nodules and detection rate of nodules was similar between the two locations (Site 1, 3649/4,730, 77.15%; Site 2, 5862/7630, 76.83%; p = 0.70).

Fig. 2.

Participant flowchart through the screening pilot in two sites. There were four stage including recruitment, CT screening, nodules detection, pathological diagnosis for lung cancer patients. Site 1, Mianzhu; Site 2, Longquan, Sichuan province, China.

In the 1-year follow-up round, 86 patients (86/12,360, 0.70%) were test positive of lung cancer including 35 (35/4730, 0.74%) in Site 1 and 51 (51/7630, 0.67%) in Site 2 (Table 2). Most patients were female (59, 68.60%), never smoking (68, 79.07%) and had no family history of lung cancer (83, 96.51%). There was almost no statistical difference in the clinical features of the two sites: sex (p = 0.47), smoking history (p = 0.93), family history of lung cancer (p = 1), except age (p 0.001). Numerous patients underwent surgery (81, 94.19%), and several patients received chemotherapy (9, 10.47%), radiotherapy (2, 2.32%), and targeted therapy (5, 5.81%). Adenocarcinoma accounted for the majority (80, 93.03%) regardless of Site 1 or Site 2 (32, 91.43%; 48, 94.12%; p = 0.43). Lung cancer patients were diagnosed mainly at stage I (73, 84.88%).

Deep learning algorithm assisted clinicians to detect nodules and stratify risk effectively (Fig. 3, Table 3). With regard to nodules detection, the recall yielded 0.8953 (95% CI: 0.8693–0.9207) and FROC score was 0.6359 (95% CI: 0.6354–0.6384) in validation dataset; the recall reached 0.9507 (95% CI: 0.9342–0.9679) and FROC score was 0.6470 (95% CI: 0.6467–0.6495) in testing set. In term of risk stratification, the performance of deep learning model was promising with ACC of 0.9036 (95% CI: 0.8715–0.9317), macro-AUC of 0.8798 (95% CI: 0.8304–0.9248) in validation set, and ACC of 0.8696 (95% CI: 0.8370–0.9022), macro-AUC of 0.8516 (95% CI: 0.7934–0.9051) in testing dataset. Especially for the identification of high-risk groups, the deep learning model achieved superior performance with recall of 0.8571 (95% CI: 0.6667–1.0000), precision of 1.0000, F1 of 0.9630 (95% CI: 0.8889–1.0000), and AUC of 0.9894 (95% CI: 0.9702–1.0000) in validation set, and recall of 0.6923 (95% CI: 0.5000–0.8947), precision of 1.0000, F1 of 0.7619 (95% CI: 0.5600–0.9231), and AUC of 0.9634 (95% CI: 0.9328–0.9906) in testing dataset.

Fig. 3.

Deep learning performance to identify nodule and predict its malignancy. Receiver operating characteristics (ROC) curves to identify nodule in (A) validation set and (B) testing set. Abbreviations: AUC, area under the receiver operating characteristic curve; CI, confidence interval.

This AI system might be a versatile tool for physicians. As an example of clinical deployment shown in Fig. 4, it could automatically locate pulmonary nodules, predict the degree of risk and recommend treatment for the next step.

Fig. 4.

Illustration of deep learning system for the detection and diagnosis of lung cancer in clinical application. This system provided localization and risk analysis of nodules on CT images.