The COVIDx-US dataset is heterogeneous in nature, containing ultrasound imaging data of various characteristics, e.g., convex and linear US probes, from multiple sources. The current version, i.e., COVIDx-US v1.5., curates ultrasound video data of four categories, i.e., COVID-19 infection, non-COVID-19 infection (e.g., bacterial infection, non-SARS-CoV-2 viral infection), other lung diseases/conditions (e.g., chronic obstructive pulmonary disease (COPD), pneumothorax, hemothorax, bad nasogastric tube placement, collapsed lung, congenital lobar overinflation, pulmonary contusion, subcutaneous emphysema), and normal control, from nine different sources: (1) The POCUS Atlas (TPA), (2) GrepMed (GM), (3) Butterfly Network (BN), (4) Life in the Fast Lane (LITFL), (5) Radiopaedia (RP), (6) Core Ultrasound (CU), (7) University of Florida (UF), (8) Clarius (CL), and (9) data collected from scientific publications that had lung ultrasound videos available and distributed as part of the publications. The POCUS Atlas is a collaborative education platform for sharing ultrasound education. GrepMed is an open-access medical image and video repository. Butterfly Network is a health-tech company that developed a technology to miniaturize ultrasounds and launched a portable ultrasound device. LITFL is a repository of emergency and critical care education materials. Radiopaedia is an open-edit educational platform that contains a radiology encyclopedia and imaging case repository. Core Ultrasound also contains ultrasound education materials including courses, clip banks, and podcasts. The University of Florida is a public university in the United States that has a teaching hospital, i.e., UF Health Shands Hospital. For UF videos, we collected lung ultrasound training videos that were publicly available on UF’s Department of Anesthesiology webpage. Clarius company is a provider of portable ultrasound machines and scanners. Table 1 shows the distribution of the LUS video files per data source in the current version of the dataset, i.e., COVIDx-US v1.5. The COVID-19 US video files account for 38% of the data. No original data are hosted on the COVIDx-US repository and we rather provide scripts to collect, process, and integrate data from the above-mentioned data sources on end users’ devices. As seen in Table 1, TPA, LITFL, and RP provide data via a Creative Commons license. Data collected from scientific publications are provided via a Creative Commons license as well. For GM, BN, CU, UF, and CL we could not find licensing information on their websites. However, all those video files have been publicly made available on the respective websites. For example, video files collected from Clarius, as announced on their website (please see this link), are public resources shared by their users who are medical doctors. Users are recommended to verify with the mentioned data sources whether their desired usage is permitted. Also, users may refer to the provided metadata on the COVIDx-US repository for more details about the video files including the license. We would like to highlight again that no original data from the nine targeted data sources are hosted on the COVIDx-US repository.

Fig. 2 shows sample ultrasound frames captured from the ultrasound video recordings in the COVIDx-US dataset. The examples are processed by the COVIDx-US scripts. These few examples illustrate the diversity of ultrasound imaging data in the dataset. The choice of the nine different data sources and the heterogeneity in the structure and format of their hosted videos resulted in a highly diverse set of videos and images in the COVIDx-US dataset that is key to the generalizability of the AI-driven solutions/models that are built/trained on the COVIDx-US dataset. Additionally, this non-homogeneous set of data points could provide researchers with more flexibility in their research projects via filtering in/out data points based on their research objectives. We will continuously grow the dataset by adding more data points and/or data sources.

Fig. 2.

Sample ultrasound frames captured from the curated ultrasound video recordings in the COVIDx-US dataset. (a), (b), and (c) COVID-19 positive cases, (d) and (e) non-COVID-19 infection cases, (f) a normal control case. The dataset comprises 173 ultrasound videos, collected and curated systematically from nine different data sources, and 21,570 carefully curated ultrasound images in the current version.

COVIDx-US v1.5. provides 173 ultrasound videos and 21,570 processed images of 147 patients categorized into (1) COVID-19 infection, (2) non-COVID-19 infection, (3) patients suffering from other lung diseases/conditions such as pneumothorax and lung collapse, and (4) normal control cases. The dataset comes with a diverse set of metadata from ultrasound video file properties to patient demographics and symptoms. In this section, we review a few of them that are related to patients/diseases/conditions. Please refer to the “Data Records” section or COVIDx-US GitHub repository for more information on data records and metadata. Fig. 3 shows the gender distribution per curated ultrasound video. Out of 173 US video files collected, the patient’s gender is available for 47 (27%) ultrasound videos. Among all the categories, the least information about patients’ gender is available for the COVID-19 infection category.

Fig. 3.

Gender distribution per curated ultrasound video (n = 173). The numbers in parentheses under each category represent the number of video files in that category. Values on the bar show proportion of video files for that specific gender in the given category. The sum of the percentages for the normal and other lung diseases categories exceeds 100 due to rounding.

Fig. 4 shows the distribution of LUS artifacts, i.e., the existence of A- and B-lines, as well as other lung features/clinical signs such as consolidations, irregular pleural line, air bronchograms, and (sub)pleural effusion, in COVIDx-US. Numbers in parentheses under each category represent the total number of US video files under the given category for which at least one of the above-mentioned metadata is available.

Fig. 4.

Distribution of lung ultrasound artifacts, i.e., A- and B-lines, as well as other lung features/clinical signs in COVIDx-US. The numbers in parentheses under each category represent the total number of US video files under the given category for which information about the listed artifacts and clinical signs is available.

Fig. 5 shows the distribution of five symptoms, i.e., fever, cough, pain, respiratory problems, and asymptomatic, observed in the patients in COVIDx-US. Numbers in parentheses under each category represent the total number of US video files under the given category for which symptoms information is available.

Fig. 5.

Distribution of symptoms per category in COVIDx-US. The numbers in parentheses under each category represent the number of US video files under the given category for which symptom information is available.

The data were curated from nine data sources, each with a different structure. To support reproducibility and ease of use, we developed data curation engines, personalized for each of the target data sources, to automatically curate lung POCUS video recordings as well as associated metadata from the target data sources and to integrate them locally in a unified, organized structure. No original data is hosted in the COVIDx-US repository and the data is rather curated and integrated locally via our publicly released COVIDx-US scripts and the parameters set by the user. The metadata provides information on the video files, e.g., dimension and framerate, along with their category, i.e., COVID-19, non-COVID-19, other lung diseases/conditions, or normal control. The scripts are designed to be highly extensible such that more data sources can be added to the pipeline, supporting the scalability of the dataset. The scripts are made available to the general public as part of each release of the dataset.

The curated data contains video recordings captured with linear or convex US probes (n = 39 and 134, respectively) that are the most common probes used in medical settings. This provides users with higher flexibility to filter in the video files based on the probe types, if required. It also enables higher generalizability of the models that are trained on the COVIDx-US dataset by covering data of different types. Fig. 6 shows examples of linear and convex US images, i.e., single snapshots of the respective video recordings. The linear probe has a flat array and appearance and provides images of higher resolution but with less tissue penetration. Convex probes, also called curved linear probes, provide a deeper and a wider view and are mostly used for abdominal scans [36]. The original data, collected from multiple sources, were noisy in some cases, containing additional information embedded in the ultrasound videos, such as measure bars, symbols, or text (Fig. 6a). We initially processed the collected videos and cropped them to remove these (often peripheral) noises from the data.

Fig. 6.

Sample frame of an ultrasound video. (a) Captured with a convex. (b) Captured with a linear probe.

To do data cropping, we treated convex and linear US video files separately. For the convex and linear US video files, we used square and rectangular windows to crop the frames, respectively. We used rectangular windows for linear US video files to include a larger portion of the original file in the processed video file. Publicly available processing scripts are released as part of COVIDx-US to automatically perform data cropping on the benchmark dataset. The parameters of the square and rectangular windows can be modified by the end-user, if desired. However, using the default parameters for the defined windows will remove artifacts such as bars and texts visible on the side or top of the collected US video files. The output of this step is a video file containing frames that were processed using the above-mentioned cropping process, along with a metadata file that includes information about the video file properties such as dimension and framerate, as well as the type of artifacts observed, e.g., static symbols or moving pointers. The cropped files are stored locally by the provided processing scripts.

As mentioned in the previous sections, the videos were curated from multiple data sources, hence, their properties differ. To ensure maximum flexibility of the COVIDx-US dataset and as part of each release, we provide end users with highly flexible data processing scripts, allowing them to extract frames from the initially processed video files based on their research objectives and requirements, using a set of parameters as follows:

• The maximum number of frames to extract from each video.

• Extract frames from either all classes or a subset of classes from the set of [‘COVID-19’, ‘Non-COVID-19’, ‘Other’, ‘Normal’].

• Extract frames from either all data sources, i.e., [‘BN’, ‘GM’, ‘LITFL’, ‘TPA’, ‘RP’, ‘CU’, ‘UF’, ‘CL’, ‘Paper’] or a subset of them.

• Extract frames from all videos or those captured with a specific probe, i.e., convex or linear.

We set the default parameters to extract all frames from all videos. Using the defined parameters, the frames are extracted from the videos and stored locally.

After extracting frames from the videos and using the metadata file from the data cropping stage, the frames are further processed as follows:

(1) Videos with moving pointers are identified.

(2) If the video contains a moving pointer:

(a) Delete frames with a moving pointer on the lung region.

(b) For the remaining frames, generate and store a frame-specific mask.

(3) If the video does not contain a moving pointer:

(a) Make a generic mask (suitable for all the extracted frames) and store it.

(4) Use the generated masks to process the frames, removing the remaining artifacts.

The generated masks are provided as part of the COVIDx-US release. Using the generated masks, we leveraged the in-painting technique introduced by Bertalmio and colleagues [37] to remove the remaining peripheral artifacts from the frames by replacing bad marks, i.e., pixels in the masked regions, with their neighbouring pixels. The clean frames as well as the clean video file, generated by appending the clean frames, are stored locally on the user’s device. Fig. 7 shows an example of a US frame, the mask generated for this specific frame, and the final clean frame obtained by applying the mask to the original frame.

Fig. 7.

Examples of an ultrasound original frame, a mask, and the clean processed frame in the COVIDx-US dataset. (a) A sample frame with a blue symbol on the top-left of the image. (b) The generated mask for the frame. (c) The clean frame resulted from applying the generated mask to the original frame.

The COVIDx-US benchmark dataset is available to the general public at the NRC COVIDx-US GitHub repository. The repository also includes the generated masks and metadata. The dataset has been constantly growing, releasing five versions of the dataset to date (May 2022). The current version of the dataset contains 173 processed and clean ultrasound videos, divided into 66 videos of COVID-19 infected patients, 31 videos of non-COVID-19 infected patients, 48 videos of patients with other lung diseases/conditions, and 28 videos of normal patients, along with 21,570 ultrasound images extracted from the clean video files (using default parameters), divided into 8312 images of COVID-19, 4142 images of non-COVID-19, 6235 images of patients with other diseases/conditions, and 2881 images of normal patients. As mentioned in the ultrasound image extraction section, users can extract frames from the US videos according to their projects’ objectives and requirements, using the codes provided and by setting their own parameters. This makes the COVIDx-US dataset highly flexible for various research objectives. Meanwhile, the modular design of the scripts allows adding/removing data sources, if required.

Running the scripts provided in COVIDx-US will extract original videos from BN, GM, TPA, LITFL, RP, CU, UF, CL, and a set of scientific publications, and will store them locally on the user’s device in the “/data/video/original” folder. The cropped videos are stored locally in the “/data/video/cropped” folder, the clean videos in the “/data/video/clean” folder, and the clean images in the “/data/image/clean” folder. Please see the Appendix for a detailed list of video files included in COVIDx-US v1.5. along with their properties and the number of frames extracted using the default parameters. Complementary information about the file properties can be found in the metadata files located in the “/utils” folder in the COVIDx-US GitHub repository. Users may refer to the data dictionary file located in the “/utils” folder for detailed information/description of all the metadata files. The original video files extracted from the nine above-mentioned data sources are named such that the filename contains information on the source and class of the video file. This naming convention was respected for all the other generated data such as clean videos and images.

COVIDx-US collects, curates, and integrates ultrasound video files from various data sources, labelled by different labellers. This might have an impact on the integrity of COVIDx-US data. To mitigate this risk, ensure reproducibility, and provide researchers with a standardized severity score, our contributing clinician (A.F.) reviewed, classified, and graded all the ultrasound video files in COVIDx-US v1.5. (n = 173). We based our scoring system on the approach proposed by Soldati et al. [38] but modified certain criteria within each score. The number of B-lines to consider a lung area as containing interstitial syndrome which is an alteration in the lung parenchyma between air and fluid filled areas that are not normally present without pathology, has been established as 3 or more by international consensus [39]. Furthermore, several studies looking to assess the use of the lung ultrasound score (LUSS) for predicting disease severity in COVID-19 patients have used the criterion of 3 B-lines or more as well as a cut-off for the consolidation size [40, 41]. Therefore, the score by Soldati et al. [38] was modified to reflect evidence appearing recently in the literature that assesses disease severity both in hospitalized and intensive care unit (ICU) patients as suggested by the mentioned studies. Also, we have eliminated from score 2, “more than 3 B-lines” as coalescing B-lines assumes the presence of multiple B-lines altogether.

Our contributing clinician (A.F.) is an Assistant Professor in the Department of Emergency Medicine and the ultrasound co-director for undergraduate medical students at McGill University. He is practicing Emergency Medicine full-time at Saint Mary’s Hospital in Montreal. For 173 ultrasound video files in COVIDx-US v1.5., a score was assigned that classifies findings on an ordinal scale of 0–3 as follows:

• Score 0: Normal pleura with A-lines.

• Score 1: (1) More than 2 B-lines are visible, and (2) irregular thick pleura is seen.

• Score 2: Coalescing B lines, white lung, small subpleural consolidations (15 mm).

• Score 3: White lung more than 50% of pleural length, subpleural consolidations ($>$15 mm).

Fig. 8 shows the distribution of lung ultrasound scores in COVIDx-US v1.5.

Fig. 8.

Distribution of the lung ultrasound score in COVIDx-US, graded by our contributing clinician.

Although ultrasound findings such as B-lines, confluent B-lines, consolidations, and pleural irregularity are easily identifiable in COVID-19 patients, the interpretation of their clinical significance poses a challenge at the bedside. In order to produce a more systematic recording and interpretation of these findings, the LUSS was described and suggested as a potential standardization tool [38]. The LUSS has been demonstrated to correlate with the need for ICU [22, 41] or hospital admission [42] and an increased hospital length of stay [43]. These indicators could assist hospital or government managers in the planning and allocation of scarce medical resources. Moreover, the LUSS has been used to identify the development of ventilator-associated pneumonia in ICU patients [40] and thus had a direct impact on their medical management. Although studies assessing the LUSS in the prediction of mortality have been mixed [22, 40, 43], the potential to incorporate this new quantified assessment into more complex prediction models would be a promising avenue for further research.

The current version of COVIDx-US is curated from nine data sources and contains data of different types and characteristics. The scripts provided will perform the processes necessary to clean the collected POCUS videos, extract frames, and store them locally on the user’s device. But, they do not validate the analyses performed and published by the research community using COVIDx-US data. As COVIDx-US will be continuously growing, feedback provided by researchers will be used in the next versions of COVIDx-US to perform additional processes/reviews. Such feedback may be addressed to ashkan.ebadi@nrc-cnrc.gc.ca.

In order to validate the quality of images in the COVIDx-US dataset and ensure the existence of markers in the processed ultrasound images, our contributing clinician (S.K.) reviewed a randomly selected set of images and reported his findings and observations. Our contributing clinician (S.K.) is a practicing Internal Medicine and ICU specialist, certified in both specialties by the Royal College of Physicians of Canada. Fig. 9 shows three select images of COVID-19 positive cases, as examples, that were reviewed. The summary of our expert clinician’s report is as follows.

Fig. 9.

Sample processed ultrasound images, reviewed and reported on by our contributing clinician. (a), (b) and (c) are all confirmed COVID-19 positive cases.

Case 1 (Fig. 9a). Our contributing clinician observed multiple pleural irregularities, including pleural thickening and the presence of sub-pleural consolidations which have been previously described as markers of COVID-19 disease severity [44]. These findings, together with the observed B-line profile, are indicative of a moderate to severe pulmonary disease.

Case 2 (Fig. 9b). This is an image of lung pleura in short depth. Our clinical expert observed abnormalities and irregularities in the pleura as it is thickened and “shredded” with hypoechoic signals suggesting consolidations and air bronchograms. Although a deeper view to assess for B-lines would be more optimal, these findings together suggest moderate airspace disease, most commonly on the basis of pneumonia.

Case 3 (Fig. 9c). This appears to be an image of lung pleura at depth of ~5 $\pm$ 2 cm. According to our contributing clinician, the pleura and underlying lung are abnormal. There is the presence of a “waterfall sign”, i.e., subpleural consolidation with a B-line. The pleura is thickened and irregular. Despite the observed abnormalities, more imaging data is needed to inform on differential diagnosis. In addition, it is not possible to comment on lung sliding as this is a static image.

Our expert clinician findings and observations confirmed the existence of identifiers and indicators of disease in the COVIDx-US dataset. AI-powered analytics solutions can exploit such indicators and patterns to monitor and classify COVID-19 cases. Based on our contributing clinician’s evolving experience, LUS has significant utility in the management of COVID-19 patients with respiratory symptoms. As a safe, rapid, reproducible, low-cost, and highly informative tool for assessing the severity of lung involvement, early studies suggest that it can be used to inform triage and treatment decisions [45]. To this end, several published LUS-based protocols are now undergoing validation in prospective clinical trials [46, 47]. Furthermore, several groups are now evaluating the potential utility of LUS in other settings, including the ICU where it could be used to track disease progression, and to evaluate patient candidacy and clinical response to various interventions including ventilator weaning, prone positioning, and lung recruitment maneuvers in patients with acute respiratory distress syndrome (ARDS) [40].

The known limitations of this modality include the observation that LUS findings are not necessarily specific to COVID-19. Moreover, they have yet to be proven as reliable markers of clinical outcome in appropriately sized clinical studies. The deployment of LUS in COVID-19 also requires strict infection control measures. Lastly, LUS requires significant operator training and experience before it can be used in the management of potentially unstable patients, or in those with suspected infectious syndromes. AI-driven solutions can aid clinicians with the screening process of COVID-19 patients, reducing the pressure on healthcare systems and healthcare providers.

Our goal is to encourage broad adoption and contribution to this initiative. The COVIDx-US dataset is an open-source open-access initiative under the terms of the GNU Affero General Public License 3.0. All the codes and materials, e.g., metadata and masks, necessary to reproduce the COVIDx-US dataset, as described and explained in this manuscript, are available to the general public at https://github.com/nrc-cnrc/COVID-US/, accessible with no restrictions. The scripts were coded in Python programming language (version 3.6.12), using pandas 1.1.3, selenium 3.141.0, and requests 2.24.0 libraries.

To the best of the authors’ knowledge, COVIDx-US is the first and largest open-access “fully-curated” benchmark LUS imaging dataset that contains a unified and standardized human lung ultrasound score. The dataset can be used in AI research for use cases such as classifying COVID-19 cases and assessing the severity of the disease. We would like to highlight that the dataset is not envisioned to be used to build fully-automated AI-enabled diagnostic solutions. This reproducible, easy-to-use, and easy-to-scale dataset benefits from a well-documented modular design, created specifically for tackling AI challenges and facilitating the building and evaluation of AI-driven analytics solutions to help clinicians with the screening process of COVID-19 patients, reducing the pressure on healthcare systems and healthcare providers.

One of the main contributions of our work is a systematic framework for data curation, data processing, and data validation to dataset creation for creating a unified, standardized POCUS dataset. We also tried our best to design our systematic framework to be very easy-to-use and easy-to-scale, even for users without deep computer science/programming knowledge, and hope the availability of this framework will contribute to the community by making it easier to scale and expand such datasets in a semi-automated manner. Finally, the COVIDx-US contains a unified and standardized human lung ultrasound score that could not only be used for further validation/evaluation of the algorithms/models trained on the dataset but also enables more AI-driven analytics research directions.