Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer

Bejnordi, Babak Ehteshami
VETA, Mitko
van Diest, Paul Johannes
van Ginneken, Bram
Halıcı, Uğur
© 2017 American Medical Association. All rights reserved.IMPORTANCE: Application of deep learning algorithms to whole-slide pathology imagescan potentially improve diagnostic accuracy and efficiency. OBJECTIVE: Assess the performance of automated deep learning algorithms at detecting metastases in hematoxylin and eosin-stained tissue sections of lymph nodes of women with breast cancer and compare it with pathologists' diagnoses in a diagnostic setting. DESIGN, SETTING, AND PARTICIPANTS: Researcher challenge competition (CAMELYON16) to develop automated solutions for detecting lymph node metastases (November 2015-November 2016). A training data set of whole-slide images from 2 centers in the Netherlands with (n = 110) and without (n = 160) nodal metastases verified by immunohistochemical staining were provided to challenge participants to build algorithms. Algorithm performance was evaluated in an independent test set of 129 whole-slide images (49 with and 80 without metastases). The same test set of corresponding glass slides was also evaluated by a panel of 11 pathologists with time constraint (WTC) from the Netherlands to ascertain likelihood of nodal metastases for each slide in a flexible 2-hour session, simulating routine pathology workflow, and by 1 pathologist without time constraint (WOTC). EXPOSURES: Deep learning algorithms submitted as part of a challenge competition or pathologist interpretation. MAIN OUTCOMES AND MEASURES: The presence of specific metastatic foci and the absence vs presence of lymph node metastasis in a slide or image using receiver operating characteristic curve analysis. The 11 pathologists participating in the simulation exercise rated their diagnostic confidence as definitely normal, probably normal, equivocal, probably tumor, or definitely tumor. RESULTS: The area under the receiver operating characteristic curve (AUC) for the algorithms ranged from 0.556 to 0.994. The top-performing algorithm achieved a lesion-level, true-positive fraction comparable with that of the pathologist WOTC (72.4% [95% CI, 64.3%-80.4%]) at a mean of 0.0125 false-positives per normal whole-slide image. For the whole-slide image classification task, the best algorithm (AUC, 0.994 [95% CI, 0.983-0.999]) performed significantly better than the pathologists WTC in a diagnostic simulation (mean AUC, 0.810 [range, 0.738-0.884]; P <.001). The top 5 algorithms had a mean AUC that was comparable with the pathologist interpreting the slides in the absence of time constraints (mean AUC, 0.960 [range, 0.923-0.994] for the top 5 algorithms vs 0.966 [95% CI, 0.927-0.998] for the pathologist WOTC). CONCLUSIONS AND RELEVANCE: In the setting of a challenge competition, some deep learning algorithms achieved better diagnostic performance than a panel of 11 pathologists participating in a simulation exercise designed to mimic routine pathology workflow; algorithm performance was comparable with an expert pathologist interpreting whole-slide images without time constraints. Whether this approach has clinical utility will require evaluation in a clinical setting.
JAMA - Journal of the American Medical Association


Parsian, Maryam; Özen, Can; Gündüz, Ufuk; Department of Biotechnology (2022-8-03)
The development of new tissue culture models is needed in cancer research and personalized medicine to better predict patients' responses to treatment. Using live tumor cells as a testing platform for drug development and identifying optimal therapy is a promising strategy. We described a new tissue culture approach that combines a microfluidic chip with the micro-dissected breast cancer tumor slices. The tumors from MDA-MB-231, SKBR-3, and MCF-7 cell lines were created in CD1 nude mice cultured in this mic...
Convolutional Neural Networks for Classification of Colorectal Cancer on Whole Slide Images
Temena, Mehmet Arda; Isildak, Ulas; Acar, Ahmet (2021-03-27)
Diagnosis of cancer is typically performed by medical pathologist visually inspecting hematoxylin and eosin (H&E) stained whole slide images. Although the early detection of various cancer types is crucial for a successful treatment, it is a challenging work due to inter- and intra-personal variability and it may often result in a disagreement between pathologists. Visual inspection of medical specimen is a well- established method for cancer diagnosis, but digital transformation is now inevitable fact, whe...
Forward problem solution for electrical conductivity imaging via contactless measurements
Gençer, Nevzat Güneri (IOP Publishing, 1999-04-01)
The forward problem of anew medical imaging system is analysed in this study. This system uses magnetic excitation to induce currents inside a conductive body and measures the magnetic fields of the induced currents. The forward problem, that is determining induced currents in the conductive body and their magnetic fields, is formulated. For a general solution of the forward problem, the finite element method (FEM) is employed to evaluate the scalar potential distribution. Thus, inhomogeneity and anisotropy...
Functional characterization of two potential breast cancer related genes
Akhavantabasi, Shiva; Erson Bensan, Ayşe Elif; Department of Biology (2012)
Cancer may arise as a result of deregulation of oncogenes and/or tumor suppressors. Although much progress has been made for the identification of such cancer related genes, our understanding of the complex tumorigenesis pathways is still not complete. Therefore, to improve our understanding of how certain basic mechanisms work in normal and in cancer cells, we aimed to characterize two different breast cancer related genes. First part of the study focused on subcellular localization USP32 (Ubiquitin Specif...
Structural investigation of donor age effect on human bone marrow mesenchymal stem cells: FTIR spectroscopy and imaging
Aksoy, Ceren; AERTS KAYA, FATİMA SUSANNA F.; KUŞKONMAZ, BÜLENT BARIŞ; Uckan, Duygu; Severcan, Feride (2014-08-01)
Stem cell studies hold enormous potential for development of new therapies for tissue regeneration and repair. Bone marrow mesenchymal stem cells (BM-MSCs) can differentiate into a variety of nonhematopoietic tissues and contribute maintenance of healthy hematopoiesis by providing supportive cellular microenvironment into BM. Here, we investigated age-related differences in BM-MSCs by using attenuated total reflection Fourier transform infrared (ATR-FTIR) spectroscopy and FTIR imaging together with hierarch...
Citation Formats
B. E. Bejnordi et al., “Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer,” JAMA - Journal of the American Medical Association, pp. 2199–2210, 2017, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/57747.