Here, we present a short overview of our implementation of label-free digital pathology by infrared (IR) imaging. We address the major issues to overcome for clinical translation. First, we elucidate that label-free IR imaging not only identifies cancer in tissue sections but also is even able to provide differential diagnostics as exemplified for colorectal and lung cancer. We obtained sensitivities and specificities far above 90% as compared to the gold standard histopathology. The information is obtained label-free, automated and observer independent. We showed that the spectral patterns observed in the infrared corresponds directly to protein alterations in tissue. This was achieved by integrating IR imaging with laser capture microdissection and subsequent proteome analysis. Furthermore, we demonstrated that IR imaging could replace immunohistochemical marker panels for differential diagnosis. Following one of the most important steps towards translation was the reduction of the measurement time, which now matches the time consumed for immediate diagnostics. Finally, we integrate deep neural networks for the automated annotation. All hurdles has been overcome and label-free digital pathology is ready to validate now the approach in clinically relevant unmet medical needs. This will facilitate acceptance of the approach by clinicians and pave the road for clinical translation.
Digital pathology is an emerging field in precise medicine that incorporates the acquisition, management, sharing and interpretation of pathology diagnosis in a digital environment. Digital images of stained tissue thin sections can be viewed on a computer screen or mobile device. The idea is to replace the visual inspection of thin sections using an analog microscope by a digital computer screen. Recent developments in the machine-learning field are also offering automated tissue classification based on the stained tissue thin section images. Digital pathology is an emerging powerful tool for clinical diagnostics.
However, digital pathology is limited to the availability and usability of tissue stains (e.g. H&E staining or immunohistochemical (IHC) staining) and to the availability of a sufficiently large quantity of tissue material for the different staining procedures. An alternative is label-free digital pathology by vibrational spectroscopic methods (infrared (IR) and Raman) to analyze tissue thin sections of biopsies. Label-free digital pathology methods are less expensive, less hand-waved, less labor-intensive, and consume lowest amounts of biopsies.
Here we will focus on the infrared (IR) imaging as label-free digital pathology method . It possesses high sensitivity, but the signatures of all bio-molecular components of a tissue sample are superimposed. Since all signatures of biomolecules fall into a reasonable narrow spectral range, they all can be detected simultaneously in a routine experiment. In this sense, IR imaging is a “top down” technique that simultaneously, and spatially resolved, integrates the entire proteome, genome, transcriptome, lipidome and metabolome of a cell or tissue. This allows correlating spectral changes from different tissue areas, with the morphology and molecular changes of the tissue for specific disease states and stages. Based on this, machine-learning algorithms can be trained that allow the label-free annotation of unknown tissue specimens.
This approach opens avenues into personalized medicine by more precise medical diagnostics. In this short review we will present the development of the methodology in the center of protein- diagnostics (ProDi) in Bochum over the past decade beginning with Fourier transform infrared (FTIR) imaging for tissue diagnostics towards differential diagnostics as far as recent technical developments – quantum cascade laser (QCL) based IR imaging – allowing very fast diagnostics for use in the clinical environment.
2.FTIR imaging of cancer tissues
Over the past decades, label-free digital pathology by FTIR imaging has been applied by a number of research groups worldwide to first identify spectral differences between different (normal) tissue types, between normal and diseased tissues [1,15,26,27,30], between tissues with different disease types , and as a method to elucidate prognostic information [8,17]. These were the first steps in establishing label-free digital pathology by IR imaging as a therapeutic tool, since proper diagnosis, as in certain cancer diseases, determines the next treatment steps and therefore the prognosis for the patient.
These early results have demonstrated that this method can distinguish morphological regions in tissue sections, which subsequently could be associated with different tissue types or areas of disease. In one of our first studies, we demonstrated that the different tissue structures that occur in the colon wall present distinctly different spectro-chemical signatures. In our contribution, the structures identified spectrally, the lamina propria mucosae, the lamina muscularis mucosae, the crypts and lumen filled with mucus, connective tissue and cancer, were classified by a random forest algorithm that was trained on a spectral dataset representing the different tissue types. This allows proofing the excellent correspondence between IR imaging, H&E images, and fluorescence images based on several IHC stains. But in label-free digital pathology automated and observer independent The classification of colorectal cancers with IR imaging reaches high accuracy, sensitivity, and specificity (96% accuracy, 94% sensitivity, and 100% specificity) . The prospect of observing individual cancer markers, or their effects on the spectral signatures, presented novel avenues towards diagnostics and prognostics of disease in general, and for cancer in particular.
By now IR imaging was applied for the automated, label-free classification of tumorous tissue , including, but not limited to, colorectal [13,21], lung [7,10], prostate , and bladder [12,29] cancer or melanoma  tissues. All these studies demonstrated sensitivities and specificities of the technique higher than 90%, as compared to histopathological as well as IHC diagnostics by pathologists. As mentioned above we also started by implementing a technique for the automated label-free classification of colorectal cancers and extended this classification by differential cancer diagnosis, including its grading a few years later . Parallel we analyzed lung and thoracic tumors, showing that FTIR imaging can be used for the identification of different lung cancer types, especially subtypes of adenocarcinomas with 96% accuracy  and diffuse pleural malignant mesothelioma subclasses (sarcomatoid and epithelioid) with 88% accuracy (Fig. 1) . These studies from our group proofed that FTIR imaging has the power to annotate the tissue morphology without staining based on spectral signatures. The results are highly comparable to the annotation by a trained expert but observer independent and automated.
3.FTIR imaging reflects the molecular background in a spectrum like a fingerprint
First and foremost, the infrared spectrum of a tissue pixel contains a snapshot of the entire proteome, transcriptome, genome, lipidome, and metabolome of the tissue, spatially resolved by this pixel; this snapshot includes overexpressed genes, oncogenes, protein and non-protein markers of disease (such as elevated glucose levels in diabetes) and biological consequences, such as apoptosis. To learn more about the diseases and the resulting spectral signatures they must be correlated to the molecular background.
We addressed this by integrating FTIR imaging and laser capture microdissection (LCM) for subsequent molecular analysis (Fig. 2). Therefore, label-free by IR imaging annotated tissue was cut out precisely and analyzed by proteomics providing in addition to spatial resolution by IR imaging also molecular resolution . Thereby the respective vibrational spectra are assigned to specific protein alterations in the diseased tissue as compared to healthy tissue. The study was performed on the two subtypes of diffuse malignant mesothelioma (DMM). The results demonstrated for the first time that FTIR imaging of tissue thin sections delivers already in one index color image the full information as biomarkers of multi-panel IHC staining’s, which are used to differentiate subtypes of cancer. Therefore, we showed that IR imaging could replace expensive IHC marker panels for differential diagnosis. Furthermore, this integrated approach of IR imaging guided microdissection and proteome analysis can be used to identify new protein biomarkers. With this technique we already identified a new bladder cancer IHC protein marker that helps to differentiate severe cystitis from carcinoma in situ (CIS), a high grade carcinoma . It is the only method that allows molecular tissue analysis without tissue alteration.
4.QCL-based IR imaging revolutionizes label-free digital pathology
Up to here, FTIR imaging proofed that it can be used to analyze label-free and automatically the morphology of tissue thin sections down to differential diagnostics and representing the molecular changes with high sensitivity and specificity. However, the clinical translation of this powerful technique is hindered mainly by the several hours long measuring time and the complex and large instrumental FTIR setup with liquid nitrogen cooling.
These issues can be overcome by using the quantum cascade laser (QCL) as high power light source instead of the globar in FTIR set-ups. The Bhargava and Petrich groups conducted pioneering work in the field of chemical imaging with home-made QCL based microscopes [11,14,18]. Initial studies using the first-generation commercially available Spero IR microscope (Daylight Solutions, San Diego, CA, USA), reported already promising results [4,23,24]. However, coherence effects, low laser stability, and offset edges in mosaic datasets due to insufficient movement accuracy of the stage had to be overcome. In the second generation of the commercial available instrument, transients due to laser stability and offset edges were minimized in an improved QCL based IR microscope (Spero QT). This allowed us in combination with a new developed classification model for colorectal cancer to analyze tissue thin sections now within few minutes, matching the same time range used for an immediate section of pathologists’ right in the operating theatre. The measuring time with the QCL-based microscope is 160 times faster as compared to FTIR-based microscopes with 4 cm−1 spectral resolution for the same measured area. In the study, 100 samples with UICC Stage II and III colorectal cancer tissue and 20 tumor-free tissue samples of 110 randomly chosen patients older than 18 years were analyzed. Resulting in a sensitivity of 96% and specificity of 100% as compared to histopathology (Fig. 3) . This speed-up of the methodology was the most important step towards possible translation of label-free digital pathology into the clinic because it allows larger studies, which enable the label-free digital pathology to follow the same path as the stain based digital pathology for validation studies and subsequent translation into clinics.
5.Deep neural networks for IR imaging
A final important step to unlock the full potential of label-free digital pathology the application of deep learning to vibrational spectroscopic images as in dye based digital pathology. This becomes possible due to the availability of large data sets and will allow for the first time in the field the correlation of spatial resolved molecular information by IR spectra with morphological information directly from the spectral data cube. It was demonstrated that convolutional neural networks (CNN) with architectures designed to process both spectral and spatial information can significantly improve classifier performance over per-pixel spectral classification for both Raman and IR imaging [5,16]. And that this information can be used to digitally stain the FTIR images allowing the transfer of the spectral information to a well-known visualization known by clinical experts . Furthermore, the use of deep representation learning allows the first time to transfer models between different tissue preparations (fresh-frozen and formalin fixed paraffinized – FFPE). This enables the training of classifiers for IR imaging with FFPE tissues from large tissue biobanks and the later transfer to fresh-frozen on a much smaller number of patients allowing fast diagnostics during surgeries . This development will revolutionize the label-free as well as the classical digital pathology.
We addressed in our studies the major issues to overcome translation of infrared-based label-free digital pathology to the clinics. We successfully applied IR imaging for differential diagnostics to different cancers, correlate changes on the proteome to changes in the spectral signature, reduced the measuring time close to the pathological workflow with lowest possible intra- and inter-operator/observer variability, and used state-of-the-art deep learning bioinformatics. In order to increase the acceptance in the medical/pathological community in the next step larger studies of unmet clinical needs will be addressed. This may propel IR-based label-free digital pathology into clinical routines. Furthermore, by IR imaging guided laser capture microdissection the approach can be used for largely improved and more successful biomarker search.
IR imaging is a powerful, non-destructive, label-free, inexpensive and now fast technique for tissue classification and biomarker research.
Conflict of interest
The authors state there is no conflict of interest, neither financial nor other.
N. Amharref, A. Beljebbar, S. Dukic, L. Venteo, L. Schneider, M. Pluot, R. Vistelle and M. Manfait, Brain tissue characterisation by infrared imaging in a rat glioma model, Biochimica et Biophysica Acta (BBA) – Biomembranes 1758 (2006), 892–899. doi:10.1016/j.bbamem.2006.05.003.
M.J. Baker, E. Gazi, M.D. Brown, J.H. Shanks, N.W. Clarke and P. Gardner, Investigating FTIR based histopathology for the diagnosis of prostate cancer, J Biophotonics 2 (2009), 104–113. doi:10.1002/jbio.200810062.
M.J. Baker, J. Trevisan, P. Bassan, R. Bhargava, H.J. Butler, K.M. Dorling, P.R. Fielden, S.W. Fogarty, N.J. Fullwood, K.A. Heys, C. Hughes, P. Lasch, P.L. Martin-Hirsch, B. Obinaju, G.D. Sockalingum, J. Sulé-Suso, R.J. Strong, M.J. Walsh, B.R. Wood, P. Gardner and F.L. Martin, Using Fourier transform IR spectroscopy to analyze biological materials, Nat Protoc 9 (2014), 1771–1791, http://www.nature.com/nprot/journal/v9/n8/pdf/nprot.2014.110.pdf. doi:10.1038/nprot.2014.110.
P. Bassan, M.J. Weida, J. Rowlette and P. Gardner, Large scale infrared imaging of tissue micro arrays (TMAs) using a tunable Quantum Cascade Laser (QCL) based microscope, The Analyst 139 (2014), 3856–3859. doi:10.1039/C4AN00638K.
S. Berisha, M. Lotfollahi, J. Jahanipour, I. Gurcan, M. Walsh, R. Bhargava, H. van Nguyen and D. Mayerich, Deep learning for FTIR histology: Leveraging spatial and spectral features with convolutional neural networks, The Analyst 144 (2019), 1642–1653. doi:10.1039/C8AN01495G.
B. Bird, M. Miljković, S. Remiszewski, A. Akalin, M. Kon and M. Diem, Infrared spectral histopathology (SHP): A novel diagnostic tool for the accurate classification of lung cancer, Lab Invest 92 (2012), 1358–1373. doi:10.1038/labinvest.2012.101.
M. Diem, A. Ergin and X. Mu, Spectral histopathology of the lung: A review of two large studies, J Biophotonics 12 (2019), e201900061.
E. Gazi, M. Baker, J. Dwyer, N.P. Lockyer, P. Gardner, J.H. Shanks, R.S. Reeve, C.A. Hart, N.W. Clarke and M.D. Brown, A correlation of FTIR spectra derived from prostate cancer biopsies with Gleason grade and tumour stage, European Urology 50 (2006), 750–761. doi:10.1016/j.eururo.2006.03.031.
F. Großerueschkamp, T. Bracht, H.C. Diehl, C. Kuepper, M. Ahrens, A. Kallenbach-Thieltges, A. Mosig, M. Eisenacher, K. Marcus, T. Behrens, T. Brüning, D. Theegarten, B. Sitek and K. Gerwert, Spatial and molecular resolution of diffuse malignant mesothelioma heterogeneity by integrating label-free FTIR imaging, laser capture microdissection and proteomics, Scientific Reports 7 (2017), 44829. doi:10.1038/srep44829.
F. Großerueschkamp, A. Kallenbach-Thieltges, T. Behrens, T. Brüning, M. Altmayer, G. Stamatis, D. Theegarten and K. Gerwert, Marker-free automated histopathological annotation of lung tumour subtypes by FTIR imaging, Analyst 140 (2015), 2114–2120. doi:10.1039/C4AN01978D.
K. Haase, N. Kröger-Lui, A. Pucci, A. Schönhals and W. Petrich, Advancements in quantum cascade laser-based infrared microscopy of aqueous media, Faraday Discuss 187 (2016), 119–134. doi:10.1039/C5FD00177C.
C. Hughes, J. Iqbal-Wahid, M. Brown, J.H. Shanks, A. Eustace, H. Denley, P.J. Hoskin, C. West, N.W. Clarke and P. Gardner, FTIR microspectroscopy of selected rare diverse sub-variants of carcinoma of the urinary bladder, J. Biophoton. 6 (2013), 73–87. doi:10.1002/jbio.201200126.
A. Kallenbach-Thieltges, F. Großerüschkamp, A. Mosig, M. Diem, A. Tannapfel and K. Gerwert, Immunohistochemistry, histopathology and infrared spectral histopathology of colon cancer tissue sections, J. Biophoton. 6 (2013), 88–100. doi:10.1002/jbio.201200132.
M.R. Kole, R.K. Reddy, M.V. Schulmerich, M.K. Gelber and R. Bhargava, Discrete frequency infrared microspectroscopy and imaging with a tunable quantum cascade laser, Anal Chem 84 (2012), 10366–10372. doi:10.1021/ac302513f.
C. Krafft and R. Salzer, Neuro-oncological applications of infrared and Raman spectroscopy, in: Handbook of Vibrational Spectroscopy, J.M. Chalmers and P.R. Griffiths, eds, John Wiley & Sons, Vol. 155, Ltd, Chichester, UK, 2006, p. 30.
S.D. Krauß, R. Roy, H.K. Yosef, T. Lechtonen, S.F. El-Mashtoly, K. Gerwert and A. Mosig, Hierarchical deep convolutional neural networks combine spectral and spatial information for highly accurate Raman-microscopy-based cytopathology, J Biophotonics 11 (2018), e201800022. doi:10.1002/jbio.201800022.
C.M. Krishna, G. Kegelaer, I. Adt, S. Rubin, V.B. Kartha, M. Manfait and G.D. Sockalingum, Combined Fourier transform infrared and Raman spectroscopic approach for identification of multidrug resistance phenotype in cancer cell lines, Biopolymers 82 (2006), 462–470. doi:10.1002/bip.20485.
N. Kröger-Lui, N. Gretz, K. Haase, B. Kränzlin, S. Neudecker, A. Pucci, A. Regenscheit, A. Schönhals and W. Petrich, Rapid identification of goblet cells in unstained colon thin sections by means of quantum cascade laser-based infrared microspectroscopy, Analyst 140 (2015), 2086–2092. doi:10.1039/C4AN02001D.
C. Kuepper, F. Großerueschkamp, A. Kallenbach-Thieltges, A. Mosig, A. Tannapfel and K. Gerwert, Label-free classification of colon cancer grading using infrared spectral histopathology, Faraday Discuss 187 (2016), 105–118. doi:10.1039/C5FD00157A.
C. Kuepper, A. Kallenbach-Thieltges, H. Juette, A. Tannapfel, F. Großerueschkamp and K. Gerwert, Quantum cascade laser-based infrared microscopy for label-free and automated cancer classification in tissue sections, Sci Rep 8 (2018), 855. doi:10.1038/s41598-017-17864-3.
P. Lasch, W. Haensch, D. Naumann and M. Diem, Imaging of colorectal adenocarcinoma using FT-IR microspectroscopy and cluster analysis, Biochim Biophys Acta 1688 (2004), 176–186. doi:10.1016/j.bbadis.2003.12.006.
M. Lotfollahi, S. Berisha, D. Daeinejad and D. Mayerich, Digital staining of high-definition Fourier transform infrared (FT-IR) images using deep learning, Appl Spectrosc 73 (2019), 556–564. doi:10.1177/0003702818819857.
M. Pilling and P. Gardner, Fundamental developments in infrared spectroscopic imaging for biomedical applications, Chemical Society reviews 45 (2016), 1935–1957. doi:10.1039/C5CS00846H.
M.J. Pilling, A. Henderson and P. Gardner, Quantum cascade laser spectral histopathology. Breast cancer diagnostics using high throughput chemical imaging, Analytical chemistry 89 (2017), 7348–7355. doi:10.1021/acs.analchem.7b00426.
A.P. Raulf, J. Butke, C. Küpper, F. Großerueschkamp, K. Gerwert and A. Mosig, Deep representation learning for domain adaptable classification of infrared spectral imaging data, Bioinformatics 36 (2019), 287–294. doi:10.1093/bioinformatics/btz505.
M. Romeo and M. Diem, Correction of dispersive line shape artifact observed in diffuse reflection infrared spectroscopy and absorption/reflection (transflection) infrared micro-spectroscopy, Vibrational Spectroscopy 38 (2005), 129–132. doi:10.1016/j.vibspec.2005.04.003.
W. Steller, J. Einenkel, L.-C. Horn, U.-D. Braumann, H. Binder, R. Salzer and C. Krafft, Delimitation of squamous cell cervical carcinoma using infrared microspectroscopic imaging, Anal Bioanal Chem 384 (2006), 145–154. doi:10.1007/s00216-005-0124-4.
N. Wald and E. Goormaghtigh, Infrared imaging of primary melanomas reveals hints of regional and distant metastases, The Analyst 140 (2015), 2144–2155. doi:10.1039/C4AN01831A.
K.E. Witzke, F. Großerueschkamp, H. Jütte, M. Horn, F. Roghmann, N. von Landenberg, T. Bracht, A. Kallenbach-Thieltges, H. Käfferlein, T. Brüning, K. Schork, M. Eisenacher, K. Marcus, J. Noldus, A. Tannapfel, B. Sitek and K. Gerwert, Integrated Fourier transform infrared imaging and proteomics for identification of a candidate histochemical biomarker in bladder cancer, The American Journal of Pathology 189(3) (2019), 619–631. doi:10.1016/j.ajpath.2018.11.018.
B.R. Wood, L. Chiriboga, H. Yee, M.A. Quinn, D. McNaughton and M. Diem, Fourier transform infrared (FTIR) spectral mapping of the cervical transformation zone, and dysplastic squamous epithelium, Gynecologic Oncology 93 (2004), 59–68. doi:10.1016/j.ygyno.2003.12.028.