AI: What Have You Done for Us Lately?

• glomerulus
• renal morphology
• pathology

Artificial intelligence, deep learning, neural networks: the phrases conjure up images of autonomous cars and humanoid interactive robots performing menial and complex human tasks with absolute precision—a source of hope or dread for the future. However, amid the buzzwords and dizzying hype related to this technologic transformation, there are some tools with palpable, near-term, practical effect that are already finding use in renal research.

Investigators have relied on computer-based analysis since systems have been in existence. Multivariate statistics using matrix algebra found expanded application early on in the genesis of computers. Also, as far back as 1964, the late Ledley,¹ a pioneer in the biomedical informatics field, touted high-speed automated image pattern recognition as a tool that “may well bring numerous problems now occupying the minds of biomedical research scientists within reach of a solution.” We have become accustomed to using every manner of computerized measure on our experimental data for decades, but without a doubt there have been significant recent advancements that portend accelerated progress in all scientific disciplines.

One of the most significant of these recent technologic developments has been the evolution of methods for pattern recognition popularized by their application to classification of friends in your Facebook and Google Photos images as well as voice recognition on your smartphone and interactive responses from customer service chatbots. Specific to images, an oft-quoted marker of progress is the result of an annual open competition for computer-based classification of a carefully characterized (curated) set of over a million images known as ImageNet. After the introduction of the first deep learning–based approaches in 2012, gains in model performance accelerated dramatically. As of 2016, performance by computer algorithms can match or arguably outperform humans on the dataset. The advancement has been potentiated by a convergence of factors: vastly more capable graphics cards that can execute the math-intensive steps efficiently, availability of digital training data, and the advent of specialized algorithms known as convolutional neural networks (CNNs) with their “deep learning” variants.

These new variants are specific forms of the general concept of machine learning—methods of statistical analysis and pattern recognition in which features are determined from a dataset in an automatic fashion rather than being handcrafted by a human expert. Neural networks are one form of machine learning, in which data are processed by layers of “neurons,” taking variable combinations of the output from prior layers to produce new output, somewhat analogously to the operation of biologic neurons. CNNs are a specialization of neural networks, in which signals are mathematically mixed (convolved) with spatial patterns (filters) aimed at producing a response signaling the presence of a target feature. In initial layers of the network, these filters are often graphically reminiscent of zebra stripes and recapitulate the manner in which mammalian visual cortex spatially filters visual input. This type of processing is particularly suited to detecting patterns in data with spatial structure, such as images, audio, or time series. The “deep learning” part refers to a combination of recent hardware and algorithm breakthroughs that have enabled much larger networks with many more layers to be used, with consequent gains in performance and successful application to many new problems.

In this issue of the Journal of the American Society of Nephrology, Bukowy et al.² apply these advanced techniques to the automated recognition and localization of rat glomeruli in whole-slide images of routinely stained histologic sections. An otherwise impossibly laborious task of culling through thousands of images to find glomeruli for scoring of damage can thereby become easily achievable. The authors adapt a widely successful (and at one time, ImageNet champion) CNN algorithm known as AlexNet.³ They couple this analysis with software for presentation and scoring of glomeruli in a standardized and reproducible fashion for significantly improved workflow efficiency.

This article fits within a series of recent computationally focused conference papers applying CNN to glomerular identification and segmentation, including those presented by Pedraza et al.⁴ and Temerinac-Ott et al.⁵ In complement to these publications, Bukowy et al.² have implemented a practical investigative mode of the CNN techniques and contributed to paving the way for further developments. It is easily inferred that the natural evolution is toward the automated detection, classification, and quantification of renal damage in disease models and human specimens. However, we are not quite there yet. A principal challenge for any automated image analysis is robustness. Biologic morphology variability is extensive, and performance suffers when algorithms encounter features not seen before, such as artifacts or uncharacterized diseased conditions. Experimentation can often lead to unexpected histology, and if improperly trained, even the most advanced recognition algorithms will fail. Therefore, extension of these techniques to other specific applications will require detailed validation as evidenced by the more limited success of the algorithm trained on rat glomeruli when applied to human specimens. Notably, Bukowy et al.² show how training by simpler “region-based” classification of images (quick and easy) rather than detailed manual segmentation (harder and more time consuming) can yield useful results. Also, they show a general approach by which well defined image pattern recognition problems in renal research can already be adequately addressed with machine learning tools.

CNN and other pattern recognition techniques will undoubtedly be further potentiated by the continuing adoption of whole-slide imaging and rapid advancements in other digital histology techniques. Routine digitization of microscopic data that can be readily correlated with electronically recorded interpretations or objective outcomes, when properly organized and collated, will help provide critically lacking training sets for more massive application of pattern recognition, even if it will take some time and effort for specific tasks to be individually trained and tested. In the interim, whether you are in the dystopic camp or the utopic camp on the future of artificial intelligence, consider that there are avenues to explore today that can make your counting and classifying of glomeruli more accurate, more informative, and much easier!

• Copyright © 2018 by the American Society of Nephrology