Table 0.1. Three types of tomographic image reconstruction algorithms in an over-simplified comparison (the penalization of image reconstruction and topology of network architecture can be complicated).
| Category | Form | Knowledge | Input | Quality | Speed |
|---|---|---|---|---|---|
| Analytic reconstruction | f=Op | Idealized model, without noise | High SNR, complete | High | High |
| Iterative reconstruction | f(k)=Op,f(k−1) | Physical model, image prior | Low in various ways | Decent | Low |
| Deep reconstruction | f=OθN…Oθ1p | Model, prior, big training data | Poor, incomplete | Superior, task-specific | High |
0.4 The field of deep reconstruction and the need for this book
The industrial revolution from the eighteenth century onwards has greatly accelerated civilization, and now we are in the intelligence revolution, synergizing big data, exploding information, instantaneous communication, sophisticated algorithms, high-performance computation, and AI/ML. Over only the past few years, as AI/ML methods have become mainstream, deep learning has affected many practical applications and generated overwhelming excitement (figure 0.5). As a result, more and more students and researchers are motivated to learn and apply AI/ML.
Figure 0.5. A Web of Knowledge search, with ‘deep learning’, ‘medical’, and ‘imaging’ as the topic terms (data collected on 11 July 2019).
Our field is tomographic image reconstruction, which is experiencing a paradigm shift towards deep-learning-based reconstruction (see our perspective on deep imaging (Wang 2016)). Simply speaking, we are interested in developing deep learning methods going from measured features to tomographic images. Currently, deep learning techniques are being actively developed worldwide for tomographic image reconstruction, delivering excellent results (figure 0.6, and also see (Wang et al 2019)).
Figure 0.6. Web of Knowledge search, with ‘deep learning’ in the article title (data collected on 11 July 2019).
While many of us share optimism about this new wave of tomographic imaging research, there are doubts and concerns regarding deep reconstruction. This conflict of opinions is natural and healthy. In retrospect, at the beginning of the development of analytic reconstruction, there was a major critique that given a finite number of projections, the tomographic reconstruction is not uniquely determined (introducing ghosts in a reconstructed image). Later, this was successfully addressed by regularization methods. When iterative reconstruction algorithms were first developed, it was observed that a reconstructed image was strongly influenced by the penalty term. In other words, it seemed that what one saw was what one wanted to see! Nevertheless, by optimizing the reconstruction parameters, iterative algorithms have been made into commercial scanners. As far as compressed sensing is concerned, it was proved that there is a chance that a sparse solution is not the truth. For example, a tumor-like structure could be introduced, or pathological vessels might be smoothed out if total variation is overly minimized. Similarly, deep learning appears to present issues in practice, such as the interpretability problem. No Maxwell equations for deep learning yet exist, and a deep network as a black box is trained to work with big data in terms of parameter adjustment. The interpretability of neural networks is currently a hot topic. Given the rapid progress being made in theoretical and practical aspects, we believe that deep learning algorithms will become the mainstream for medical imaging.
With the encouraging results and insights, some of which are in this book, we are highly confident that, in principle, AI/ML methods for deep reconstruction ought to outperform iterative reconstruction (IR) and compressed sensing (CS) for medical imaging. To convince the reader that AI/ML will dominate tomographic imaging, let us highlight three key arguments: (i) IR/CS can be used as a component in a neural network (such as in our ‘LEARN’ network in chapter 5); (ii) the result from IR/CS can be used as the baseline (such as for the denoising, despeckling, or de-blurring networks mentioned in several chapters in this book); and (iii) IR/CS reconstruction algorithms can be enhanced or even replaced by powerful neural networks with advanced architectures trained with big data with unprecedented domain priors.
There are a good number of deep learning books of high quality. However, they are either on general deep learning methods or other specific deep learning applications. This book is dedicated to the emerging area of deep reconstruction, representing a new frontier of machine learning, and offers a unified treatment of this theme. In particular, this book is focused on medical imaging, which is a primary example of tomographic imaging that affects all people worldwide, spans a huge business, and remains a major driver for technical innovations.
0.5 The organization of this book
This book reflects the state-of-the-art, since all of the co-authors are active researchers in the deep imaging field. Also, the materials are presented in a reader-friendly way, covering classic reconstruction ideas and human vision inspired insights, naturally leading to deep artificial neural networks and deep tomographic reconstruction. There are four parts in this book, with two to three chapters per part.
The first part consists of chapters 1–3, laying out the foundation for the remaining parts. The first chapter describes general principles for imaging, with an emphasis on the importance of prior information when data are imperfect, inconsistent, or incomplete, either in the Bayesian framework or in the context of the human vision system (HVS). From these perspectives, the concepts of regularization and sparsity naturally arise. The second chapter focuses on regularized image reconstruction in the Bayesian and compressed sensing perspectives, with an emphasis on dictionary learning, whose computational structure can be viewed as a single-layer neural network. As a good example, a statistical reconstruction algorithm is empowered with either a global or adaptive dictionary for low-dose computed tomography (CT). Based on the materials covered in chapters 1 and 2, chapter 3 offers a basic but quite complete presentation of neural network architectures, including the concepts and components of deep neural networks, representative networks such as auto-encoder, VGG, U-Net, ResNet, generative adversarial network (GAN), and graph convolutional network (GCN), as well as training, validation, and testing strategies.
The second part includes chapters 4 and 5, exclusively dedicated to CT. Chapter 4 reviews the CT data acquisition process and the development of CT scanners. Also, both analytic and iterative reconstruction algorithms are exemplified. In addition to analytic and iterative algorithms, chapter 5 covers the latest developments of the new type of reconstruction algorithm that employs deep neural networks. A number of recently published deep learning based methods are presented to show the feasibility,