2 Overview of Deep Neural Networks
2.1 Attributes of Connections Within a Layer
2.2 Attributes of Connections Between Layers
2.3 Popular Types of Layers in DNNs
2.3.1 CONV Layer (Convolutional)
2.3.2 FC Layer (Fully Connected)
2.4 Convolutional Neural Networks (CNNs)
2.6.3 Popular Datasets for Classification
2.6.4 Datasets for Other Tasks
PART II Design of Hardware for Processing DNNs
3 Key Metrics and Design Objectives
3.3 Energy Efficiency and Power Consumption
3.7 Interplay Between Different Metrics
4.1 Matrix Multiplication with Toeplitz
4.2 Tiling for Optimizing Performance
4.3 Computation Transform Optimizations
4.3.1 Gauss’ Complex Multiplication Transform
4.3.2 Strassen’s Matrix Multiplication Transform
4.3.3 Winograd Transform
4.3.4 Fast Fourier Transform
4.3.5 Selecting a Transform
4.4 Summary
5.1 Evaluation Metrics and Design Objectives
5.2 Key Properties of DNN to Leverage
5.3 DNN Hardware Design Considerations
5.4 Architectural Techniques for Exploiting Data Reuse
5.4.1 Temporal Reuse
5.4.2 Spatial Reuse
5.5 Techniques to Reduce Reuse Distance
5.6 Dataflows and Loop Nests
5.7 Dataflow Taxonomy
5.7.1 Weight Stationary (WS)
5.7.2 Output Stationary (OS)
5.7.3 Input Stationary (IS)
5.7.4 Row Stationary (RS)
5.7.5 Other Dataflows
5.7.6 Dataflows for Cross-Layer Processing
5.8 DNN Accelerator Buffer Management Strategies
5.8.1 Implicit versus Explicit Orchestration
5.8.2 Coupled versus Decoupled Orchestration
5.8.3 Explicit Decoupled Data Orchestration (EDDO)
5.9 Flexible NoC Design for DNN Accelerators
5.9.1 Flexible Hierarchical Mesh Network
5.10 Summary
6 Operation Mapping on Specialized Hardware
6.1 Mapping and Loop Nests
6.2 Mappers and Compilers
6.3 Mapper Organization
6.3.1 Map Spaces and Iteration Spaces
6.3.2 Mapper Search
6.3.3 Mapper Models and Configuration Generation
6.4 Analysis Framework for Energy Efficiency
6.4.1 Input Data Access Energy Cost
6.4.2 Partial Sum Accumulation Energy Cost