Block Arithmetic Techniques for the Implementation of Deep Neural Networks

Abstract

Performance is crucial for the evolution of DNNs, particularly as computational requirements are surging. Along with increasing DNN model scale, the training cost is becoming a new problem for DNNs.

One of the critical techniques for energy-efficient training is low-precision arithmetic.

Block arithmetic is a promising technique that reduces precision requirements and power consumption. This method further reduces the word length of the element, and the shared exponent expands their dynamic range.

This thesis aims to develop an improved block arithmetic algorithm and implementation

methodology. At the arithmetic level, this work investigates the implementation of block

arithmetic. At the GEMM kernel level, this dissertation examines kernel design under different block arithmetic implementations. For rescaling, this work further addresses the challenges associated with block arithmetic and introduces the proposed delayed scaling method called the delay update. At the application level, this work utilizes N-BEATS based inference and training accelerators to demonstrate the advantages of block arithmetic.

The contributions of this work are as follows: Firstly, we propose the block minifloat (BM)

implementation for inference, the first implementation of an FPGA based accelerator using BM arithmetic during publication, demonstrating hardware efficiency and accuracy benefits over integer and floating point on N-BEATS. Secondly, we propose the BM implementation for training, in the form of the first FPGA implementation of a 4-bit BM, mixed-precision neural network training of N-BEATS. Thirdly, we propose the delay update method to reduce the rescaling computation in block arithmetic. Empirical studies show that the delay update scheme achieves nearly the same accuracy as the commonly used maximum calibration method, with a significant hardware implementation advantage.