LSTM FOR T-DLA+: EFFICIENT COMPUTATION OF QUANTIZED LSTM NETWORKS

Harisrikanth , Keshav

Permalink

https://hdl.handle.net/2142/125090 Copy

Description

Title

LSTM FOR T-DLA+: EFFICIENT COMPUTATION OF QUANTIZED LSTM NETWORKS

Author(s)

Harisrikanth , Keshav

Issue Date

2020-05-01

Keyword(s)

LSTM; RNN; Neural Network Accelerator; Quantization; Ternarization; Field Programmable Gate Array; Embedded Neural Network; Accelerator architectures; Reconfigurable architectures

Abstract

Neural networks represent a complex computation which can be extremely resource intensive. This can limit their usability in contexts where very small amounts of hardware are deployed on low power budgets. One key way in which the computational cost of neural networks can be significantly reduced is quantization, in which the values throughout the network are represented in fewer bits. A ternarized network is specifically a network in which every weight has been quantized to 3 values, +1,-1 and 0. Past works have shown that, despite their simple weight systems, ternarized neural networks can achieve much closer accuracy to full floating point networks than might be expected. In order to further extract computational efficiency from these networks, we have designed and analyzed DNN acceleration on embedded FPGAs through the creation of a ternarized deep neural network coprocessor with custom designed ISA. We have prior built a basic ternarized neural network accelerator capable of basic CNN computation. A key point of improvement for this design over past works is specifically supporting LSTM operations, and the efficient hardware specifically for LSTM computation. This continuing work faces the significant challenge of making ISAs that are both general enough for any task, but specific enough to be well executed in hardware, and reduce code density. This thesis in particular will primarily focus on the LSTM hardware computation units.

Type of Resource

text

Language

eng

Owning Collections