ACCELERATING LARGE SPARSE DEEP NEURAL NETWORKS INFERENCE

Liu, Hanhaotian

Permalink

https://hdl.handle.net/2142/125077 Copy

Description

Title

ACCELERATING LARGE SPARSE DEEP NEURAL NETWORKS INFERENCE

Author(s)

Liu, Hanhaotian

Issue Date

2020-05-01

Keyword(s)

Deep Neural Networks; Sparse Networks; GPU

Abstract

This thesis presents a few methods to accelerate the inference of Deep Neural Networks that are large and sparse using GPUs. Deep Neural Networks are now widely used in many applications in various fields, such as computer vision and speech recognition. Deep Neural Networks tend to work more accurately when the model is larger with more layers and neurons, but this makes the model size grow, which causes problems in transferring the data and storing the model in limited fast memory, and it also increase the number of computations, which slows the speed of network inference. The first problem can be solved by using sparse networks with comparable accuracy that contain less weights and thus are smaller in size, and this thesis intends to solve the inference speed problem caused by increased number of computations. To achieve the goal, various ways to manipulate the computation process and to parallelize the inference with multiple devices are tested against networks of different sizes and MNIST dataset as input. The characteristics of the networks and the intermediate results after each layer were also examined for optimizing the implementations. Each method used in the implementation was able to improve the inference performance by some amount, and they showed that this kind of networks have a great potential to be parallelized and accelerated.

Type of Resource

text

Language

eng

Owning Collections