Efficient convolutional neural network inference on microcontrollers

Tuttle, Michael

Permalink

https://hdl.handle.net/2142/116129 Copy

Description

Title

Efficient convolutional neural network inference on microcontrollers

Author(s)

Tuttle, Michael

Issue Date

2022-07-21

Director of Research (if dissertation) or Advisor (if thesis)

Shanbhag, Naresh R

Department of Study

Electrical & Computer Eng

Discipline

Electrical & Computer Engr

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Machine Learning
• TinyML
• Microcontrollers
• Convolutional neural networks
• CNN
• Pruning

Abstract

Convolutional Neural Networks provide state-of-the-art performance on a wide variety of computer vision tasks. However, the large size and computational complexity of these models makes their deployment on resource-constrained edge devices difficult. To remedy this, efficient versions of these layers such as depthwise-separable convolutions and sparse convolutions have been proposed which dramatically reduce the number of parameters and operations required for accurate inference. This work explores various optimizations for these layers on Cortex-M4 MCUs. Memory optimizations such as in-place DWS convolutions and patch-based inference reduce MobileNetV1 peak memory usage by $3.75\times$. The typically inefficient DW layers are sped up by $2\times$ over the CMSIS-NN reference kernel, and memory-aware sparsity enables up to $5.7\times$ speed-up over dense convolutional layers at $95\%$ sparsity.

Graduation Semester

2022-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2022 Michael Tuttle

Owning Collections