University of Illinois Urbana-Champaign

VISION MODEL PRUNING

Xinglong Sun

Content Files
FA21-ECE499Thesis-Sun, Xinglong.pdf

Permalink

https://hdl.handle.net/2142/115000

Description

Title
VISION MODEL PRUNING
Author(s)
Xinglong Sun
Issue Date
2021
Keyword(s)
  • model compression
  • multitask learning
  • computer vision
  • model
  • pruning
Language
en
Abstract
Neural models have seen great success in computer vision in recent years, especially in fundamental vision tasks such as image classification, object detection, and segmentation. As a plethora of new model architectures have been developed or continue to be proposed, how to effectively reduce their sizes has become an increasingly popular task to enable the models to run on edge-devices where memory and storage capacity are relatively limited. Model pruning, as one of such compression techniques, aims to remove parameters in models in a structured or unstructured way while preserving the model performance as much as possible. We believe there’s still big potential in furthering the state-of-the-art performance of pruning methods in vision tasks and explore specific scenarios where previous works failed to deal with. In this thesis, we first present a novel unstructured pruning method called DiSparse targeting the popular Multitask Learning (MTL) scenario. The method takes the task entanglement of the multitask model parameters into account and perform pruning by disentangling the loss and importance measurement. Besides pruning, the proposed scheme is also helpful to assist pre-training multi-task model design. Next, we present a novel structured pruning method by us called Deep Singular Pruning (DSP), which extracts importance measurements after a reorganization step, separating us from almost all of the other structured pruning works. The method operates in a single-shot fashion, saving significant computation and labor cost while achieving state-of-the-art compression performance. Theoretical analysis and detailed empirical studies will both be demonstrated to evaluate the importance of the proposed methods.
Graduation Semester
2022-10-14T13:34:40-05:00
Type of Resource
text
Handle URL
https://hdl.handle.net/2142/115000

Owning Collections

Undergraduate Theses at Illinois PRIMARY