University of Illinois Urbana-Champaign

Toward building more accessible large language models: A preliminary empirical study on data scarcity in knowledge distillation and algorithm complexity in alignment

Wang, Ziqi

Permalink

https://hdl.handle.net/2142/122036

Description

Title
Toward building more accessible large language models: A preliminary empirical study on data scarcity in knowledge distillation and algorithm complexity in alignment
Author(s)
Wang, Ziqi
Issue Date
2023-12-01
Director of Research (if dissertation) or Advisor (if thesis)
Ji, Heng
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
  • Large language models
  • Distillation
  • Data Augmentation
  • Alignment
  • Controllable Generation
Abstract
Developing large language models (LLMs) boosts various downstream tasks such as question answering. However, for various reasons, most people have to use commercial application programming interfaces (APIs) instead of training LLMs themselves. The limited accessibility of LLMs calls for efforts in democratization. This thesis mainly explores two critical technical bottlenecks that limit LLMs' access to a broader community: data scarcity and algorithm complexity. Data scarcity makes it difficult for individuals to train their own models or distill knowledge from LLMs. Therefore, we explore suitable ways to augment text data to distill knowledge from large language models better. Besides, the complex alignment algorithm (i.e., reinforcement learning from human feedback, RLHF for short) requires lots of engineering effort, which hinders individuals from training their own models. Although there are simple substitutional algorithms, they have different drawbacks. This thesis proposes to improve controllable generation, a simple substitutional algorithm of RLHF, to achieve better alignment performance. The results of this thesis can help the community toward a more democratized LLM research environment.
Graduation Semester
2023-12
Type of Resource
Thesis
Copyright and License Information
Copyright 2023 Ziqi Wang

Owning Collections

Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at Illinois