Crafting unusual programs for fuzzing deep learning libraries
Yang, Shujing
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/120389
Description
Title
Crafting unusual programs for fuzzing deep learning libraries
Author(s)
Yang, Shujing
Issue Date
2023-04-20
Director of Research (if dissertation) or Advisor (if thesis)
Zhang, Lingming
Department of Study
Computer Science
Discipline
Computer Science
Degree Granting Institution
University of Illinois at Urbana-Champaign
Degree Name
M.S.
Degree Level
Thesis
Keyword(s)
Software engineering
Language models
Abstract
Deep Learning (DL) applications play a vital role in modern society. Bugs in DL libraries can significantly impact a wide range of downstream DL applications, making it crucial to develop effective testing techniques for these libraries. Generating valid input programs for fuzzing DL libraries is challenging, as they must adhere to both the syntax/semantics of the supported languages (e.g., Python) and the tensor/operator constraints required for constructing valid computational graphs. The recent TitanFuzz work has shown, for the first time, that modern Large Language Models (LLMs) can be directly employed to implic- itly learn all language and DL computation constraints to create valid programs for fuzzing DL libraries. However, LLMs tend to generate ordinary programs that follow patterns/to- kens similar to typical programs found in their vast training corpora (e.g., GitHub), whereas fuzzing favors unusual inputs that cover edge cases or are less likely to be manually produced.
To address this challenge, we propose AtlasFuzz, the first technique to prime LLMs for synthesizing unusual programs to enhance fuzzing effectiveness. AtlasFuzz is based on the well-established hypothesis that historically bug-triggering programs may contain rare and valuable code elements crucial for bug discovery. While traditional techniques leveraging such historical information demand extensive human effort to design dedicated generators and ensure the syntactic/semantic validity of the generated programs, AtlasFuzz demon- strates that this process can be fully automated through the intrinsic capabilities of LLMs (including fine-tuning and in-context learning) and is generalizable and applicable to chal- lenging domains. Furthermore, AtlasFuzz also highlights the potential of directly utilizing the instruction-following capability of the recent ChatGPT for effective fuzzing.
Our experimental study on two popular DL libraries (PyTorch and TensorFlow) reveals that AtlasFuzz is an effective fuzzer for DL libraries, detecting 18 bugs, including 10 already confirmed as previously unknown bugs.
Use this login method if you
don't
have an
@illinois.edu
email address.
(Oops, I do have one)
IDEALS migrated to a new platform on June 23, 2022. If you created
your account prior to this date, you will have to reset your password
using the forgot-password link below.
