Understanding and improving the fidelity of local developer testing for cloud-based software

Mazhar, Anna

Permalink

https://hdl.handle.net/2142/124353 Copy

Description

Title

Understanding and improving the fidelity of local developer testing for cloud-based software

Author(s)

Mazhar, Anna

Issue Date

2024-04-26

Director of Research (if dissertation) or Advisor (if thesis)

Xu, Tianyin

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• Cloud-based software testing
• Emulators
• Root cause analysis
• Azure and AWS cloud services
• API discrepancies
• Local testing
• Deployment safety
• Software quality assurance.

Abstract

Modern software projects have been progressing towards a cloud-based, serverless programming model, where software applications use cloud services as important components. Such cloud-based programming practice greatly simplifies software development by harvesting cloud benefits (e.g., high availability and elasticity). However, it imposes significant challenges for software testing and analysis, due to the opaqueness of cloud backends and the monetary cost of invoking cloud services for continuous integration and deployment. As a result, software developers commonly use cloud emulators for offline development and testing, before online testing and deployment. This thesis presents a systematic analysis of cloud emulators from the perspective of cloud-based software testing. Our goal is to (1) understand the discrepancies introduced by cloud emulation with regard to software quality assurance and deployment safety and (2) address inevitable gaps between emulated and real cloud services. The analysis results are concerning. Among 255 APIs of five cloud services from Azure and Amazon Web Services (AWS), we detected discrepant behavior between the emulated and real services in 94 (37%) of the APIs. These discrepancies lead to inconsistent testing results, threatening deployment safety, introducing false alarms, and creating debuggability issues. The root causes are diverse, including accidental implementation defects and essential emulation challenges. We discuss potential solutions and develop a practical mitigation technique to address discrepancies of cloud emulators for software testing.

Graduation Semester

2024-05

Type of Resource

Thesis

Copyright and License Information

Copyright 2024 Anna Mazhar

Owning Collections