Reflection-aware static regression test selection

Hadzi-Tanovic, Milica

Permalink

https://hdl.handle.net/2142/101065 Copy

Description

Title

Reflection-aware static regression test selection

Author(s)

Hadzi-Tanovic, Milica

Issue Date

2018-04-26

Director of Research (if dissertation) or Advisor (if thesis)

Marinov, Darko

Department of Study

Computer Science

Discipline

Computer Science

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.S.

Degree Level

Thesis

Keyword(s)

• regression test selection
• reflection
• static analysis

Abstract

Regression test selection (RTS) aims to speed up regression testing by rerunning only tests that are affected by code changes. RTS can be performed using dynamic or static analysis techniques. A recent study showed that static and dynamic RTS can perform similarly for some medium-sized Java projects. However, the results also showed that static RTS can be sometimes unsafe, missing to select some tests that dynamic RTS selects, and reflection was the only cause of unsafety among the evaluated projects. In this thesis, we investigate five techniques—three purely static techniques and two hybrid static-dynamic techniques—to make static RTS safe with respect to reflection. We implemented four of these reflection-aware techniques as extensions to one reflection-unaware (RU) static RTS technique in a tool called STARTS. We evaluated the fifth technique but did not yet fully implement it. To assess reflection-aware SRTS techniques, we measured benefits and costs of four implemented reflection-aware techniques by comparing their end-to-end times with the RU technique and with RetestAll—the latter runs all tests after every code change. We also compared safety and precision of all five static RTS techniques relative to Ekstazi, a state-of-the-art dynamic RTS technique. Our results on 805 revisions of 22 open-source Java projects show that all reflection-aware techniques we evaluated can make static RTS safe with respect to reflection, but their costs vary widely. The best purely static technique in our study is based on border analysis with minimal border methods which avoids analyzing JDK and saves, on average, 14.1% of the end-to-end time of RetestAll. Furthermore, the results show that a hybrid technique based on per-test analysis is very promising in terms of safety and precision. On the other hand, the worst techniques were based on string analysis; these techniques are imprecise and often lead to selecting to re-run all tests. Taken together, these results show the need for more research into purely static techniques for making static RTS reflection aware.

Graduation Semester

2018-05

Type of Resource

text

Permalink

http://hdl.handle.net/2142/101065

Copyright and License Information

Copyright 2018 Milica Hadzi-Tanovic

Owning Collections