Sequential multiple testing

Xing, Yiming

Permalink

https://hdl.handle.net/2142/121361 Copy

Description

Title

Sequential multiple testing

Author(s)

Xing, Yiming

Issue Date

2023-07-14

Director of Research (if dissertation) or Advisor (if thesis)

Fellouris, Georgios

Doctoral Committee Chair(s)

Fellouris, Georgios

Committee Member(s)

• Shao, Xiaofeng
• Liang, Feng
• Zhu, Ruoqing

Department of Study

Statistics

Discipline

Statistics

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

Ph.D.

Degree Level

Dissertation

Keyword(s)

• sequential multiple testing
• asymptotic optimality
• SPRT
• multistage test
• prior information
• generalized error metrics

Abstract

Given multiple data sources from which data become available sequentially and, for each data stream, given one pair of hypotheses about its underlying distribution, sequential multiple testing refers to the problem of simultaneously testing all pairs of hypotheses subject to prescribed error control. It is a statistical tool that is widely used in many scientific and industrial fields, such as signal discovery [1], anomaly detection [2], clinical trials [3], spectrum sensing in cognitive radio [4], searching for regions of interest (ROI) in an image or other mediums [5] and so on. In this thesis, three setups about the problem of sequential multiple testing are presented in three chapters. The first is a decentralized setup, where the time of stopping sampling in a stream, the time of making a decision about its testing problem, and the decision itself must be based only on observations from that stream. In this setup and when controlling the familywise error rates (FWER), simply applying a Sequential Probability Ratio Test (SPRT) with Bonferroni-corrected error probabilities to each stream can be shown to minimize the expected sample size and expected decision time in each stream as the FWER go to zero. In practice, however, there are situations where a fully-sequential test, such as the SPRT, is not suitable, as it requires continuous monitoring of the sampling process and the cost for deciding at every time instance whether to stopping sampling and making a decision can be high. In such situations, it is usually more desirable to implement a multistage test, where one only needs to check the stopping criterion and the decision rule at a small number of time instants. In Chapter 1, we propose a multistage testing procedure, named the General Multistage Test (GMT), and we show that it has the same asymptotic optimality property as the SPRT. As a comparison, existing multistage testing procedures in the literature, e.g., [1], [6], has the same property only when the two types of FWER go to zero in a symmetric way or when all streams follow their null hypotheses. The second should be the most general possible setup about sequential multiple testing based on the author’s understanding. In this setup, sampling in all streams can be continued until all decisions are made, decisions in different streams can be made at different times, and each decision can use data from all streams before the time it is made. There are two special cases. One is the decentralized setup discussed in the previous paragraph, and the other is a synchronous setup where sampling in all streams and decisions about all testing problems must be made at one single time instant. To make the problem more interesting, we incorporate prior information (exact number or lower and upper bounds) on the number of signals (true alternatives). In Chapter 2, we propose a multiple testing procedure that is asymptotically optimal in the sense that it minimizes the expected decision time simultaneously for every stream and for every subset of signals consistent with the prior information, as the FWER go to zero. As a comparison, we compute the asymptotic relative efficiency of the asymptotically optimal decentralized procedure, which is simply the parallel SPRT, and of the asymptotically optimal synchronous procedure, which was proposed in [7], and visualize the actual relative efficiencies through simulation studies. The third setup is where each decision can be made at a different time, based on available data from all streams before that time, but sampling in a stream must be stopped once a decision about its corresponding testing problem has been made. In the case of FWER and no prior information, the parallel SPRT is trivially the asymptotically optimal procedure in the sense that it minimizes the expected sample size simultaneously in every stream, as the FWER go to zero. However, when we extend the FWER to the generalized FWER [8] or incorporate prior information, things become much more complicated. In Chapter 3, we propose two multiple testing procedures, one controlling the generalized misclassification rate and the other controlling the generalized FWER, that are asymptotically optimal in the sense of minimizing the expected weighted total sample size for every subset of signals.

Graduation Semester

2023-08

Type of Resource

Thesis

Copyright and License Information

Copyright 2023 Yiming Xing

Owning Collections