Withdraw
Loading…
Asymptotically robust permutation tests for two-sample goodness-of-fit testing problems
Olivares Gonzalez, Mauricio
This item is only available for download by members of the University of Illinois community. Students, faculty, and staff at the U of I may log in with your NetID and password to view the item. If you are trying to access an Illinois-restricted dissertation or thesis, you can request a copy through your library's Inter-Library Loan office or purchase a copy directly from ProQuest.
Permalink
https://hdl.handle.net/2142/115521
Description
- Title
- Asymptotically robust permutation tests for two-sample goodness-of-fit testing problems
- Author(s)
- Olivares Gonzalez, Mauricio
- Issue Date
- 2022-04-14
- Director of Research (if dissertation) or Advisor (if thesis)
- Chung, EunYi
- Doctoral Committee Chair(s)
- Chung, EunYi
- Committee Member(s)
- Koenker, Roger
- Shao, Xiaofeng
- Lee, Ji Hyung
- Department of Study
- Economics
- Discipline
- Economics
- Degree Granting Institution
- University of Illinois at Urbana-Champaign
- Degree Name
- Ph.D.
- Degree Level
- Dissertation
- Keyword(s)
- Permutation Tests
- Goodness-of-fit Testing
- Heterogeneous Treatment Effects
- Covariate-Adaptive Randomization
- Abstract
- The three chapters of this dissertation introduce novel permutation tests for two relevant hypotheses in economics: heterogeneous treatment effects and equality of distributions under covariate-adaptive randomization (CAR). Permutation tests have several attractive properties. First, they are exact in finite samples under the randomization hypothesis. Second, they are easy to implement since they are nonparametric and their construction is the same regardless of the test statistic. Third, they are asymptotically as powerful as standard optimal procedures for many testing problems. Last, even when the randomization hypothesis does not hold, permutation tests could still be asymptotically valid for general non-exchangeable settings if we replace the test statistic used in the permutation test by a test statistic that is asymptotically distribution-free. Despite these advantages, the direct application of permutation-based inference to the problems we investigate in this dissertation is not automatic. The reason is two-fold. First, the randomization hypothesis is hard to justify in the testing problems we consider. Second, studentization in higher dimensions is difficult since we consider two-sample goodness-of-fit hypotheses with associated Kolmogorov-Smirnov-type test statistics. Thus, establishing the asymptotic validity of the permutation test is not straightforward either. This work provides asymptotic arguments to handle these difficulties by leveraging empirical process techniques. Throughout this thesis, we will invoke martingale transformations and prepivoting to restore the asymptotic pivotality of the test statistic. The main contribution of this work is to show these techniques provide a path forward to carry on asymptotically valid inference for testing heterogeneous treatment effects and equality of distributions under CAR. Below are the individual abstracts for each chapter. Chapter 1: Robust Permutation Test for Equality of Distributions under Covariate- Adaptive Randomization Though stratified randomization achieves more balance on baseline covariates than pure randomization, it does affect the way we conduct inference. This paper considers the classical two-sample goodness-of-fit testing problem in randomized controlled trials (RCTs) when the researcher employs a particular type of stratified randomization—covariate-adaptive randomization (CAR). When testing the null hypothesis of equality of distributions between experimental groups in this setup, we first show that stratification leaves a mark on the test statistic’s limit distribution, making it difficult, if not impossible, to obtain critical values. We instead propose an alternative approach to conducting inference based on a permutation test that i) is asymptotically exact in the sense that the limiting rejection probability under the null hypothesis equals the nominal α level, ii) is viable under relatively weak assumptions commonly satisfied in practice, iii) works for randomization schemes that are popular among empirically oriented researchers, and iv) is applicable without requiring knowledge about the pre-treatment covariates used for stratification. The proposed test’s main idea is that by transforming the original statistic by one minus its bootstrap p-value, it becomes asymptotically uniformly distributed on [0,1]. Thus, the transformed test statistic—also called prepivoted—has a fixed limit distribution that is free of unknown parameters, effectively removing the effect of stratification. Consequently, a permutation test based on the prepivoted statistic produces a test whose limiting rejection probability equals the nominal level. We present further numerical evidence of the proposed test’s advantages in a Monte Carlo exercise, showing our permutation test outperforms the existing alternatives. We illustrate our method’s empirical relevance by revisiting a field experiment by Butler and Broockman (2011) on the effect of race on state legislators' responsiveness to help their constituents register to vote during elections in the United States. Lastly, we provide the companion RATest R package to facilitate and encourage applying our test in empirical research. Chapter 2: Permutation Test for Heterogeneous Treatment Effects with a Nuisance Parameter This paper proposes an asymptotically valid permutation test for heterogeneous treatment effects in the presence of an estimated nuisance parameter. Not accounting for the estimation error of the nuisance parameter results in statistics that depend on the particulars of the data generating process, and the resulting permutation test fails to control the Type 1 error, even asymptotically. In this paper we consider a permutation test based on the martingale transformation of the empirical process to render an asymptotically pivotal statistic, effectively nullifying the effect associated with the estimation error on the limiting distribution of the statistic. Under weak conditions, we show that the permutation test based on the martingale-transformed statistic results in the asymptotic rejection probability of α in general while retaining the exact control of the test level when testing for the more restrictive sharp null. We also show how our martingale-based permutation test extends to testing whether there exists treatment effect heterogeneity within subgroups defined by observable covariates. Our approach comprises testing the joint null hypothesis that treatment effects are constant within mutually exclusive subgroups while allowing the treatment effects to vary across subgroups. Monte Carlo simulations show that the permutation test presented here performs well in finite samples, and is comparable to those existing in the literature. To gain further understanding of the test to practical problems, we investigate the gift exchange hypothesis in the context of two field experiments from Gneezy and List (2006). Lastly, we provide the companion RATest R package to facilitate and encourage the application of our test in empirical research. Chapter 3: Quantile-based Test for Heterogeneous Treatment Effects One way to look at the distributional effects of a policy intervention comprises estimating the quantile treatment effect at different quantiles. We exploit this idea and develop a new permutation test for heterogeneous treatment effects based on a modified quantile process. To establish the asymptotic validity of our test, we transform the test statistic using a martingale transformation so that its limit behavior is distribution-free. Numerical evidence shows our permutation test outmatches other popular quantile-based tests in terms of size and power performance. We discuss a fast implementation algorithm and illustrate our method using experimental data from a welfare reform.
- Graduation Semester
- 2022-05
- Type of Resource
- Thesis
- Copyright and License Information
- Copyright 2022 Mauricio Olivares Gonzalez
Owning Collections
Graduate Dissertations and Theses at Illinois PRIMARY
Graduate Theses and Dissertations at IllinoisManage Files
Loading…
Edit Collection Membership
Loading…
Edit Metadata
Loading…
Edit Properties
Loading…
Embargoes
Loading…