Paper on Inference with Few Clusters

Screen Shot 2014-06-27 at 2.28.49 PM
We just finished the first draft of the paper Randomization Tests Under an Approximate Symmetry Assumption . This paper develops a theory of randomization tests under an approximate symmetry assumption. Randomization tests provide a general means of constructing tests that control size in finite samples whenever the distribution of the observed data exhibits symmetry under the null hypothesis. Here, by symmetry we mean that the distribution of the observed data remains invariant with respect to a group of transformations. In this paper, we provide conditions under which the same construction can be used to construct tests that asymptotically control size whenever the distribution of the observed data exhibits approximate symmetry in the sense that the limiting distribution of a function of the data exhibits symmetry under the null hypothesis. An important application of this idea is in settings where the data may be grouped into a fixed number of “clusters” with a large number of observations within each cluster. In such settings, we show that the distribution of the observed data satisfies our approximate symmetry requirement under weak assumptions. In particular, our results allow for the clusters to be heterogeneous and also have dependence not only within each cluster, but also across clusters. We show that our approach enjoys several advantages over other approaches in these settings. Among other things, our approach leads to a test that is asymptotically similar, which, as shown in a simulation study, translates into improved power at many alternatives. Finally, we use our results to revisit the analysis of Angrist and Lavy (2009), who examine the impact of a cash award on exam performance for low-achievement students in Israel.