The Exact Size of the Chi-Squared Test for Comparing Two Binomial Proportions

Thursday, March 26, 2009

Speaker: Roger L. Berger

Title: The Exact Size of the Chi-Squared Test for Comparing Two Binomial Proportions

Abstract: The chi-squared test is commonly used to test the homogeneity of two binomial proportions. We will show by accurate computations and analytical proof that even for very large sample sizes the exact size of this test can be 80% larger than the nominal level. For example, a nominal .05 test can have a size of over .09. Whether the test has this inflated size depends only on the ratio of the binomial sample sizes. A closer examination of the error properties of this test may indicate that this inflation of size may not be of great concern. We will compare the size of the chi-squared test to an exact unconditional test that has remarkably good size properties.