TY - JOUR
T1 - Blending Bayesian and classical tools to define optimal sample-size-dependent significance levels
AU - Gannon, Mark Andrew
AU - de Braganca Pereira, Carlos Alberto
AU - Polpo, Adriano
PY - 2019/3/29
Y1 - 2019/3/29
N2 - This article argues that researchers do not need to completely abandon the p-value, the best-known significance index, but should instead stop using significance levels that do not depend on sample sizes. A testing procedure is developed using a mixture of frequentist and Bayesian tools, with a significance level that is a function of sample size, obtained from a generalized form of the Neyman-Pearson Lemma that minimizes a linear combination of alpha, the probability of rejecting a true null hypothesis, and ,beta, the probability of failing to reject a false null, instead of fixing alpha and minimizing beta. The resulting hypothesis tests do not violate the Likelihood Principle and do not require any constraints on the dimensionalities of the sample space and parameter space. The procedure includes an ordering of the entire sample space and uses predictive probability (density) functions, allowing for testing of both simple and compound hypotheses. Accessible examples are presented to highlight specific characteristics of the new tests.
AB - This article argues that researchers do not need to completely abandon the p-value, the best-known significance index, but should instead stop using significance levels that do not depend on sample sizes. A testing procedure is developed using a mixture of frequentist and Bayesian tools, with a significance level that is a function of sample size, obtained from a generalized form of the Neyman-Pearson Lemma that minimizes a linear combination of alpha, the probability of rejecting a true null hypothesis, and ,beta, the probability of failing to reject a false null, instead of fixing alpha and minimizing beta. The resulting hypothesis tests do not violate the Likelihood Principle and do not require any constraints on the dimensionalities of the sample space and parameter space. The procedure includes an ordering of the entire sample space and uses predictive probability (density) functions, allowing for testing of both simple and compound hypotheses. Accessible examples are presented to highlight specific characteristics of the new tests.
KW - Hardy-Weinberg equilibrium
KW - Neyman-Pearson lemma
KW - Predictive distribution
KW - Significance test
KW - TESTS
U2 - 10.1080/00031305.2018.1518268
DO - 10.1080/00031305.2018.1518268
M3 - Article
SN - 0003-1305
VL - 73
SP - 213
EP - 222
JO - American Statistician
JF - American Statistician
IS - sup1
ER -