On two-stage Monte Carlo tests of composite hypotheses

Adrian Baddeley, Andrew Hardegen, Thomas Lawrence, Robin K. Milne, Gopalan Nair, Suman Rakshit

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

A major weakness of the classical Monte Carlo test is that it is biased when the null hypothesis is composite. This problem persists even when the number of simulations tends to infinity. A standard remedy is to perform a double bootstrap test involving two stages of Monte Carlo simulation: under suitable conditions, this test is asymptotically exact for any fixed significance level. However, the two-stage test is shown to perform poorly in some common applications: for a given number of simulations, the test with the smallest achievable significance level can be strongly biased. A ‘balanced’ version of the two-stage test is proposed, which is exact, for all achievable significance levels, when the null hypothesis is simple, and which performs well for composite null hypotheses.

Original languageEnglish
Pages (from-to)75-87
Number of pages13
JournalComputational Statistics and Data Analysis
Volume114
DOIs
Publication statusPublished - 1 Oct 2017

Fingerprint

Monte Carlo Test
Composite Hypothesis
Significance level
Null hypothesis
Composite materials
Biased
Double Bootstrap
Composite
Bootstrap Test
Simulation
Monte Carlo Simulation
Infinity
Tend
Monte Carlo simulation

Cite this

Baddeley, Adrian ; Hardegen, Andrew ; Lawrence, Thomas ; Milne, Robin K. ; Nair, Gopalan ; Rakshit, Suman. / On two-stage Monte Carlo tests of composite hypotheses. In: Computational Statistics and Data Analysis. 2017 ; Vol. 114. pp. 75-87.
@article{053177394f234afea1df35b51cd7770a,
title = "On two-stage Monte Carlo tests of composite hypotheses",
abstract = "A major weakness of the classical Monte Carlo test is that it is biased when the null hypothesis is composite. This problem persists even when the number of simulations tends to infinity. A standard remedy is to perform a double bootstrap test involving two stages of Monte Carlo simulation: under suitable conditions, this test is asymptotically exact for any fixed significance level. However, the two-stage test is shown to perform poorly in some common applications: for a given number of simulations, the test with the smallest achievable significance level can be strongly biased. A ‘balanced’ version of the two-stage test is proposed, which is exact, for all achievable significance levels, when the null hypothesis is simple, and which performs well for composite null hypotheses.",
keywords = "Conservative test, Dao–Genton test, Double bootstrap, Nested bootstrap, p-value",
author = "Adrian Baddeley and Andrew Hardegen and Thomas Lawrence and Milne, {Robin K.} and Gopalan Nair and Suman Rakshit",
year = "2017",
month = "10",
day = "1",
doi = "10.1016/j.csda.2017.04.003",
language = "English",
volume = "114",
pages = "75--87",
journal = "Computational Statistics & Data Analysis",
issn = "0167-9473",
publisher = "Elsevier",

}

On two-stage Monte Carlo tests of composite hypotheses. / Baddeley, Adrian; Hardegen, Andrew; Lawrence, Thomas; Milne, Robin K.; Nair, Gopalan; Rakshit, Suman.

In: Computational Statistics and Data Analysis, Vol. 114, 01.10.2017, p. 75-87.

Research output: Contribution to journalArticle

TY - JOUR

T1 - On two-stage Monte Carlo tests of composite hypotheses

AU - Baddeley, Adrian

AU - Hardegen, Andrew

AU - Lawrence, Thomas

AU - Milne, Robin K.

AU - Nair, Gopalan

AU - Rakshit, Suman

PY - 2017/10/1

Y1 - 2017/10/1

N2 - A major weakness of the classical Monte Carlo test is that it is biased when the null hypothesis is composite. This problem persists even when the number of simulations tends to infinity. A standard remedy is to perform a double bootstrap test involving two stages of Monte Carlo simulation: under suitable conditions, this test is asymptotically exact for any fixed significance level. However, the two-stage test is shown to perform poorly in some common applications: for a given number of simulations, the test with the smallest achievable significance level can be strongly biased. A ‘balanced’ version of the two-stage test is proposed, which is exact, for all achievable significance levels, when the null hypothesis is simple, and which performs well for composite null hypotheses.

AB - A major weakness of the classical Monte Carlo test is that it is biased when the null hypothesis is composite. This problem persists even when the number of simulations tends to infinity. A standard remedy is to perform a double bootstrap test involving two stages of Monte Carlo simulation: under suitable conditions, this test is asymptotically exact for any fixed significance level. However, the two-stage test is shown to perform poorly in some common applications: for a given number of simulations, the test with the smallest achievable significance level can be strongly biased. A ‘balanced’ version of the two-stage test is proposed, which is exact, for all achievable significance levels, when the null hypothesis is simple, and which performs well for composite null hypotheses.

KW - Conservative test

KW - Dao–Genton test

KW - Double bootstrap

KW - Nested bootstrap

KW - p-value

UR - http://www.scopus.com/inward/record.url?scp=85018748177&partnerID=8YFLogxK

U2 - 10.1016/j.csda.2017.04.003

DO - 10.1016/j.csda.2017.04.003

M3 - Article

VL - 114

SP - 75

EP - 87

JO - Computational Statistics & Data Analysis

JF - Computational Statistics & Data Analysis

SN - 0167-9473

ER -