A different view on evaluating the performance of bootstrap critical values

Abstract

Two methods to evaluate the performance of bootstrap-based tests are discussed. The first is one that is traditionally used in the literature, while the second is an alternative method that we propose. To illustrate the differences between the two evaluation methods, we consider testing the mean of a population. Theoretical properties of the corresponding bootstrap critical values are derived. The results of a simulation study are presented to provide empirical comparison of the two techniques with regard to size and power of the test, and to substantiate the theoretical findings. It is found, among others, that the new evaluation method can detect defects of bootstrap critical values that cannot be observed if the current evaluation method is employed. Similar results hold for the "smooth function model", which allows us to study testing problems where the parameter is for example a variance, a ratio of means or variances, or a correlation coefficient