Inference for linear regression models in the presence of heteroskedasticity

Abstract

The linear regression model is a highly versatile tool with which one can model a response variable in terms of one or more predictor variables. The classical linear model is based on six primary theoretical assumptions. In this study the main focus is on the assumption of “homoskedasticity” and the violation thereof, called “heteroskedasticity”. Heteroskedasticity refers to the property where the error terms in the linear model do not all have the same variance. This text explores methods for conducting hypothesis tests of regression coefficients in the presence of heteroskedasticity. Various approaches for performing inference in the presence of heteroskedasticity are investigated, including tests that incorporate heteroskedasticity consistent covariance matrix estimators (HCCMEs), the wild bootstrap, and a newly proposed method based on a modification of the bootstrap residuals approach. A simulation study was conducted to compare the proposed new modified bootstrap residuals approach to the wild bootstrap in terms of size and power to determine if the new approach had any merit. The impact on the wild bootstrap’s size and power was also investigated when using different residual transformations, HCCMEs and ancillary distributions. It was found that when large leverage values were present in small homoskedastic samples, the sizes of the tests were highly elevated (tending to over-reject); the sizes of the test decreased to the specified significance level when the heteroskedasticity increased. The power of tests based on data with one or more large leverage values were more powerful than those based on data with small to moderate leverages. From the result in the study it is clear that the newly proposed modified bootstrap residuals approach is competitive compared to the wild bootstrap approach. We thus recommend that this new method be studied in more (theoretical) detail as a possible future research project.