Discontinuous Distribution of Test Statistics Around Significance Thresholds in Empirical Accounting Studies
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review
Author(s)
Related Research Unit(s)
Detail(s)
Original language | English |
---|---|
Journal / Publication | Journal of Accounting Research |
Online published | 30 Sept 2024 |
Publication status | Online published - 30 Sept 2024 |
Link(s)
Abstract
Examining test statistics from articles in six leading accounting journals, we detect discontinuities in their distributions around conventional significance thresholds (p-values of 0.05 and 0.01) and find an unusual abundance of test statistics that are just significant. Further analysis reveals that these discontinuities are more prominent in studies with smaller samples and are more salient in experimental than in archival studies. The discontinuity discrepancy between experimental and archival studies relates to several proxies for researcher degrees of freedom. Nevertheless, this evidence does not imply that experimental research is more prone to questionable research practices than archival studies. Overall, our findings speak to the concern of whether accounting researchers could exercise undisclosed discretion to obtain and report statistically significant results. Based on our results, a healthy skepticism of some just-significant test statistics is warranted. © 2024 The Chookaszian Accounting Research Center at the University of Chicago Booth School of Business.
Research Area(s)
- archival accounting, experimental accounting, P-value discontinuity, researcher degrees of freedom
Citation Format(s)
Discontinuous Distribution of Test Statistics Around Significance Thresholds in Empirical Accounting Studies. / CHANG, Xin; GAO, Huasheng; LI, Wei.
In: Journal of Accounting Research, 30.09.2024.
In: Journal of Accounting Research, 30.09.2024.
Research output: Journal Publications and Reviews › RGC 21 - Publication in refereed journal › peer-review