A Monte Carlo Study of the Effects of Common Method Variance on Significance Testing and Parameter Bias in Hierarchical Linear Modeling

Research output: Journal Publications and Reviews (RGC: 21, 22, 62)21_Publication in refereed journal

49 Scopus Citations
View graph of relations

Author(s)

  • Xin Lai
  • Fuli Li
  • Kwok Leung

Related Research Unit(s)

Detail(s)

Original languageEnglish
Pages (from-to)243-269
Journal / PublicationOrganizational Research Methods
Volume16
Issue number2
Online published8 Jan 2013
Publication statusPublished - 1 Apr 2013

Abstract

Despite that common method variance (CMV) is widely regarded as a serious threat to the validity of findings based on self-reports, there is insufficient research on its confounding influence. We extend Evans's (1985) pioneering work, and the more recent works by Ostroff, Kinicki, and Clark (2002) and Siemsen, Roth, and Oliveira (2010), to delineate the influence of CMV in a two-level hierarchical linear model based on self-report data. Our simulation results clearly show that in the absence of true effects, it is extremely unlikely for CMV to generate significant cross-level interactions. In fact, if a true cross-level interaction exists, CMV tends to lower the likelihood of its identification and erroneously underestimate the regression coefficient. Our simulation results also show that CMV may lead to a false significant cross-level main effect and overestimate the regression coefficient when no true effect exists. To reduce the probability of Type I errors, we show that raising the significance level to.01, the split sample strategy, and the addition of more CMV contaminated variables are effective in the vast majority of real-life situations and are more effective than increasing the number of groups or persons in each group. Both the split sample strategy and the addition of more CMV contaminated variables are also effective in reducing parameter bias when no true cross-level main effect exists. Trade-offs associated with different strategies are discussed. © The Author(s) 2013.

Research Area(s)

  • common method variance, cross-level relationships, hierarchical linear modeling, Monte Carlo approach, self-report data