A New Class of Multiprocess Responses and Response Time Models

Project: Research

View graph of relations

Description

In recent decades, item response theory (IRT) models have been extended to incorporate response time (RT) during educational testing, which has been found to improve validity and assessment accuracy. However, most existing approaches target achievement tests and cannot be directly adopted for survey questionnaires because they differ both in nature and by design. Most studies of achievement tests have assumed that accuracy is affected by response speed due to their limited administration time, and often a linear relationship between the intended-to-be-measured latent trait and RT entails a trade-off effect between accuracy and speed. However, this linear relationship may not be applicable to personality and attitudinal scales, as empirical studies have suggested an inverted-U relationship between the measured trait and RT. In addition, existing RT models often implicitly assume that all examinees use the same strategy to respond to items, thereby ignoring a wide variety of response behaviors when respondents answer survey questions (e.g., the tendency to choose or avoid extreme response categories or to use random responses). Moreover, there is no consensus on the relationship between the measured attitude/ability, response behaviors, and RTs, which requires further investigation of these issues.This project will aim to address the limitations of existing RT models using an IRT framework to develop a new branch of RT models for survey questionnaires. The proposed project will consist of three studies, formulating a new RT framework to accommodate various response behaviors in survey research, developing a non-dominance multiprocess IRT model, and integrating the non-dominance multiprocess IRT model with RT. We will adopt current multiprocess IRT models that simultaneously estimate the latent ability and the response preference of respondents to formulate a general form while considering the impact of item design (called the unfolding decision tree model, UDtree model). The proposed RT models will then be integrated with the UDtree model to depict the cognitive process entailed in choosing a specific response category on Likert scales and to examine the relationship between response preference, latent ability, and RTs.The proposed study will go beyond existing psychometric theories to model questionnaire design and respondent behaviors by incorporating RT into psychometric models. Upon successful completion, powerful alternative approaches will have been developed to investigate mental activities and response behaviors during attitude and personality assessments. In turn, this will advance our understanding of cognitive processes when answering survey questionnaires and may lead to further studies on decision-making. In addition, the new methodology will significantly contribute to item quality assurance. In practice, our proposed model will provide the speed traits of the respondents, which could be used as indicators to detect aberrant responses from unmotivated respondents who do not pay attention to test items and simply want to complete the questionnaire in a short time. This method will also provide item speed parameters that could be used for item selection when a shorter scale is preferred to reduce fatigue and inattentive responses during a long questionnaire or assessment. This will be particularly helpful in medicine, public policy, and marketing. For instance, clinical practitioners need a streamlined tool to quickly measure the mobility of acute stroke patients for immediate treatment. This new methodology will make it possible to select informative items providing similar information to other items but requiring shorter RTs. 

Detail(s)

Project number9043264
Grant typeGRF
StatusNot started
Effective start/end date1/01/22 → …