Uncategorized

LNIRT NCME Training Session April 4 2019

A one-day training session will be given using the LNIRT R-package for the joint modeling of response accuracy and response times.

The 2019 Pre-Conference Training Sessions of the NVME 2019 (Toronto) will be held at the Fairmont Royal York on Thursday April 4th, from 8:00am to 5:00pm.

Recently, a new version LNIRT 0.4.0 has been developed. We will use this version at the training, which can be shared at the session (if it is not yet on CRAN). The option to define joint models for incomplete designs have been integrated in the program. The output of an LNIRT object has been improved with EAP-estimates and MCMC chains of the parameters. The MCMC chains can be easily analyzed with the Coda package. A link will be provided to get the course material during the training (I will not distribute paper copies).

We have updated the software description in a Rmarkdown LNIRT document, you can find the latest version here: LNIRT-Demo-2019

Large scale testing programs in educational measurement often use response accuracy (RA) and response time (RT) data to make inferences about test taker’s ability and speed, respectively. Computer-based testing offers the possibility to collect item-response time information, by recording the total time spent on each item. Together with the RA data, this kind of information can be used in test design to make more profound inferences about response behavior of the candidates.

When following the popular modeling framework of van der Linden (2007) and Klein Entink, Fox, and van der Linden (2009), Fox, Klein Entink, and van der Linden (2007), and van der Linden and Fox (2016), joint models are constructed by connecting an IRT model with an RT model thereby defining a relationship between the person and item parameters. The RT model is defined from the same perspective as an IRT model. A random person parameter, representing the speed of working, is used to model dependencies between RTs. Furthermore, item-specific parameters are introduced to model characteristics with respect to the average time to solve the item. This separation of parameters makes it possible to make inferences about response behavior as well the test characteristics.

With the R-package LNIRT (Fox, Entink, & Klotzke, 2016), an open-source software solution is presented to simultaneously analyze responses and RTs in an IRT modeling framework. The item parameters of the joint model, i.e. item difficulty, item discrimination, time intensity and time discrimination are related through a common covariance structure. Similarly, a correlation parameter explains the relationship between the person parameters, i.e. ability and speed. Estimation of the parameters and the covariance structures is done within a Bayesian framework. Finally, covariates can be specified for the item and person parameters and a guessing parameter can be included.

The proposed training session conveys an understanding of the theoretical foundation of integrating responses and RTs in a hierarchical nonlinear and generalized linear modeling framework. More concrete, the participants learn how a two-parameter or three-parameter IRT model and the log-normal RT model are connected through a common multivariate distribution for each the item and the person parameters. Furthermore, the participants learn about the capabilities and the practical applications of the LNIRT software. Finally, attention is paid to the validity of inferences made from sequences of Markov Chain Monte Carlo (MCMC) samples and the utility of convergence diagnostics in the given context. For this purpose, the R-package coda will be utilized.

Literature.

Fox, J.-P., Marianti, S. (2017). Person-Fit Statistics for Joint Models for Accuracy and Speed,  Journal of Educational Measurement. DOI: 10.1111/jedm.12143. Volume 54, Issue 3, 394.

Fox, J.-P., and Marianti, S. (2016). Joint modeling of ability and differential speed using responses and response times. Multivariate Behavioural Research. http://dx.doi.org/10.1080/00273171.2016.1171128.

Fox, J.-P., Klein Entink, R. E., Klotzke, K. (2016). R-Package LNIRT: Lognormal response time item response theory models. https://cran.r-project.org/web/packages/LNIRT/index.html.

Fox, J.-P., Klein Entink, R.H., van der Linden, W.J. (2007). Modeling of responses and response times with the package cirt. Journal of Statistical Software, 20, issue 7.

Klein Entink, R.H., Fox, J.-P., van der Linden, W.J. (2009). A multivariate multilevel approach to the modeling of accuracy and speed of test takers. Psychometrika, 74, 21-48

van der Linden, W.J. and Fox, J.-P. (2016) Joint hierarchical modeling of responses and response times. In Handbook of Modern Item Response Theory, W.J van der Linden (Ed.), Vol 1, Chapter 29, Chapman and Hall/CRC Press.

van der Linden, W.J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika,72, 287–308.

Leave a Reply