To simulate or not? Comment on Steingroever, Wetzels, and Wagenmakers (2014)

Konstantinidis, E., Speekenbrink, M., Stout, J. C., Ahn, W.-Y., Shanks, D. R. 2014. Decision, 1(3), 184-191.


Steingroever, Wetzels, and Wagenmakers (2014) conducted a detailed investigation of 3 popular reinforcement-learning models for the Iowa gambling task using 2 model comparison techniques: a post hoc fit criterion and a simulation method. However, these 2 methods yield inconsistent results regarding which model should be preferred as a description of underlying psychological processes. Here, we describe the benefits of each method in an attempt to develop a more balanced view of how to utilize these model comparison techniques, and we outline the risks of focusing on a single method to make inferences about the overall utility of a model. Also, we make several suggestions about how applied research should evaluate candidate cognitive models, and we offer guidelines for future research aimed at identifying “good” models for decomposing and explaining participants’ performance.