Adaptive information source selection during hypothesis testing

Abstract

We consider how the information sources people use to test hypotheses change as the sparsity of the hypotheses – the proportion of items in the hypothesis space they include – changes. Specifically, we focus on understanding how requests for positive and negative evidence, which have been shown to be sensitive to hypothesis sparsity (Hendrickson, Navarro, & Perfors, in prep), are influenced by requests for specific instances, which show a positive bias and less sensitivity to sparsity (Markant & Gureckis, 2013). We find that people modify their information requests as a function of the sparsity of the hypotheses and they do so in this task primarily by by manipulating the rate of requesting positive and negative evidence. Furthermore, by simulating the set of possible remaining hypotheses, we find that people were most likely to select the information source that maximized the expected reduction in uncertainty across hypotheses. We conclude by discussing the implications of these results for models of hypothesis testing.