Putting bandits into context (pre-print)

A pre-print of our contextual multi-armed bandit paper is now available on the biorxiv here. We model participants’ behavior in contextual multi-armed bandit tasks, that is tasks that require both function learning and decision making, by using a combination of Gaussian process regression and a diverse set of acquisition functions. We find that participants’ learning is universal but very local and that they seem to directly trade-off between rewards and uncertainty.

Student Award for Outstanding Scientific Contribution

Ishita Dasgupta has received the Student Award for Outstanding Scientific Contribution at the International Conference on Thinking for our joint work on stochastic hypothesis generation. Below is the abstract of the presented talk:

 

Ishita Dasgupta, Eric Schulz & Samuel Gershman

ABSTRACT. How do humans approximate Bayesian inference when the task requires them to generate hypotheses? Previous models (e.g., Thomas et al., 2008) crucially depend on cued recall to generate hypotheses. However, this strategy is impractical in combinatorially complex hypothesis spaces, where relevant hypotheses may have to be constructed de novo rather than retrieved from memory. We present a novel algorithmic model of hypothesis generation based on Markov chain Monte Carlo sampling. The Markov chain generates hypotheses using local proposals and accepts them based on their probability. The accepted hypotheses are then used to construct a sample-based approximation of the posterior. As the number of generated hypotheses increases, the approximation converges to the true posterior. However, following Lieder et al. (2013), we assume that humans run the chain for a finite length, producing several well-known probability biases such as subadditivity, superadditivity, variance in responses, and anchoring. Additionally, our model makes a new prediction: the context-dependent inversion of subadditivity and superadditivity. These simulations suggest that resource-bounded sampling provides a plausible account of human hypothesis generation.

ICT & CogSci

Below are the presentations and poster I’ve presented at the International Conference on Thinking and at the Annual Meeting of the Cognitive Science Society:

 

ICT:

  1. Exploration and exploitation in contextual multi-armed bandits [TALK]
  2. The compositional nature of intuitive functions [TALK]

 

CogSci:

  1. Better safe than sorry: Risky function exploitation through safe optimization [POSTER]
  2. Simple trees in complex forests: Growing Take The Best by Approximate Bayesian Computation [TALK]

Exploring the Unknown

Charley Wu, a PhD-student at MPI Berlin, will present our joint work about using Gaussian Process models to describe human spatial exploration-exploitation behavior at the 2016 Summer School on Computational and Mathematical Modeling of Cognition in Dobiacco, Italy. The poster can be found here: [PDF]

Pre-print of our “compositionality of intuitive functions”-paper

A pre-print of our paper “Probing the Compositionality of Intuitive Functions” is available on the Center for Brains, Minds & Machines page here

The paper contains a couple of experiments that we have run during my research stay at Harvard/MIT. These experiments tried to elicit whether human inductive biases of functions can be seen as compositional by nature.

For this, we assessed human intuitions of functions by a Gaussian Process regression framework, parameterized by three different kernels: a “one-size-fits-all” radial basis kernel, a “non-parametric all the way” spectral mixture kernel, and a compositional kernel made of different consecutive building blocks. We find that when choosing different extrapolations, when performing Markov chain Monte Carlo with people, when generating extrapolations manually as well as when deciding which functions are more predictable, people always seem to have a bias for the compositional kernel.

As compositionality speeds up intuitive inference, this might explain why participants are able to learn functions efficiently in the real world.