I am interested in learning and decision making from a computational and cognitive perspective. I work on the following 3 broad topics:
- Function and reinforcement learning in humans and machines:
If you have ever talked to me or seen me presenting my work, you will know that –sooner or later– I will almost surely start talking about Gaussian processes (or GPs as I call them affectionately). Gaussian process regression is a non-parametric regression method that can be used to model distributions over functions in a Bayesian way, as described in more detail in our tutorial article.
I use GPs to assess how human subjects approximate unknown functions, especially when the goals it to learn about a function while generating rewards at the same time. Within a diverse set of experiments, such as the contextual multi-armed bandit, the spatially-correlated multi-armed bandits, as well as in safe optimization tasks, we have found that participants seem to optimize unknown functions by trading off between an option’s expected reward and its attached uncertainty while being slightly pessimistic about the function’s smoothness. This lead to the question of why a sensible agent (aka humans) might be so pessimistic about the underlying function and we therefore assessed how the different machine learning tools perform in scenarios of mismatched optimization. To our surprise, it can actually sometimes pay off to be rather pessimistic than optimistic, at least within Bayesian optimization scenarios (this is not a recommendation for life in general!). We are currently investigating how to overcome mismatched assumptions by utilizing Deep Gaussian Processes.
Recently, we have also started assessing human behavior in more complex tasks such as in adaptive control scenarios. In the (hopefully near) future, my plan is to build up a web page that contains human behavioral data across a diverse set of computer games, readily available for others to download and analyze, a little like OpenAI’s universe but with human agents instead. Current working title is Earth but it’s hard to buy the IP rights for that.
- Compositionality of cognition
The center piece of my PhD-thesis defined and tested a compositional theory of function learning. The main idea behind this approach is that people make sense of complex functions by decomposing them into simpler parts, that can be re-combined and re-used later on. Our model seems to explain experimental data across a series of experiments (from pattern completion to memory tasks) reasonably well. Moreover, if people’s functional inductive biases are indeed compositional, then that might explain why they are able to learn as fast about novel functions as they do, for example in one-shot-learning tasks.
I have also played around with a compositional model of how tree-like heuristics can emerge via Approximate Bayesian Computation, where different nodes and rules are combined and assessed in sub-sampled learning sets paired with probabilistic updates.
Currently, we are planning to assess how people solve computational problems via compositional meta-programming and I hope to have results on this soon (hopefully).
- Algorithmic rationality
Together with Ishita Dasgupta and Sam Gershman, I am also working on the question of how people generate and evaluate hypotheses from a computationally rational perspective. More specifically, we have developed a sampling-based model of human hypothesis generation that can explain many of the “biases” described in the literature by assuming that people generate a finite number of samples. Moreover, we have recently used this model to assess if and how people might cache and re-use prior samples in tasks allowing for amortized hypothesis generation.
We are currently planning to follow up on our amortization experiments as well as potentially assessing the neural substrates of the amortization effect.
If you are interested in any of these or related topics, simply get in contact.