Understanding multisensory processing in a wider context using a model-based approach

Abstract

An overarching goal of multisensory research is to understand multisensory behaviour in a wider context but still within a unified framework. Modelling has been a useful tool to understand how information from different senses is combined to produce behaviour. For example, different models have been used to understand the improvement in speed or precision of multisensory relative to unisensory decisions (Ernst & Banks, 2002; Raab, 1962). However, these modelling approaches are limited to address a single measure of multisensory decisions, for example, models of speed cannot explain precision and models of precision cannot explain speed. The field is still working towards developing a unified modelling framework that can link the different aspects of multisensory decisions together. In aim of this, I addressed three gaps in research on multisensory decision making, which I believe is essential to address the missing linkage between studies of multisensory behaviour. Firstly, most multisensory modelling work has used an implicit assumption, which states that processing of a unisensory signal and processing of the corresponding unisensory component of a multisensory signals are identical. However, the validity of this assumption, has not been tested. I offered a way to understand this assumption under experimental settings and proved that it is most likely violated. This suggests that studies of multisensory behaviour should rather consider the influence of context in modelling work. Secondly, I addressed that multisensory response at neuronal and behavioural level can be linked. The spatial principle, which is found in neuronal studies, can be mapped to behaviour and accounted for by a race model. Thirdly, I attempt to address the link between speed and precision of multisensory behaviour. In the past, the two have rarely been studied within a unified paradigm. I found that when both measures were required, participants are far from achieving optimality in either speed or precision, and it is possible that they are switching strategies towards favouring one of the measures in such decisions. The race model is a strong candidate to be refined and developed in future research to incorporate more aspects of multisensory decisions, as I have address it is not limited to explain speedup in simple detection task, but also influence of context, space, and precision in more complex experimental tasks.

Date of Award	2 Jul 2021
Original language	English
Awarding Institution	University of St Andrews
Supervisor	Thomas Otto (Supervisor)