Understanding multisensory processing in a wider context using a model-based approach

  • Yue Liu

Student thesis: Doctoral Thesis (PhD)


An overarching goal of multisensory research is to understand multisensory behaviour in a wider context but still within a unified framework. Modelling has been a useful tool to understand how information from different senses is combined to produce behaviour. For example, different models have been used to understand the improvement in speed or precision of multisensory relative to unisensory decisions (Ernst & Banks, 2002; Raab, 1962). However, these modelling approaches are limited to address a single measure of multisensory decisions, for example, models of speed cannot explain precision and models of precision cannot explain speed. The field is still working towards developing a unified modelling framework that can link the different aspects of multisensory decisions together. In aim of this, I addressed three gaps in research on multisensory decision making, which I believe is essential to address the missing linkage between studies of multisensory behaviour. Firstly, most multisensory modelling work has used an implicit assumption, which states that processing of a unisensory signal and processing of the corresponding unisensory component of a multisensory signals are identical. However, the validity of this assumption, has not been tested. I offered a way to understand this assumption under experimental settings and proved that it is most likely violated. This suggests that studies of multisensory behaviour should rather consider the influence of context in modelling work. Secondly, I addressed that multisensory response at neuronal and behavioural level can be linked. The spatial principle, which is found in neuronal studies, can be mapped to behaviour and accounted for by a race model. Thirdly, I attempt to address the link between speed and precision of multisensory behaviour. In the past, the two have rarely been studied within a unified paradigm. I found that when both measures were required, participants are far from achieving optimality in either speed or precision, and it is possible that they are switching strategies towards favouring one of the measures in such decisions. The race model is a strong candidate to be refined and developed in future research to incorporate more aspects of multisensory decisions, as I have address it is not limited to explain speedup in simple detection task, but also influence of context, space, and precision in more complex experimental tasks.
Date of Award2 Jul 2021
Original languageEnglish
Awarding Institution
  • University of St Andrews
SupervisorThomas Otto (Supervisor)


  • Multisensory decision
  • Race model
  • Audiovisual
  • Space
  • RT
  • Precision
  • Maximum likelihood estimation

Access Status

  • Full text embargoed until
  • 11th June 2022

Cite this