Influences of envelope-wise visual modulation on speech perception

Comprehension of speech in background noise can be challenging, especially when suffering from hearing impairment and when relying on a hearing aid. However, witnessing a speaker’s lip-movements can increase speech comprehension. This is enabled by the integration of auditory and visual information. In speech comprehension, slow frequency oscillations in the brain, phase aligned to the speech envelope, seem to be relevant for comprehension of speech. Strikingly, recent research has found that there is cortical synchronization to unheard auditory speech when only videos of lip-movements are presented. Instead of the articulatory information, it has also been shown that listeners can benefit from a visually presented speech-envelope. The advantage of visual presentation of the auditory speech envelope is, that we reduce the information from two modalities to only auditory information. In this MEG study, we will be the first to implement a circle corresponding to the auditory speech envelope instead of the lip movements while recording underlying brain activity. For the presentation of this circle, we hypothesize that there is an enhanced comprehension at the neural level, leading to a stronger cortical speech synchronization in challenging listening conditions. For better understanding of these processes, the presentation of the timing and the location of the circle will be varied. Thereby, we will gain insights in the neuronal basis of audiovisual integration and will make first approaches towards a visually enhanced hearing aid.