Modulation of visual responses in the superior temporal sulcus by audio-visual congruency

Christoph D. Dahl, Nikos K. Logothetis, Christoph Kayser

Research output: Contribution to journalArticle

20 Citations (Scopus)

Abstract

Our ability to identify or recognize visual objects is often enhanced by evidence provided by other sensory modalities. Yet, where and how visual object processing benefit from the information received by the other senses remains unclear. One candidate region is the temporal lobe, which features neural representations of visual objects, and in which previous studies have provided evidence for multisensory influences on neural responses. In the present study we directly tested whether visual representations in the lower bank of the superior temporal sulcus (STS) benefi t from acoustic information. To this end, we recorded neural responses in alert monkeys passively watching audio-visual scenes, and quantified the impact of simultaneously presented sounds on responses elicited by the presentation of naturalistic visual scenes. Using methods of stimulus decoding and information theory, we then asked whether the responses of STS neurons become more reliable and informative in multisensory contexts. Our results demonstrate that STS neurons are indeed sensitive to the modality composition of the sensory stimulus. Importantly, information provided by STS neurons' responses about the particular visual stimulus being presented was highest during congruent audio-visual and unimodal visual stimulation, but was reduced during incongruent bimodal stimulation. Together, these findings demonstrate that higher visual representations in the STS not only convey information about the visual input but also depend on the acoustic context of a visual scene.

Original languageEnglish
JournalFrontiers in Integrative Neuroscience
Issue numberAPRIL 2010
DOIs
Publication statusPublished - Apr 1 2010
Externally publishedYes

Fingerprint

Temporal Lobe
Neurons
Acoustics
Information Theory
Photic Stimulation
Automatic Data Processing
Haplorhini

Keywords

  • Cross-modal
  • Multisensory integration
  • Temporal lobe
  • Visual scene

ASJC Scopus subject areas

  • Sensory Systems
  • Cognitive Neuroscience
  • Cellular and Molecular Neuroscience

Cite this

Modulation of visual responses in the superior temporal sulcus by audio-visual congruency. / Dahl, Christoph D.; Logothetis, Nikos K.; Kayser, Christoph.

In: Frontiers in Integrative Neuroscience, No. APRIL 2010, 01.04.2010.

Research output: Contribution to journalArticle

@article{acab3418debc45008afab5c8f873fb2c,
title = "Modulation of visual responses in the superior temporal sulcus by audio-visual congruency",
abstract = "Our ability to identify or recognize visual objects is often enhanced by evidence provided by other sensory modalities. Yet, where and how visual object processing benefit from the information received by the other senses remains unclear. One candidate region is the temporal lobe, which features neural representations of visual objects, and in which previous studies have provided evidence for multisensory influences on neural responses. In the present study we directly tested whether visual representations in the lower bank of the superior temporal sulcus (STS) benefi t from acoustic information. To this end, we recorded neural responses in alert monkeys passively watching audio-visual scenes, and quantified the impact of simultaneously presented sounds on responses elicited by the presentation of naturalistic visual scenes. Using methods of stimulus decoding and information theory, we then asked whether the responses of STS neurons become more reliable and informative in multisensory contexts. Our results demonstrate that STS neurons are indeed sensitive to the modality composition of the sensory stimulus. Importantly, information provided by STS neurons' responses about the particular visual stimulus being presented was highest during congruent audio-visual and unimodal visual stimulation, but was reduced during incongruent bimodal stimulation. Together, these findings demonstrate that higher visual representations in the STS not only convey information about the visual input but also depend on the acoustic context of a visual scene.",
keywords = "Cross-modal, Multisensory integration, Temporal lobe, Visual scene",
author = "Dahl, {Christoph D.} and Logothetis, {Nikos K.} and Christoph Kayser",
year = "2010",
month = "4",
day = "1",
doi = "10.3389/fnint.2010.00010",
language = "English",
journal = "Frontiers in Integrative Neuroscience",
issn = "1662-5145",
publisher = "Frontiers Research Foundation",
number = "APRIL 2010",

}

TY - JOUR

T1 - Modulation of visual responses in the superior temporal sulcus by audio-visual congruency

AU - Dahl, Christoph D.

AU - Logothetis, Nikos K.

AU - Kayser, Christoph

PY - 2010/4/1

Y1 - 2010/4/1

N2 - Our ability to identify or recognize visual objects is often enhanced by evidence provided by other sensory modalities. Yet, where and how visual object processing benefit from the information received by the other senses remains unclear. One candidate region is the temporal lobe, which features neural representations of visual objects, and in which previous studies have provided evidence for multisensory influences on neural responses. In the present study we directly tested whether visual representations in the lower bank of the superior temporal sulcus (STS) benefi t from acoustic information. To this end, we recorded neural responses in alert monkeys passively watching audio-visual scenes, and quantified the impact of simultaneously presented sounds on responses elicited by the presentation of naturalistic visual scenes. Using methods of stimulus decoding and information theory, we then asked whether the responses of STS neurons become more reliable and informative in multisensory contexts. Our results demonstrate that STS neurons are indeed sensitive to the modality composition of the sensory stimulus. Importantly, information provided by STS neurons' responses about the particular visual stimulus being presented was highest during congruent audio-visual and unimodal visual stimulation, but was reduced during incongruent bimodal stimulation. Together, these findings demonstrate that higher visual representations in the STS not only convey information about the visual input but also depend on the acoustic context of a visual scene.

AB - Our ability to identify or recognize visual objects is often enhanced by evidence provided by other sensory modalities. Yet, where and how visual object processing benefit from the information received by the other senses remains unclear. One candidate region is the temporal lobe, which features neural representations of visual objects, and in which previous studies have provided evidence for multisensory influences on neural responses. In the present study we directly tested whether visual representations in the lower bank of the superior temporal sulcus (STS) benefi t from acoustic information. To this end, we recorded neural responses in alert monkeys passively watching audio-visual scenes, and quantified the impact of simultaneously presented sounds on responses elicited by the presentation of naturalistic visual scenes. Using methods of stimulus decoding and information theory, we then asked whether the responses of STS neurons become more reliable and informative in multisensory contexts. Our results demonstrate that STS neurons are indeed sensitive to the modality composition of the sensory stimulus. Importantly, information provided by STS neurons' responses about the particular visual stimulus being presented was highest during congruent audio-visual and unimodal visual stimulation, but was reduced during incongruent bimodal stimulation. Together, these findings demonstrate that higher visual representations in the STS not only convey information about the visual input but also depend on the acoustic context of a visual scene.

KW - Cross-modal

KW - Multisensory integration

KW - Temporal lobe

KW - Visual scene

UR - http://www.scopus.com/inward/record.url?scp=77957345465&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77957345465&partnerID=8YFLogxK

U2 - 10.3389/fnint.2010.00010

DO - 10.3389/fnint.2010.00010

M3 - Article

AN - SCOPUS:77957345465

JO - Frontiers in Integrative Neuroscience

JF - Frontiers in Integrative Neuroscience

SN - 1662-5145

IS - APRIL 2010

ER -