Event Date:Â Tuesday, 1 December, 2015 – 16:00
Location: Via Santa Maria, 36, Pisa, PI, Italia [2nd floor seminar room]
Speaker: Andrew J. Anderson (University of Rochester)
Title: Decomposing fMRI sentences into words, then words into embodied neural features, then reassembling the pieces to predict new words within sentences
Abstract: We introduce new methods that enable model-based predictions of word and feature-level semantic content of embodied neural activity automatically induced in sentence-comprehension. How meaning is coded in the brain is a major question, and important recent advances have used semantic-models to predict neuro-semantic representations and break down elements of the brain’s code by linking model-features to neural activity components. To date analyses have largely focused on brain patterns associated with isolated nouns. However it is more natural for words to appear in sentences, when not only are there multiple words, but combinatorial meanings induced by word order and semantic enrichment arising from inferences. Nevertheless behavioral tests predict that there will be some degree of context-invariant activity associated with individual words even when they are presented in sentences. It is unknown what forms the semantic content of context-invariant word representations and how they are distributed throughout the brain. As an early step toward understanding sentence-level neural activity we target these questions by recording fMRI for 240 sentences describing everyday situations using 242 nouns, verbs and adjectives (distributed across sentences). We apply a recently developed semantic model that uses naive human ratings to estimate the strength of association between individual word-labels and different elements of neural activity in sensory, motor, social, emotional and cognitive networks. In a series of tests we show: where in the brain context-invariant word activity can be detected, where multiple words accumulate, and how different embodied-model components contribute to explaining neural activity patterns across the brain.