Overt sentence production in event-related fMRI

doi:10.1016/j.neuropsychologia.2004.09.007

Neuropsychologia

Volume 43, Issue 5, 2005, Pages 807-814

https://doi.org/10.1016/j.neuropsychologia.2004.09.007 Get rights and content

Abstract

The use of syntactic structures on a sentence level is a unique human ability. Functional imaging studies have usually investigated syntax comprehension. However, language production may be performed by different neuronal resources. We have investigated syntax generation on a sentence level with functional magnetic resonance imaging (fMRI).

BOLD contrast was measured while subjects articulated utterances aloud. In the active condition ‘sentence generation’ (SG), subjects had to produce subject verb object (SVO) sentences (e.g. “The child throws the ball”) according to syntactically incomplete stimuli (e.g. “throw ball child”) presented visually. In the control condition ‘word reading’ (WR), subjects had to read identical stimuli without completing the syntactic structure, while in a second control condition ‘sentence reading’ (SR), subjects had to read complete sentences. The semantic meaning of all expressions was obvious despite the syntactically incomplete structure in conditions SG and WR.

In both contrasts, SG minus WR and SG minus SR, activation was mainly present in the left inferior frontal (BA 44/45) and medial frontal (BA 6) gyri, the superior parietal lobule (BA 7) and the right insula (BA 13). A region of interest analysis revealed significantly stronger left-dominant activation in BA 45 compared to BA 44.

Our data illustrates the crucial involvement of the left BA 45 in syntactic encoding and is in line with more recent imaging and brain lesion data on syntax processing on a sentence level, emphasizing the involvement of a distributed left and right hemispheric network in syntax generation.

Introduction

Fundamental aspects of language include syntax, i.e. grammatically well formed words and sentences, and semantics, i.e. the meaning of a word or phrase. However, syntax and semantics are inseparable in natural language. Compare as an example the headlines “Dog bites man” and “Man bites dog”. Regarding the neural processes underlying human language, a further distinction has to be made between production and comprehension. During speech production, a mental concept is expressed (Kircher et al., 2004). Words with appropriate semantic meaning are retrieved from a store and an expression is created according to syntactic rules (Levelt, 1989, Levelt, 1995). During speech perception, the inverse process is performed. The final result of the comprehension process is the idea or the concept of a given expression. There is evidence from brain lesion studies involving patients with aphasia that distinct neural networks compute these two processes (Ferstl, Guthke, & von Cramon, 1999; Munhall, 2001). In particular, it has been shown that aphasic patients are unable to produce syntactically correct sentences, while being able to judge the syntactical correctness of presented sentences (Linebarger, Schwartz, & Saffran, 1983).

Most previous neuroimaging studies on language processing on a sentence level focused on the investigation of comprehension (Cabeza & Nyberg, 2000; Indefrey & Levelt, 2000; Kaan & Swaab, 2002). Many of these studies used sentences with spelling errors or wrong word order (Sakai, Hashimoto, & Homae, 2001) to test for different aspects of syntactic processing. Subjects usually performed plausibility judgments, i.e. they had to decide whether a sentence was correct or not (Embick, Marantz, Miyashita, O’Neil, & Sakai, 2000). In other studies, subjects had to decide as to whether or not two sentences were similar (Dapretto & Bookheimer, 1999). Language production should ideally be investigated by analyzing natural speech (Kircher, Brammer, Williams, & McGuire, 2000). However, it is difficult to control for the underlying cognitive processes involved in the generation of longer utterances. Therefore, the majority of previous functional neuroimaging studies on language production reduced verbal output to single words. Particularly well investigated are verbal fluency tasks, where subjects are instructed to produce words starting with a particular letter or of a semantic category (Cabeza & Nyberg, 2000). However, higher language properties such as syntax or the meaning of a phrase cannot be covered by these paradigms. Consequently, little is known about the cerebral substrates of sentence-level production processes (Indefrey & Levelt, 2000). In the only systematic imaging study (using positron emission tomography, PET) on syntax during language production on a sentence level (Indefrey, Brown, et al., 2001), subjects had to generate utterances of different syntactic complexity that described the motion of visually presented objects. The authors provided evidence that a region caudally adjacent to Broca's area is involved in both sentence-level and local (phrase-level) syntactic encoding during speaking.

In the present study, we investigated syntax production on a sentence level. Subjects had to produce simple subject verb object (SVO) sentences like e.g. “The child throws the ball”. We systematically varied the demand on syntax production in three conditions, while subjects spoke overtly. We hypothesed increasing signal changes in the left inferior frontal gyrus (IFG) with increased syntactic production demand.

Section snippets

Subjects

Fifteen right-handed (score ranging from 14 to 20 in the online version of the Edinburgh Handedness Inventory (Oldfield, 1971), http://airto.loni.ucla.edu [scores range from −20 (left handed) to +20 (right handed)]) male subjects (aged 19–44, mean 26.2, S.D. 6.5 years) gave their written informed consent prior to inclusion in the study. Subjects had no history of medical, neurological or psychiatric disorders. All subjects were native German speakers. The study has been approved by the local

Behavioral data

The correct response was given in 99.2, 100 and 95.8% of trials for conditions WR, SR and SG, respectively. The most common incorrect response was neglect of the second article (e.g. ‘The woman cooks noodles’ instead of ‘The woman cooks the noodles’), which occurred in 3.3% of trials in condition SG. There was a significant difference in the group comparison (p < 0.05 one-way ANOVA). Pair-wise analysis revealed significant difference between SR and SG (p < 0.05 Bonferroni's Multiple Comparison

Discussion

In the present study, subjects had to produce simple SVO sentences. The demand on syntactic production load was varied between three different conditions. In line with our predictions, activation was mainly observed in left inferior frontal gyrus/Broca's area for the higher syntactic demand condition.

In particular, a region of interest analysis revealed, that within the IFG, the maximum BOLD signal change of the active condition was significantly stronger in BA 45 compared to BA 44. Further,

Conclusions

In conclusion, we could demonstrate that language production on a sentence level can be investigated in fMRI using overt speech. In particular, activation associated with syntactic encoding during overt sentence production was predominant in the left inferior frontal gyrus and other left hemispheric areas. We emphasize the need to further investigate speech production directly, despite difficulties of controlling conceptual processing underlying the generation of sentences.

Acknowledgement

We thank Peter Indefrey for helpful comments on earlier versions of the manuscript.

References (34)

D.M. Barch et al.
Overt verbal responding during fMRI scanning: Empirical investigations of problems and potential solutions
NeuroImage
(1999)
R.M. Birn et al.
Magnetic field changes in the human brain due to swallowing or speaking
Magnetic Resonance in Medicine
(1998)
S. Bookheimer
Functional MRI of language: New approaches to understanding the cortical organization of semantic processing
Annual Review of Neuroscience
(2002)
R. Cabeza et al.
Neural bases of learning and memory: Functional neuroimaging evidence
Current Opinion in Neurology
(2000)
J.M. Chein et al.
Functional heterogeneity within Broca's area during verbal working memory
Physiology and Behaviour
(2002)
M. Dapretto et al.
Form and content: Dissociating syntax and semantics in sentence comprehension
Neuron
(1999)
F. Dick et al.
Language deficits, localization, and grammar: Evidence for a distributive model of language breakdown in aphasic patients and neurologically intact individuals
Psychological Review
(2001)
D. Embick et al.
A syntactic specialization for Broca's area
Proceedings of the National Academy of Sciences of the United States of America
(2000)
E.C. Ferstl et al.
Change of perspective in discourse comprehension: Encoding and retrieval processes after brain injury
Brain Language
(1999)
M.A. Gernsbacher et al.
Neuroimaging studies of language production and comprehension
Annual Review of Psychology
(2003)

Y. Grodzinsky

The neurology of syntax: Language use without Broca's area

The Behavioural and Brain Sciences

(2000)

J. Huang et al.

Comparing cortical activations for silent and overt speech using event-related fMRI

Human Brain Mapping

(2002)

P. Indefrey et al.

Syntactic processing in left prefrontal cortex is independent of lexical meaning

NeuroImage

(2001)

P. Indefrey et al.

A neural correlate of syntactic encoding during speech production

Proceedings of the National Academy of Sciences of the United States of America

(2001)

E. Kaan et al.

The brain circuitry of syntactic comprehension

Trends Cognitive Sciences

(2002)

A.M. Kang et al.

An event-related fMRI study of implicit phrase-level syntactic and semantic processing

NeuroImage

(1999)

Cited by (88)

Neural bases of elements of syntax during speech production in patients with aphasia
2021, Brain and Language
Citation Excerpt :
Selecting a word in connected speech requires a balance of target word activation and inhibition of alternative forms (previous, future, competitors), all of which are coordinated across multiple hierarchical levels (Garrett, 1975; Lashley, 1951). Therefore, selecting a word during discourse likely depends on cognitive operations that extend beyond lexical processing (Fergadiotis, Wright, & West, 2013), and as such, relies on areas beyond inferior frontal gyrus, extending to the middle and superior frontal cortex, middle and superior temporal gyri encompassing even their more posterior aspects, as well as different portions of the parietal lobe (den Ouden, Hoogduin, Stowe, & Bastiaanse, 2008; Haller, Radue, Erb, Grodd, & Kircher, 2005; Kircher, Oh, Brammer, & McGuire, 2005; Menenti, Segaert, & Hagoort, 2012; Mirman, Kraft, Harvey, Brecher, & Schwartz, 2019; Schonberger et al., 2014; Timmers et al., 2015; Vigneau et al., 2006). While most of our understanding of the neural underpinnings of syntax stems from functional approaches, as detailed above, lesion-symptom mapping of sentence production has been limited, with most studies focusing rather on syntax processing, usually with the use of non-natural tasks.
The ability to string together words into a structured arrangement capable of conveying nuanced information is key to speech production. The assessment of the neural bases for structuring sentences has been challenged by the need of experts to delineate the aberrant morphosyntactic structures in aphasic speech. Most studies have relied on focused tasks with limited ecological validity. We characterized syntactic complexity during connected speech produced by patients with chronic post-stroke aphasia. We automated this process by employing Natural Language Processing (NLP). We conducted voxel-based and connectome-based lesion-symptom mapping to identify brain regions crucially associated with sentence production and syntactic complexity. Posterior-inferior aspects of left frontal and parietal lobes, as well as white matter tracts connecting these areas, were essential for syntactic complexity, particularly the posterior inferior frontal gyrus. These findings suggest that sentence structuring during word production depends on the integrity of Broca’s area and the dorsal stream of language processing.
Messages must be tuned to the target language: Some implications of crosslinguistic semantic diversity for neurolinguistic research on speech production
2019, Journal of Neurolinguistics
There are nearly 6,500 languages in the world, and they vary greatly with regard to both lexical and grammatical semantics. Hence, an early stage of utterance planning involves "thinking for speaking"—i.e., shaping the thoughts to be expressed so they fit the idiosyncratic meanings of the symbolic units that happen to be available in the target language. This paper has three main sections that cover three distinct types of crosslinguistic semantic diversity. Each type is initially elaborated with examples, and then its implications for the neurobiology of speech production are considered. Type 1: Semantic field partitions. These are exemplified by huge crosslinguistic differences in many domains of meaning, including colors, body parts, household containers, events of cutting and breaking, and topological spatial relations. When such differences are viewed from the perspective of contemporary neurocognitive theories which assume that most concrete concepts are subserved by both modal (i.e., sensory, motor, and affective) and transmodal (i.e., integrative) cortical systems, they imply that speakers must access language-specific semantic structures at multiple levels of representation in the brain. Type 2: Semantic conflation classes. Some languages have whole sets of words that systematically encode two or more components of meaning. For instance, in the roughly 53 Athabaskan languages there are no generic verbs like give, carry, or throw; instead, there are entire sets of 9–13 verbs for these kinds of actions, with each set making the same distinctions between the types of objects that are given, carried, or thrown, such as animate objects, round objects, stick-like objects, flat objects, etc. This regular conflation of [action + object] in Athabaskan verb meanings predicts that speakers frequently co-activate both action-related and object-related cortical regions in a functionally integrated fashion. Type 3: Grammatically obligatory semantic categories. This kind of crosslinguistic variation involves not only the particular dimensions of experience that speakers are forced to track for grammatical purposes, but also the precise contrasts that they must make along those dimensions, with examples including systems of nominal classification, tense, and evidentiality. It is still not known exactly where or how the meanings of grammatically necessary closed-class morphemes are implemented in the brain, but it is quite clear that whatever the neural substrates of these meanings turn out to be, they are strongly influenced by crosslinguistic differences. In all, by focusing on three separate forms of crosslinguistic semantic diversity, this paper reinforces Levelt's (1989) point that "messages must be tuned to the target language," and it also shows that this point continues to have significant consequences for neurolinguistic research on speech production.
Roles of ventral versus dorsal pathways in language production: An awake language mapping study
2019, Brain and Language
Human language is organized along two main processing streams connecting posterior temporal cortex and inferior frontal cortex in the left hemisphere, travelling dorsal and ventral to the Sylvian fissure. Some views propose a dorsal motor versus ventral semantic division. Others propose division by combinatorial mechanism, with the dorsal stream responsible for combining elements into a sequence and the ventral stream for forming semantic dependencies independent of sequential order. We acquired data from direct cortical stimulation in the left hemisphere in 17 neurosurgical patients and subcortical resection in a subset of 10 patients as part of awake language mapping. Two language tasks were employed: a sentence generation (SG) task tested the ability to form sequential and semantic dependencies, and a picture-word interference (PWI) task manipulated semantic interference. Results show increased error rates in the SG versus PWI task during subcortical testing in the dorsal stream territory, and high error rates in both tasks in the ventral stream territory. Connectivity maps derived from diffusion imaging and seeded in the tumor sites show that patients with more errors in the SG than in the PWI task had tumor locations associated with a dorsal stream connectivity pattern. Patients with the opposite pattern of results had tumor locations associated with a more ventral stream connectivity pattern. These findings provide initial evidence using fiber tract disruption with electrical stimulation that the dorsal pathways are critical for organizing words in a sequence necessary for sentence generation, and the ventral pathways are critical for processing semantic dependencies.
The priming of basic combinatory responses in MEG
2018, Cognition
Priming has been a powerful tool for the study of human memory and especially the memory representations relevant for language. However, although it is well established that lexical access can be primed, we do not know exactly what types of computations can be primed above the word level. This work took a neurobiological approach and assessed the ways in which the complex representation of a minimal combinatory phrase, such as red boat, can be primed, as evidenced by the spatiotemporal profiles of magnetoencephalography (MEG) signals. Specifically, we built upon recent progress on the neural signatures of phrasal composition and tested whether the brain activities implicated for the basic combination of two words could be primed. In two experiments, MEG was recorded during a picture naming task where the prime trials were designed to replicate previously reported combinatory effects and the target trials to test whether those combinatory effects could be primed. The manipulation of the primes was successful in eliciting larger activity for adjective-noun combinations than single nouns in left anterior temporal and ventromedial prefrontal cortices, replicating prior MEG studies on parallel contrasts. Priming of similarly timed activity was observed during target trials in anterior temporal cortex, but only when the prime and target shared an adjective. No priming in temporal cortex was observed for single word repetition and two control tasks showed that the priming effect was not elicited if the prime pictures were simply viewed but not named. In sum, this work provides evidence that very basic combinatory operations can be primed, with the necessity for some lexical overlap between prime and target suggesting combinatory conceptual, as opposed to syntactic processing. Both our combinatory and priming effects were early, onsetting between 100 and 150 ms after picture onset and thus are likely to reflect the very earliest planning stages of a combinatory message. Thus our findings suggest that at the earliest stages of combinatory planning in production, a combinatory memory representation is formed that affects the planning of a relevantly similar combination on a subsequent trial.
Improving language mapping in clinical fMRI through assessment of grammar
2017, NeuroImage: Clinical
Citation Excerpt :
Among all the language ROI, the CYCLE-N and the standard tests produced the highest activation in the left hemisphere BA 44. The region has been identified as the primary processor of syntax in the brain (Dapretto and Bookheimer, 1999; Friederici, 2011; Haller et al., 2005; Skeide et al., 2014; Tyler et al., 2013). BA44 participates in building of syntactic structures (Friederici, 2011).
Brain surgery in the language dominant hemisphere remains challenging due to unintended post-surgical language deficits, despite using pre-surgical functional magnetic resonance (fMRI) and intraoperative cortical stimulation. Moreover, patients are often recommended not to undergo surgery if the accompanying risk to language appears to be too high. While standard fMRI language mapping protocols may have relatively good predictive value at the group level, they remain sub-optimal on an individual level. The standard tests used typically assess lexico-semantic aspects of language, and they do not accurately reflect the complexity of language either in comprehension or production at the sentence level. Among patients who had left hemisphere language dominance we assessed which tests are best at activating language areas in the brain.
We compared grammar tests (items testing word order in actives and passives, wh-subject and object questions, relativized subject and object clauses and past tense marking) with standard tests (object naming, auditory and visual responsive naming), using pre-operative fMRI. Twenty-five surgical candidates (13 females) participated in this study. Sixteen patients presented with a brain tumor, and nine with epilepsy. All participants underwent two pre-operative fMRI protocols: one including CYCLE-N grammar tests (items testing word order in actives and passives, wh-subject and object questions, relativized subject and object clauses and past tense marking); and a second one with standard fMRI tests (object naming, auditory and visual responsive naming). fMRI activations during performance in both protocols were compared at the group level, as well as in individual candidates.
The grammar tests generated more volume of activation in the left hemisphere (left/right angular gyrus, right anterior/posterior superior temporal gyrus) and identified additional language regions not shown by the standard tests (e.g., left anterior/posterior supramarginal gyrus). The standard tests produced more activation in left BA 47. Ten participants had more robust activations in the left hemisphere in the grammar tests and two in the standard tests. The grammar tests also elicited substantial activations in the right hemisphere and thus turned out to be superior at identifying both right and left hemisphere contribution to language processing.
The grammar tests may be an important addition to the standard pre-operative fMRI testing.
Composition of complex numbers: Delineating the computational role of the left anterior temporal lobe
2016, NeuroImage
What is the neurobiological basis of our ability to create complex messages with language? Results from multiple methodologies have converged on a set of brain regions as relevant for this general process, but the computational details of these areas remain to be characterized. The left anterior temporal lobe (LATL) has been a consistent node within this network, with results suggesting that although it rather systematically shows increased activation for semantically complex structured stimuli, this effect does not extend to number phrases such as ‘three books.’ In the present work we used magnetoencephalography to investigate whether numbers in general are an invalid input to the combinatory operations housed in the LATL or whether the lack of LATL engagement for stimuli such as ‘three books’ is due to the quantificational nature of such phrases. As a relevant test case, we employed complex number terms such as ‘twenty-three’, where one number term is not a quantifier of the other but rather, the two terms form a type of complex concept. In a number naming paradigm, participants viewed rows of numbers and depending on task instruction, named them as complex number terms (‘twenty-three’), numerical quantifications (‘two threes’), adjectival modifications (‘blue threes’) or non-combinatory lists (e.g., ‘two, three’). While quantificational phrases failed to engage the LATL as compared to non-combinatory controls, both complex number terms and adjectival modifications elicited a reliable activity increase in the LATL. Our results show that while the LATL does not participate in the enumeration of tokens within a set, exemplified by the quantificational phrases, it does support conceptual combination, including the composition of complex number concepts.

View all citing articles on Scopus

View full text

Overt sentence production in event-related fMRI

Abstract

Introduction

Section snippets

Subjects

Behavioral data

Discussion

Conclusions

Acknowledgement

Overt verbal responding during fMRI scanning: Empirical investigations of problems and potential solutions

NeuroImage

Magnetic field changes in the human brain due to swallowing or speaking

Magnetic Resonance in Medicine

Functional MRI of language: New approaches to understanding the cortical organization of semantic processing

Annual Review of Neuroscience

Neural bases of learning and memory: Functional neuroimaging evidence

Current Opinion in Neurology

Functional heterogeneity within Broca's area during verbal working memory

Physiology and Behaviour

Form and content: Dissociating syntax and semantics in sentence comprehension

Neuron

Language deficits, localization, and grammar: Evidence for a distributive model of language breakdown in aphasic patients and neurologically intact individuals

Psychological Review

A syntactic specialization for Broca's area

Proceedings of the National Academy of Sciences of the United States of America

Change of perspective in discourse comprehension: Encoding and retrieval processes after brain injury

Brain Language

Neuroimaging studies of language production and comprehension

Annual Review of Psychology

The neurology of syntax: Language use without Broca's area

The Behavioural and Brain Sciences

Comparing cortical activations for silent and overt speech using event-related fMRI

Human Brain Mapping

Syntactic processing in left prefrontal cortex is independent of lexical meaning

NeuroImage

A neural correlate of syntactic encoding during speech production

Proceedings of the National Academy of Sciences of the United States of America

The brain circuitry of syntactic comprehension

Trends Cognitive Sciences

An event-related fMRI study of implicit phrase-level syntactic and semantic processing

NeuroImage