We all predict the future every time we listen to someone speak or read a book. If I say “barbed,” for example, what word comes next? How about “undivided”? (see answers below, along with other top word pairings) The ability to predict words helps us take mental shortcuts in language. And a new study finds that the frequency with which these words have occurred together in our past conversations is key to our predictive skills.
“We often have a pretty good guess about what word the person we’re talking to is going to say, typically because people we’ve talked to in the past have tended to use this word in this particular context,” says Tal Linzen of New York University. “Predicting what the other person is about to say gives us more time to plan our next move in the conversation and helps us to understand the person we’re talking to even if we can’t hear them very well.”
Fascinated by the process of prediction in language comprehension, Linzen wanted to explore new ways to study it. He teamed up with Joseph Fruchter and colleagues to design a study using magnetoencephalography (MEG). Fruchter was specifically interested in using various statistical measures, such as word frequency, to investigate neural activity during language processing.
Their results, just published in the Journal of Cognitive Neuroscience, presents new evidence for “lexical preactivation” – the process by which we anticipate words – with adjective and noun combinations, showing the role of word frequency in linguistic prediction.
Linzen and Fruchter talked with CNS about the study, its unique design and the implications of their findings for understanding language.
CNS: Would you please explain what lexical preactivation is and what we know about how it?
Linzen and Fruchter: Lexical preactivation is the process by which we activate words that we believe will come up next. If the word is already “active,” we can understand it more easily when we actually hear or read it. For example, when people read the word stainless, they have enough information to anticipate that the next word is going to be steel. If they then reach the word steel, all they need to do is verify their prediction, instead of having to read the word from scratch, so to speak.
CNS: Why is it important to study?
Linzen and Fruchter: By studying lexical preactivation we can gain insight into the nature of predictive language processing. Most prior research has investigated the brain response to words that have already been presented. How similar is predictive processing of upcoming words? Are the same cortical networks for language processing simply recruited at an earlier time point? Can we find effects of the same variables known to typically affect the brain response, such as word frequency?
Prediction isn’t specific to language. In the last decade more and more neuroscientists have been arguing that prediction is one of the fundamental organizing principles of the brain. Instead of passively absorbing information from the environment, we constantly predict what we are about to see, hear or feel, and then match our perceptual input against those predictions. Understanding prediction in language, then, could help us understand how the brain works more generally.
CNS: What was the goal of this study?
Linzen and Fruchter: The main goal of this study was to use MEG to provide direct evidence of lexical preactivation. How is the brain response to predictive adjectives, such as stainless, different from the brain response to less predictive adjectives, such as brilliant, which can be followed by a wide range of different nouns?
Can we find evidence that people preactivate specific upcoming words, such as steel, when presented with predictive adjectives such as stainless?
CNS: Can you please highlight any novel elements to your study design?
Linzen and Fruchter: Our participants read two-word phrases, such as stainless steel or brilliant career, while we recorded their neural activity using MEG. To isolate the response to the first word (the adjective), we showed the words one after the other: first stainless, then steel. Our design differed from most previous work on prediction in two ways. First, the focus on the first word is new; most previous studies have looked at the neural response to the word that matches or doesn’t match a prediction, not at the response to the word where the prediction is first generated.
A second difference is that most studies of prediction have used full sentences, presumably because it’s easier to find sentences that lead to a very strong prediction. If a sentence starts with “he loosened the tie around his…”, for example, most of us would have a strong intuition that the last word will be “neck.” The downside of that approach is that there’s no single point where the prediction is generated: The prediction builds up gradually over the course of the sentence. Short phrases like “stainless steel,” on the other hand, are ideal to study the process that we’re interested in: the only point at which a prediction could possibly be generated is at the adjective stainless.
CNS: Why did you explore this topic via adjective-noun relationship? And how did you chose the phrases to use?
Linzen and Fruchter: We wanted all of our phrases to have the same syntactic structure to make sure that we weren’t accidentally picking up syntactic effects that were unrelated to our main research question. In principle, any type of two-word phrase would have worked, including verb-noun phrases like “drink water” or verb-adverb phrases like “dress well.” We used adjective-noun phrases mainly because we were able to find more phrases with this structure where a strong prediction can be made based on the first word. We extracted all adjective-noun phrases from the Corpus of Contemporary American English, provided that they met a number of selection criteria (described more fully in the paper).
CNS: What were you most excited to find?
Linzen and Fruchter: We were excited to find that the well-established effects of word frequency could be used to provide evidence for processing of words that hadn’t been presented yet. This finding is consistent with the predictions of several classical models of word recognition, namely that word-frequency effects should be fundamentally tied to accessing the internal linguistic representation of particular words, regardless of the input modality. Additionally, the linguistic prediction literature contains a longstanding debate about whether the reduction in the neural signal for predictable words is due to preactivation of those words, or whether it is just due to the fact that they fit better into the context than more surprising words, which makes it easier to integrate them into the meaning of the sentence. While several previous studies have provided evidence consistent with the preactivation account, it has been challenging to demonstrate evidence of individual words being preactivated, so we were excited to see that word frequency could be used for that purpose.
CNS: What is the significance of your findings for the general population?
Linzen and Fruchter: We believe that our findings highlight the potential of neuroscience techniques to help answer difficult questions in cognitive science and linguistics. While it is not particularly surprising that people can anticipate predictable words, it has been harder to study the process by which this takes place. MEG allowed us both to confirm the existence of lexical preactivation during reading of adjective-noun phrases, as well as to provide details on the cognitive and neural mechanisms that subserve this process.
CNS: What’s next for this work? What do you ultimately hope to accomplish?
Linzen and Fruchter: There are a lot of questions that remain open about prediction in language comprehension. Do we only predict a single word, or can we predict many different words at once? Do we predict larger linguistic structures in addition to individual words? How does lexical preactivation work in real conversations? For example, what happens if we believe that a word will come up at a later point in the sentence but we don’t know when exactly?
More generally, our goal is to unify the cognitive theories that linguists and psychologists have been developing for decades with what neuroscientists are starting to discover about the brain. We see this as a two-way street: You can only design smart neuroscience experiments on language if you have a solid understanding of linguistics and cognitive science. And in the other direction, if the results of our neuroscientific research are not compatible with our cognitive theories, then we need to revise our cognitive theories.
-Lisa M.P. Munoz
The paper, “Lexical Preactivation in Basic Linguistic Phrases” by Joseph Fruchter, Tal Linzen, Masha Westerlund, and Alec Marantz,” was published in the Journal of Cognitive Neuroscience online on May 11, 2015.
Some top word pairings from Fruchter et al.:
unsalted butter
stainless steel
barbed wire
umbilical cord
iced tea
soapy water
renewable energy
undivided attention
untimely death
concerted effort
