I can see it all in my mind like a mini-movie: my family and I eating breakfast at the kitchen table, pouring cereal, drinking juice and coffee, and chatting. The body positioning, senses, and actions are all vividly recreated in my mind’s eye.
We all do it to some extent every day – mentally recreate events to represent what we experience in the real world. Exactly how this ability develops and is affected as we age or experience various neurological disorders is a big area of neuroscientific investigation. In a new paper, cognitive neuroscientists lay out how they think we create these event representations, proposing that their primary role is to help us predict the future and respond appropriately when it’s time.
CNS spoke with the lead author Lauren Richmond of Washington University in St. Louis about the paper, just published in Trends in Cognitive Sciences, including what drives her work, new ways of thinking about event models, and how scientists go about studying these models in the lab.
CNS: How did you become interested in this research area?
Richmond: Before graduate school, I got a job as a research assistant in a lab that studied patients with dementia and healthy controls. My job was to visit the patients in their homes and do behavioral, cognitive and neuropsychological tests with them. In observing these people in their homes, I was struck by difference between their performance on our laboratory tasks and how they seemed to be faring in the real world. From here, I knew that I wanted to study something closer to the real-world situations that I observed.
In graduate school, I also became very interested in intervention, but for me, the interventions have to have some outcome that the participants actually care about! So, it’s important to me to test the effectiveness of any interventions that I design against performance in the real world. Studying the representation of everyday events affords me the ability to, in the lab, tap into lots of the kinds of mental computations that we think our participants are doing outside of the lab as well.
CNS: How do you like to explain “event representations”?
Richmond: “Event representations” can be thought of as the mental re-representation of the external world. It is what one pictures in their mind’s eye as they observe something unfold over time. We laid out a simple example in the paper in which we ask the reader to imagine observing two people sitting at a table together drinking coffee. If all that were represented in the brain about what is being observed were the actual perceptual and sensory inputs (body positions, muscle torques, and perhaps sounds and scents), it might be very difficult to figure out what the two people at the table are actually doing. But because people generally have some knowledge about coffee drinking, it is easy to figure out the person with a bent elbow and a hand grasping a cylinder-shaped object probably intends to start drinking their coffee, whereas the other person holding a utensil and moving their hand in a circular motion is probably adding cream or sugar to their coffee.
CNS: What have we known previously about how people construct experiences?
Richmond: We know that people do have mental representations of what is happening in the external world. There is a good deal of older work about how knowledge can support the representations of events, and providing evidence that people do build mental models of situations that they read about. Until recently, we lacked a good hypothesis about why we have these and about how they work across input modalities. It is metabolically costly to create a mental representation of something that can be observed in the external world, so why does the brain bother to do it?
We’re tapping into something that likely happens outside of the lab, and has real-world consequences for everyday memory and the ability to perform everyday activities.
CNS: What was the motivation for the paper?
Richmond: Most importantly, we wanted to provide an up-to-date account of how event models work. We also wanted to propose a hypothesis about why our system goes to the effort of creating and maintaining event models when there’s such a big up-front cost to doing so. We suggest that one reason it might be a worthwhile investment to build and maintain event models is thatthey can buy you something beyond what can be observed directly from the outside world. It allows you to predict what is likely to happen in the short-term future, can help you to infer the goals and intentions of other actors, and can even prepare you to respond appropriately in the event that you are called upon to spring into action. Going back to the coffee example, in the context of an event model, when the person stirring their coffee stops and puts the spoon down it is easy to guess that they might pick up the mug and bring it to their mouth to have a sip.
CNS: How do researchers go about simulating and measuring naturalistic everyday activity?
Richmond: There are many different approaches that have been used in the past, but one powerful way we study it in our lab is to use videos of actors performing everyday activities. These might be things like watching someone prepare a meal, unpack some boxes, or set up a camping tent. We then ask participants to watch the videos and to press a button whenever they believe that one natural and meaningful unit of activity ends and another begins. Although this task is somewhat artificial, there is good agreement across individuals about where these natural and meaningful ‘breakpoints’ between sub-parts of an activity occur. The better an individual’s button pressing lines up with the button-pressing pattern exhibited by the group, the better their memory for what they saw tends to be; we’ve called this metric ‘segmentation agreement.’
Previous work from our lab has also shown that higher segmentation agreement is related to more efficient everyday action execution. We also know from fMRI studies that the brain responds differently to breakpoints than non-breakpoints, even when people are passively watching these videos – meaning, not pressing a button when they think that natural and meaningful units are ending and beginning. So, using this task, we’re tapping into something that likely happens outside of the lab, and has real-world consequences for everyday memory and the ability to perform everyday activities.
CNS: What were your most excited to share in the paper?
Richmond: It was exciting to have the opportunity to articulate some of these ideas in print! There has been a lot of work on the fact that mental representations of observed events exist and the consequences of building and maintaining those representations for memory and action, but little work to address the question of why we have these representations in the first place. It was also fun to tie in work from different fields, including emotion perception and language, with work closer to our own to paint a broader picture for why the brain creates these types of mental representations.
CNS: What do you most want people to understand about this work?
Richmond: Really, this paper is just the beginning. Much of what we talk about in the paper is ripe for exploration. We hope that people read this paper and are inspired to do some of the hard, but important, work to either support or negate our speculations about why and how mental representations of events are constructed and maintained.
CNS: What’s next for this line of work?
Richmond: As I’ve already alluded to, I hope that this work sparks research on this topic from a variety of different angles. In our lab, we’re currently doing studies on how to improve event segmentation, on the cyclical relationship between episodic encoding and retrieval, and on the different neural and computational mechanisms that could support effective event segmentation. A big ongoing concern in the lab continues to be how these mechanisms are affected by age and early Alzheimer’s disease; we’ve also been interested in conditions including PTSD and traumatic brain injury, and in child development.
-Lisa M.P. Munoz
