Study Reveals Facial Expressions as Indicators of Hidden Cognitive States
Can what’s happening on a face show what’s going on inside the mind? A new study in Nature Communications explored this exciting idea. Scientists used advanced computer tools to closely watch the facial expressions of mice and monkeys. They found that face movements reliably show internal cognitive processes, suggesting these are surprisingly similar across these animal species. This study helps us understand if the inner states that guide animal behavior are alike across different kinds of animals. It also highlights why we need new ways to compare these natural states without complicated tasks.
Note: This article is intended for general information and educational purposes. It summarizes scientific research in accessible language for a broad audience and is not an official scientific press release.
A new study, published on June 4, 2025, in the journal Nature Communications, investigates whether internal cognitive states are comparable across different species. The research team, including Alejandro Tlaie, Muad Y. Abd El Hay, and Marieke L. Schölvinck, among others, developed a method to study and directly compare spontaneously occurring internal states in macaques and mice. Scientists from the Ernst Strüngmann Institute for Neuroscience, Universidad Politécnica de Madrid, and the Princeton Neuroscience Institute contributed to this work.
The study used virtual reality and analyzed facial expressions from video recordings. The goal was to identify internal states that could predict how the animals would react and perform a task. The authors reported that the connection between the identified states and how well the animals performed the tasks was similar in both mice and monkeys. Additionally, each state corresponded to specific facial features that partly overlapped between the two species.
What the Researchers Investigated
The research team explored how similar internal cognitive states, like “attention,” are between macaques and mice. The authors observe that animal behaviors, such as hunting or foraging, are not simply passive reactions to their surroundings. Instead, these behaviors are influenced by changing internal states, like alertness or vigilance.
Scientists noted that traditional methods for studying internal states often use complex tasks with simple stimuli, which makes cross-species comparisons difficult due to differences in the tasks themselves. For example, attention tasks for monkeys are very different from those for rodents, hindering direct comparisons of underlying behavioral patterns across species.
To address this challenge, the authors proposed a behavioral approach that uses natural behaviors to reflect spontaneously occurring internal states. Their method aimed to identify these states based on data, without imposing pre-defined “concepts” of what attention or other states are, and to track how they evolve over time. This required very precise and detailed tracking of behavior. The study was conducted with male mice and macaques.
How the Study Was Conducted
The study combined a virtual reality (VR) foraging task with a deep learning tool for automated tracking of behavioral features. Two monkeys and seven mice participated across multiple experimental sessions (18 for monkeys, 29 for mice), totaling over 33,000 trials. The animals navigated a VR environment projected inside a custom-made dome. Monkeys used a trackball to move in VR, while mice ran on a spherical treadmill, with their movements translated into VR.
The animals performed a two-choice perceptual decision task: they had to approach a target stimulus (a spike-shaped leaf) and avoid a distracting one (a round-shaped leaf) in a virtual meadow. Task performance was measured by trial outcomes (correct, incorrect, missed) and reaction time. The authors noted that success rates and reaction times were largely comparable across species, though mice showed less consistent performance.
While the animals performed the task, their faces were recorded. For macaques, video footage from a frontal camera and eye-tracking data were analyzed. For mice, video footage from a side camera was used. Facial features, such as eyebrow, nose, and ear movements (18 for monkeys, 9 for mice), were extracted using the DeepLabCut software. These data were averaged over a 250-millisecond window before stimuli appeared, to capture internally generated processes rather than immediate reactions to stimuli.
The extracted facial features were fed into a special computer model called Markov-Switching Linear Regression (MSLR). This model predicted the animals’ reaction time based on their facial expressions, by assuming there were “hidden” internal states. Each state suggested a different relationship between facial features and reaction time. The model identified 4 such states for monkeys and 3 for mice. The authors found that using multiple hidden states significantly improved the model’s predictions compared to using only one state. The model performed well across all animals and could reliably predict the behavior of new animals.
What Makes This Study New
The authors highlight that this study utilized advanced technologies, specifically virtual reality and deep-learning algorithms, to identify and directly compare spontaneously occurring internal states in macaques and mice. They emphasize that their approach, which uses facial expressions to infer states from natural behavior, represents a “drastic departure” from older methods that forced animals into restrictive tasks to study internal states.
This study introduces a method that, by focusing on a detailed analysis of entire facial expressions, aimed to map out the range of naturally occurring internal states based on data, in a way that is directly comparable across species for the first time. The authors highlighted the reliability of their findings, as similar results were obtained when using another type of model (GLM-HMM).
Compared to previous research, which mostly looked at simple connections between single facial features (like pupil size) and pre-defined cognitive states, this study considered the whole face. The authors state that this helped to reduce confusion compared to earlier work and “adds one of the first rigorous estimates of which aspects of internal states truly generalize across species—and which do not.” They also note that finding strong similarities in spontaneously occurring internal states across species opens the door for more research into the common and evolutionarily preserved principles of these states.
Key Findings from the Study
- Evolutionary Conservation of Facial Cues: Each internal state was connected to a unique pattern of facial features, which were distinct and clearly identifiable. Importantly, the specific facial features that predicted similar behaviors in mice and monkeys overlapped significantly, suggesting that the way internal states are expressed through facial movements is robust and has been well-preserved throughout evolution.
- Facial Expressions Predict Behavior: The animals’ facial expressions before a trial accurately predicted their reaction times and even the outcome of the task (correct, incorrect, or missed trial). This showed that face movements were reliable indicators of underlying internal states, and the model didn’t rely only on simple arousal signs like pupil size.
- State Stability Varies by Species: Macaques showed very stable internal states that rarely changed, suggesting steady internal processing. In contrast, mice switched between states more frequently. This might be due to real differences in how they process information or potentially influenced by the macaques’ more extensive training.
- States Reflect Performance Styles: The identified hidden states were clearly linked to different ways animals performed the task, such as their response speed. For instance, some states consistently led to correct and fast responses, while others were associated with slower or incorrect outcomes. Both mice (with 3 states) and monkeys (with 4 states) showed comparable patterns of performance across their respective states.
- Diverse Behavioral Strategies: Both species showed states related to slow, unsuccessful performance, as well as multiple states for fast and accurate work. Some states indicated a careful, thorough approach, while others suggested speed or impulsiveness. The authors proposed these states might relate to different levels of “attention” needed for the task.
Authors’ Conclusions
- Reliable Prediction from Facial Features: The authors conclude that their study successfully identified internal states using facial features. These states reliably predicted how animals would react to stimuli and and how well they’d perform tasks. The connection between these states and task performance was similar in both mice and monkeys.
- Genuine Cognitive States: The internal states revealed by the MSLR model accurately predicted behavioral outcomes (like correct or incorrect trials), even without being directly given this information. The authors suggest this shows these hidden states are “genuine, dynamically changing cognitive states” that lead to different behavioral results.
- Species Differences in State Dynamics: While the number of states (3 for mice, 4 for monkeys) was similar, how often states changed over time differed: mice seemed to switch between states more frequently than monkeys. This might point to a real difference in how these species process information or could be because the monkeys had more extensive training.
- Shared Principles Across Evolution: The study found that the overall types of behavior covered by these states were largely comparable across species. Authors noted that facial expressions in both species convey cognitive and motivational information (like focus or mental effort), similar to humans, even outside social settings. They suggest these strong similarities in spontaneously occurring internal states open the door for more research into common, evolutionarily preserved principles of animal cognition.
- Future Research Directions: The authors acknowledged some limitations, such as their model providing only single estimates of internal states per trial. For the future, they propose using more advanced models to track the continuous changes of internal states within a single trial. This could offer a more precise understanding of how brain activity and behavior interact.
The original study can be found here: DOI: https://doi.org/10.1038/s41467-025-60296-1
The information in this article is provided for informational purposes only and is not medical advice. For medical advice, please consult your doctor.
