Cognitive flexibility is essential to navigating a changing world – new research in mice shows how your brain learns new rules

Being flexible and learning to adapt when the world changes is something you practice every day. Whether you run into a new construction site and have to reroute your commute or download a new streaming app and have to relearn how to find your favorite show, changing familiar behaviors in response to new situations is an essential skill.

To make these adaptations, your brain changes its activity patterns within a structure called the prefrontal cortex – an area of the brain critical for cognitive functions such as attention, planning and decision-making. But which specific circuits “tell” the prefrontal cortex to update its activity patterns in order to change behavior have been unknown.

We are a team ofneuroscientists who study how the brain processes information and what happens when this function is impaired. In our newly published research, we discovered a special class of neurons in the prefrontal cortex that may enable flexible behavior and, when they malfunction, may lead to conditions such as schizophrenia and bipolar disorder.

Inhibitory neurons and learning new rules

Inhibitory neurons dampen the activity of other neurons in the brain. Researchers have traditionally assumed they send their electrical and chemical outputs only to nearby neurons. However, we found a particular class of inhibitory neurons in the prefrontal cortex that communicate across long distances to neurons in the opposite hemisphere of the brain.

We wondered whether these long-range inhibitory connections are involved in coordinating changes in activity patterns across the left and right prefrontal cortex. By doing so, they might provide the critical signals that help you change your behavior at the right moment.

To test the function of these long-range inhibitory connections, we observed mice performing a task that required them to learn a rule to receive a reward and then later adapt to a new rule in order to continue receiving the reward. In this task, mice dug in bowls to find hidden food. Initially, the smell of garlic or the presence of sand within a bowl might indicate the location of the hidden food. The specific cue associated with the reward would later change, forcing the mice to learn a new rule.

We found that silencing the long-range inhibitory connections between the left and right prefrontal cortex caused the mice to get stuck, or perseverate, on one rule and prevented them from learning new ones. They were unable to change gears and learn that the old cue was now meaningless and the new cue signaled food.

Brain waves and flexible behavior

We also made surprising discoveries about how these long-range inhibitory connections create behavioral flexibility. Specifically, they synchronize a set of “brain waves” called gamma oscillations across the two hemispheres. Gamma oscillations are rhythmic fluctuations in brain activity that occur roughly 40 times per second. These fluctuations can be detected during many cognitive functions, such as when performing a task that requires holding information in your memory or making different movements based on what you see on a computer screen.

Though scientists have observed the presence of gamma oscillations for many decades, their function has been controversial. Many researchers think that the synchronization of these rhythmic fluctuations across different brain regions doesn’t serve any useful purpose. Others have speculated that synchronization across different brain regions enhances communication between those regions.

We found a completely different potential role for gamma synchrony. When long-range inhibitory connections synchronize gamma oscillations across the left and right prefrontal cortex, they seem to also gate communication between them. When mice learn to disregard a previously established rule that no longer leads to a reward, these connections synchronize gamma oscillations and seem to stop one hemisphere from maintaining unneeded activity patterns in the other. In other words, long-range inhibitory connections seem to stop input from one hemisphere from “getting in the way” of the other when it is trying to learn something new.

For example, the left prefrontal cortex can “remind” the right prefrontal cortex about your usual route to work. But when long-range inhibitory connections synchronize these two areas, they also seem to shut off these reminders and enable new patterns of brain activity corresponding to your new commute to take hold.

Finally, these long-range inhibitory connections also trigger long-lasting effects. Shutting off these connections just once caused mice to have trouble learning new rules several days later. Conversely, rhythmically stimulating these connections to artificially synchronize gamma oscillations can reverse these deficits and restore normal learning.

Cognitive flexibility and schizophrenia

Long-range inhibitory connections play an important role in cognitive flexibility. The inability to appropriately update previously learned rules is one hallmark form of cognitive impairment in psychiatric conditions such as schizophrenia and bipolar disorder.

Research has also seen deficiencies in gamma synchronization and abnormalities in a class of prefrontal inhibitory neurons, which includes the ones we studied, in people with schizophrenia. In this context, our study suggests that treatments that target these long-range inhibitory connections may help improve cognition in people with schizophrenia by synchronizing gamma oscillations.

Many details of how these connections affect brain circuits remain unknown. For example, we do not know exactly which cells within the prefrontal cortex receive input from these long-range inhibitory connections and change their activity patterns to learn new rules. We also do not know whether there are specific molecular pathways that produce the long-lasting changes in neural activity. Answering these questions could reveal how the brain flexibly switches between maintaining and updating old information and potentially lead to new treatments for schizophrenia and other psychiatric conditions.  Vikaas Sohal University of California, Kathleen Cho Inserm, MDT/THE CONVERSATION