The last time you studied a painting in an art gallery or museum, you probably observed its textures, shapes, size, colors and/or vibrancy — basically a diverse array of its details and elements. By the time you get home and describe the painting to your family or roommate, you probably won’t be able to recall all its details. But that’s OK. You remember enough for your goal here, to share your excitement with someone else. What remains in your memory will be the most important piece of information that you want to convey. My research tries to peek behind the curtains to see what’s going on in the brain when we form these selective memories.

For example, let’s say you were impressed by the painting’s colors. The image in your mind a short time later typically would be primarily a collection of colors, but that collection would not be associated with specific objects from the painting. However, if you were impressed with the painting’s objects and shapes, your mind might retain those but render them only in black and white in your memory. Psychologists have noticed this trend across many individuals — our memory is selective in response to our primary goal.

In both cases, you probably would not recall other environmental elements, such as the lighting in the gallery, the frame of the painting, etc. This pattern in how people form memories tells us that what we remember is not necessarily the entirety of what we have seen. So, what is the nature of how our brains represent memories? And how might these representations vary with our intentions after we take in information?

To tackle these questions, we examined human brain activity in experiments that asked people to whom we showed images to remember them for a short period after the images disappear. To keep it simple, the images were oriented Gabor patches. These are parallel arrays of black and white stripes portrayed in different orientations. People are asked to look at the Gabor patches through different apertures, as if they were viewing them through differently shaped windows. Some of the images are behind a pinwheel-shaped aperture, while others are behind a donut-shaped aperture. Examples of Gabor patches rotating their orientations behind different apertures are showing below.


donut-shaped aperturedonut-shaped aperture
pinwheel-shaped aperturepinwheel-shaped aperture

These apertures can affect the neural activity in the back of your brain (visual cortex) in a specific way. However, your task, or goal, in our experiments has nothing to do with those apertures; we only ask you to remember the orientation of the Gabor patches and recall them after a short delay.

What we found is that while those apertures do influence how neurons respond when you see those images, they don’t affect your memory representations following the delay period. When we reconstructed the memory representations of those Gabor patches with different apertures based on the brain activity, we found an interchangeable image that was as simple as a line, aligning with the remembered orientation, no matter what the apertures were.


Illustration of what you see vs. what you rememberIllustration of what you see vs. what you remember

It turns out that what you remember in your mind is not a hard copy of the sensory input, my research shows. Instead, our mind selects, or abstracts, the simplest bits of information that are essential for our goal, such as remembering the orientation. Our finding helps to explain why our memories are not always accurate and can be biased based on our individual purposes. That’s to say, if you want to share all the details of a painting, don’t forget to take a photo. Otherwise, you might be surprised by how different it looks from what you remembered the next time you see it.

If you’re interested in more details, check out the Post or the Paper!

Big thanks to Robin Lloyd, who taught the Science Communication Workshop at NYU! I learned so much about how to share science with a general audience. I really value that experience and hope to keep up the habit of doing more science communication in the future. This is my first piece, with lots of help and feedback from Robin and my amazing classmates in the workshop.