Your Ability to Remember a Work of Art May Be Predictable

A new study from the Brain Bridge Lab used Art Institute of Chicago paintings and a neural network to predict naturalistic memory.

By Sarah Steimer

Our memory of art may not be as subjective as we previously thought, according to a new study from UChicago’s Brain Bridge Lab and published in PNAS — the first real-world experiment of this type. The team, led by then-4^th year College student Trent Davis, studied visual properties of paintings and gallery influences to build a model capable of predicting people’s memory for artwork in the real-world.

According to Wilma Bainbridge, study co-author and an assistant professor in the Department of Psychology, the team wanted to determine how well they could predict what people remember when they visit an art museum. The assumption would be that people might have very individual perspectives on how they connect to and appreciate art, so there wouldn’t be much consistency in people's memory of different pieces.

The researchers used a neural network for the study called ResMem, a publicly available deep learning model built by the Brain Bridge Lab for estimating the memorability of an image. Bainbridge says they tested their key study questions by having ResMem predict how memorable each painting was, and seeing if its predictions correlated with human performance.

The team chose paintings featured at the Art Institute of Chicago (the film location, as Bainbridge notes for pop culture reference, “where the really poignant art museum scene was filmed in Ferris Bueller's Day Off”). First, they ran an online experiment showing more than 3,000 people a unique subset of the 4,021 paintings from the Art Institute as a stream of pictures, prompting them to press a button whenever they recognized a painting from earlier in the stream.

“From that memory performance, we see that people do tend to remember and forget the same pieces — but it’s in an artificial computer task,” Bainbridge says.

Because one of the few times we view images in the real world with the intention of trying to remember the experience is when visiting an art museum, the researchers then sent 20 participants to the Art Institute in a freeform visit. They were told to look at every piece in the American Art wing, but otherwise they could explore the exhibit as they would naturally. They could see the artwork in any order, spend as much time as they wanted, and could bring along a companion. Bainbridge also noted that the experiment took place during the pandemic when capacity was capped at 25%, giving participants more space and time to spend with the art.

After their visit, the participants were asked to take a survey-style memory test on their phones for the pieces they saw in exhibit. These works were mixed with pieces that might trip them up, such as those from the same artist, with similar content, and from the same time period, but not in the exhibit. They did not include famous pieces at the museum in the survey, as people were far more likely to remember those.

What the researchers found was that ResMem could significantly predict what people were going to remember on their visit. Further, when they looked at these famous pieces they had left out of the experiment, what the team found was particularly surprising. ResMem predicted that famous pieces were significantly more memorable than non-famous pieces. Yet ResMem has no knowledge about artwork, art history, or culture, which means that what makes a piece famous might partially be certain intrinsic visual features that are remembered by people. Another surprising finding: the beauty and emotionality of a piece were not related to how memorable it was.

“It means that our memory is surprisingly predictable, even though we're looking at something very subjective, like artwork and looking at it in the real world,” Bainbridge says.

The team also looked at other measures for the art in relation to memorability, such as location, size of the work, and more. For example, they found that larger pieces were better remembered. By combining these different features and the predictions of memorability, the researchers could predict as much as half of the variance in people's memory performance. The findings could be very useful for artists and curators, Bainbridge says, as they can design work and exhibits that are meant to last in people's memory.

Next, the team is planning to hold an art contest to further test their theory of art memorability. “We want to ask artists to try and create the most memorable piece or the most forgettable painting that they can, then submit it to us,” Bainbridge says. “And we're then going to see which are the most memorable and forgettable pieces and make an exhibit that's entirely memorable and an exhibit that's entirely forgettable.” For more information on entering this contest, please visit https://brainbridgelab.uchicago.edu/artcontest/.