VISUALIZATION

Computer scientists at Harvard and cognitive scientists at MIT team up to settle a debate over 'chart junk'

It's easy to spot a "bad" data visualization—one packed with too much text, excessive ornamentation, gaudy colors, and clip art. Design guru Edward Tufte derided such decorations as redundant at best, useless at worst, labeling them "chart junk." Yet a debate still rages among visualization experts: Can these reviled extra elements serve a purpose?

Taking a scientific approach to design, researchers from Harvard University and Massachusetts Institute of Technology are offering a new take on that debate. The same design elements that attract so much criticism, they report, can also make a visualization more memorable.

Detailed results were presented this week at the IEEE Information Visualization (InfoVis) conference in Atlanta, hosted by the Institute of Electrical and Electronics Engineers.

For lead author Michelle Borkin, a doctoral student at the Harvard School of Engineering and Applied Sciences (SEAS), memorability has a particular importance:

"I spend a lot of my time reading these scientific papers, so I have to wonder, when I walk away from my desk, what am I going to remember? Which of the figures and visualizations in these publications are going to stick with me?"

But it's more than grad-school anxiety. Working at the interface of computer science and psychology, Borkin specializes in the visual representation of data, looking for the best ways to communicate and interpret complex information. The applications of her work have ranged from astronomy to medical diagnostics and may already help save lives.

Her adviser, Hanspeter Pfister, An Wang Professor of Computer Science at Harvard SEAS, was intrigued by the chart junk debate, which has flared up on design blogs and at visualization conferences year after year.

Together, they turned to Aude Oliva, a principal research scientist at MIT's Computer Science and Artificial Intelligence Lab, and a cognitive psychologist by training. Oliva's lab has been studying visual memory for about six years now. Her team has found that in photographs, faces and human-centric scenes are typically easy to remember; landscapes are not.

"All of us are sensitive to the same kinds of images, and we forget the same kind as well," Oliva says. "We like to believe our memories are unique, that they're like the soul of a person, but in certain situations it's as if we have the same algorithm in our heads that is going to be sensitive to a particular type of image. So when you find a result like this in photographs, you want to know: is it generalizable to many types of materials—words, sound, images, graphs?"

"Speaking with [Pfister] and his group, it became very exciting, the idea that we could study what makes a visualization memorable or not," Oliva recalls. "If it turned out to be the same for everyone, we thought this would be a win-win result."

For Oliva's group, it would provide more evidence of cognitive similarities in the brain's visual processing, from person to person. For Pfister's group, it could suggest that certain design principles make visualizations inherently more memorable than others.

With Harvard students Azalea A. Vo '13 and Shashank Sunkavalli SM '13, as well as MIT graduate students Zoya Bylinskii and Phillip Isola, the team designed a large-scale study—in the form of an online game—to rigorously measure the memorability of a wide variety of visualizations. They collected more than 5,000 charts and graphics from scientific papers, design blogs, newspapers, and government reports and manually categorized them by a wide range of attributes. Serving them up in brief glimpses—just one second each—to participants via Amazon Mechanical Turk, the researchers tested the influence of features like color, density, and content themes on users' ability to recognize which ones they had seen before.

The results meshed well with Oliva's previous results, but added several new insights.

"A visualization will be instantly and overwhelmingly more memorable if it incorporates an image of a human-recognizable object—if it includes a photograph, people, cartoons, logos—any component that is not just an abstract data visualization," says Pfister. "We learned that any time you have a graphic with one of those components, that's the most dominant thing that affects the memorability."

Visualizations that were visually dense proved memorable, as did those that used many colors. Other results were more surprising.

"You'd think the types of charts you'd remember best are the ones you learned in school—the bar charts, pie charts, scatter plots, and so on," Borkin says. "But it was the opposite."

Unusual types of charts, like tree diagrams, network diagrams, and grid matrices, were actually more memorable.

"If you think about those types of diagrams—for example, tree diagrams that show relationships between species, or diagrams that explain a molecular chemical process—every one of them is going to be a little different, but the branching structures feel very natural to us," explains Borkin. "That combination of the familiar and the unique seems to influence the memorability."

The best type of chart to use will always depend on the data, but for designers who are required to work within a certain style—for example, to achieve a recognizable consistency within a magazine—the results may be reassuring.

"A graph can be simple or complex, and they both can be memorable," explains Oliva. "You can make something familiar either by keeping it simple or by having a little story around it. It's not really that you should choose to use one color or many, or to include additional ornaments or not. If you need to keep it simple because it's the style your boss likes or the style of your publication, you can still find a way to make it memorable."

At this stage, however, the team hesitates to issue any sweeping design guidelines for an obvious reason: memorability isn't the only thing that matters. Visualizations must also be accurate, easy to comprehend, aesthetically pleasing, and appropriate to the context.

"A memorable visualization is not necessarily a good visualization," Borkin cautions. "As a community we need to keep asking these types of questions: What makes a visualization engaging? What makes it comprehensible?"

As for the chart junk, she says diplomatically, "I think it's going to be an ongoing debate."