When it comes to assessing creative ideas, perception is reality.
I've been eating a lot of cheeseburgers lately. I'm doing so in the name of science, kind of.
Some friends and I have been working through a March Madness-style bracket to find the tastiest cheeseburger in Chicago. Every few weeks we buy cheeseburgers from two different restaurants, enjoy them as a group, and when everyone is finished, we vote on which burger won the match-up. Afterwards, we debate the juicy details of texture, flavor, cheesiness, et cetera, and the bracket is updated. None of us are trained chefs or otherwise qualified to be cheeseburger "experts," but there tends to be pretty strong agreement among us, even in the absence of calibration on what makes for a tasty burger. We all have our own criteria.
In other words, we know tasty burger when find one.
Coincidentally, our system or rating burgers closely mirrors what's been described as the "gold standard" in assessing creativity, the Consensual Assessment Technique, or CAT.
Creative potential and creative performance are really hard to measure because it's difficult to set a standard for something that doesn't exist yet. Also, it's very tricky to compare creative thinking across domains. For example, was Van Gogh more creative as a painter than Ravel was as a composer? Or, is the best burger in town better than the best burrito in town? (I am prepared to collect data on this too, btw.)
The CAT offers a pragmatic solution to one of the most vigorous debates in creativity research: is creativity domain-specific? In other words, do scientists, musicians, and software engineers all use the same creative thinking skills, or are they different? How can different creative products be compared to one another?
"An idea or product is creative to the extent that appropriate observers agree it is creative." - Teresa Amabile
To address these questions, the CAT is predicated on the idea that a product is creative when a group of experts from that domain agree it's creative. Teresa Amabile, the researcher who developed the CAT, demonstrated that measurement of creativity is a social process. If creative ideas are, by definition, unique and new, it's tough to set a standard for evaluation. But we can measure how consistently people perceive a product as creative. It's a simple idea that requires rather tedious execution. It goes like this:
Collect a bunch of creative products. This can be drawings, math equations, cheeseburgers, whatever.
Then have a set of domain experts (e.g., artists, mathematicians, cheeseburger aficionados) provide ratings on how creative the products are. Not how much they like the product, or how technically good the products are, but which products were most creative.
Each expert judges each product relative to all the other products in the set, not against any kind of standard.
Even without any sort of training, people tend to provide very consistent assessments of what is creative versus what isn't. Importantly, the CAT allows for comparisons across domains (like burritos vs. cheeseburgers, for example).
The CAT is cumbersome to execute. It's logistically complicated, and finding "experts" can be expensive. Even the very idea of who's an expert can get tricky and have real implications. This is why we see differences from expert reviewers and the rest of us on sites like IMDb.
Still, we've learned some important things about creativity from the CAT. For example, even though there are some personality traits or abilities that are generally linked to creativity, we also know that people who produce creative ideas in one domain don't often distinguish themselves in other domains. It's also an excellent demonstration of how when it comes to the value of creative ideas, perceptions are reality. The only standard is what people tend to think at any given point in time.
Importantly, the CAT and the research it has facilitated has other implications for organizations:
Creative performance requires specificity. It's not enough for a manager or an entire business to expect creativity from its employees. Describing in what ways new ideas are needed, what outcomes are desired, and what "creative" success looks like for an organization is essential.
Creative value is social. In other words, an idea isn't creative on its own merits; people familiar with the domain have to agree it is. And those people's expectations will change over time as the zeitgeist evolves, and what's considered creative moves in and out of style (like tie-dye clothes, apparently).
Assessing creativity doesn't need to be complicated. In fact, research shows that objective measures of creativity such as patent disclosures are correlated with managers' and employees' ratings of creativity. In other words, people's opinions about creativity are often corroborated by actual results.
Part of creativity's allure is its breadth. It exists in so many places and in so many ways, measuring it can feel elusive. On the other hand, the ubiquity of creativity means we can find it almost anywhere we look. You'll know it when you see it.