I recently finished a story I’d spent several months obsessing over. When I pitched the piece to my editor, I knew that I’d found a worthy subject, but I couldn’t quite articulate what the story was about. It wasn’t that I didn’t have the words — it’s that I didn’t have an answer yet. All I knew was that I had something interesting that I couldn’t help pursuing, even if I had no clue what it would become.
After months of dissecting research papers, interviewing experts, stumbling down “dabbit holes” (as we call them here at FiveThirtyEight1
) and not writing a single draft, my editor gave me a non-negotiable deadline, and I spiraled into a well of despair. I had a desk cluttered with scientific papers, a hard drive stuffed with gigabytes of research and three chalkboards covered in illegible notes, yet still no tangible form for my obsession. Only in the final hours, with the deadline closing in, did something resembling a story emerge. The first draft that I puked out was no masterpiece, but it was finally something. All those scribbles and stacks of paper were necessary steps, but only in retrospect can I see where they were pointing me.When I told this story to University of Central Florida computer scientist Kenneth Stanley, he nodded in recognition. I met Stanley, a mild-mannered artificial intelligence researcher, without intending to. We were at the Santa Fe Institute, where he was spending a sabbatical and I was in residence as a journalism fellow. Stanley had stumbled upon an algorithmic principle that pointed the way to creativity in science, art, culture and life, a principle he outlines in a new book, “Why Greatness Cannot Be Planned: The Myth of the Objective.” He told me that a computer algorithm he’d created suggested that my chaotic, unstructured writing process was the ideal way to produce creative work.
The story of how he discovered this concept, Stanley told me, is an example of the idea itself. He invented an algorithm called NeuroEvolution of Augmenting Topologies (NEAT) with the purpose of evolving artificial neural networks. Soon, other programmers were building computer apps that used NEAT to evolve pictures. “I was fascinated,” Stanley said. Although he’d invented the algorithm, it had never occurred to him that it could be used to make art. He was hooked, and he and some colleagues created a website called Picbreeder where anybody could use NEAT to breed pictures the way you might breed a dog or cat.
In Picbreeder’s nine years of operation, its users have evolved nearly 10,000 images. It works by first showing the user a grid of 15 images like the one below.
Stanley calls these “blobs.” You select a blob — the one in the center, say — and then the algorithm takes it and produces children.
You pick one of the children, which then has its own children, and so on. The starting blobs have random topologies and don’t look like real objects. But as users kept selecting the new, spin-off blobs, something larger emerged. The images below were created via Picbreeder’s evolutionary process — users evolved color along with everything else. “These are not artist renderings or anything like that,” Stanley said, “and yet they look like things from nature.” Furthermore, this wasn’t just one fluke result. Users were regularly evolving complex pictures that looked like real things. Stanley started wondering, why are people so consistently finding interesting things?
He found an answer by playing Picbreeder. Out of the blue one day, Stanley created a car. “I wasn’t looking for a car,” he told me, and when he discovered it, his immediate reaction was, “What just happened? How did I just do that?”
Stanley had started with an image that had two circles inside of an oval. “I thought it looked like an alien face, sort of like E.T.,” he said. He picked the alien face thinking that he’d evolve some more E.T.s. But it didn’t happen that way. As the images evolved, the eyes of the alien started to descend, and at some point they began looking like wheels. “In hindsight, you can see the eyes turning into the wheels, but looking forward, you’d never be able to predict something like that,” he said. Most people wouldn’t look at the first picture and say, yeah, I’m going to turn that face into a car, yet that’s exactly what happened.
Stanley could have assumed that he’d won the Picbreeder lottery with a lucky event. But when he looked back at the path that users had taken to get butterflies and skulls and teapots and sunsets, he found that it was always the same story. None of the precursors looked like the final products. No one got a butterfly by selecting the blob that looked most butterfly-like. In fact, Stanley says, if you pick the picture in each round that looks most like the butterfly, you never get the butterfly.
The same sort of blind process happened in another series of experiments where Stanley and Joel Lehman instructed robots to work toward defined objectives. In one experiment a bipedal robot programmed to walk farther and farther actually ended up walking less far than one that simply was programmed to do something novel again and again, Stanley writes. Falling on the ground and flailing your legs doesn’t look much like walking, but it’s a good way to learn to oscillate, and oscillation is the most effective motion for walking. If you lock your objectives strictly on walking, you won’t hit that oscillation stepping stone. Stanley calls this the “objective paradox” — as soon as you create an objective, you ruin your ability to reach it.
It would be easy to dismiss these results as some weird quirk of Picbreeder or robots, but research by psychologists such as Dean Keith Simonton at the University of California, Davis, has shown that the kind of blind searching that Stanley observed in his experiments is at the root of many creative innovations. Most creative geniuses don’t start with a specific goal and follow it through with deliberate practice, said Scott Barry Kaufman, scientific director of the Imagination Institute at the University of Pennsylvania. Instead, they maintain an openness to discovering whatever arises.
Although this openness to new ideas might sound like just waiting around for serendipity to strike, it’s a more deliberate process. For instance, Picbreeder users don’t select pictures completely at random — they pick ones that have some intrinsic potential. They don’t know that a particular blob will lead to a butterfly, Stanley said, “but they understand that this is a path that’s worth going down for nonrandom reasons.” Simonton’s research has similarly shown that the best predictor of creative achievement is an openness to experience and cognitive exploration.
None of this means that goals don’t have a place, but they’re not a great driver of creativity. Rather than beginning with a specific goal, most creative people “start out with with a hazy intuition or vision,” Kaufman told me. “After a lot of trial and error they get closer and closer to discovering what their idea is and then they become really, really gritty to flesh it out.”
Objectives are fine when you have a modest goal and the path to get there is clear. “I would sound like a kook if I was like, no one ever should have an objective ever again,” Stanley said. “If I want some lunch, I’m not going to just wander around until I stumble upon a sandwich.” But if you’re trying to create something new, an objective can stand in your way.
Seeking novelty instead of objectives is risky — not every interesting thread will pay off — but just like with stocks, the potential payoffs are higher. (It’s no coincidence that Stanley and I were having our conversation at the Santa Fe Institute, where patron writer Cormac McCarthy has written failure into the institute’s mission statement and offices are shared among researchers from different fields, because you never know what they’ll learn from each other.)
When I’m mired in a pile of overwhelming reportage, sudden insights arrive when my frustration is at its peak. It’s the point where, like that robot flailing its legs, I’m forced to try something completely different because I’ve depleted the most obvious options. After reading Stanley’s book, I’ve started to think of those moments of frustration as prerequisites to creativity instead. I’ve also gained a little more faith in my messy methods. When I give myself space to let ideas percolate, good things happen naturally.