Transcription
In this video I’d like to talk about the power of creating and comparing alternatives. And to do that I’m going to share some research that Steven Dow did as a postdoctoral scholar with me at Stanford University.
When you’re designing, does it make more sense to go for quality and try to come up with the best possible design?
Or does it make more sense to go for quantity first as a path to try and learn and understand?
There’s a story that Bayles and Orland tell about an art teacher who divides the class in half, and he tells one half of the class, “You’re going to be graded exclusively on the quality of the very best thing that you make.”
He tells the other half of the class, “You’re going to be graded on the quantity of things that you make.
Doesn’t matter how good it is; all that matters is how much that you make.”
And what this teacher found was that while the quantity group was busily churning our piles of work — and learning from their mistakes — the quality group sat around theorizing, and at the end of the day they had little more to show for their efforts than grandiose theories and piles of dead clay.
So this gives us some intuition that rapidly producing many alternatives has a lot of value.
And to explore this further, Steven and I had people create egg drop devices. You may have done this when you were in high school.
If you haven’t, it’s a lot of fun, and I suggest trying it out. And what you can do with an egg drop device, is you’re building a contraption that will protect an egg from a fall.
Here we threw one out my third-story office window and, lo and behold, the egg survives. And we tested a whole bunch of people in variance of this design and people come up with all sorts of stuff. They come up with good ideas, and bad ideas, and creative solutions, and really unimaginative ones. And one thing that is really interesting is that, in aggregate, people often pick one idea early on, and they stick with it to their detriment.
And so here is a couple participants talking about that experience.
No, I don’t know, for some reason this is… this seems to be the only idea, in that there needs to be a platform and then it’s going to cushion, if possible, with the materials. I… I don’t see any, any other way. >> I’m not a very good outside-the-box thinker, so I kind of just had one idea and I was going to try and make it work. >> I kind of went with the whole parachute idea, and what I had from the beginning. So. >> This is the best approach for such a design.) What we see here is an example of what Karl Duncker called “functional fixation.” In a number of experiments that he ran in the 1940’s he gave people tasks like this: “Attach the candle to the wall such that none of the wax drips on the table.” Ten, twenty percent of the people figured it out. Take a moment and see if you can figure it out. The solution — as a couple of you have got, but I bet many people didn’t — is to take the box that holds the tacks and use that as a container for the candle. That will protect the wax from dripping on the table.
And what’s interesting about this is that, because the tacks are in a box, we don’t see the box. If you give people the exact same set up, where the tacks are outside the box, all of a sudden the box becomes obviously available as a resource and nearly everybody solves exactly the same problem.
So Stephen and I set off and tried to figure out whether we could augment people’s design process to get them to explore more alternatives.
And one of the things that we did, is we forced people to come up with multiple alternatives in parallel. We call this parallel prototyping, and in this particular study we had people design graphical advertisements for the web.
So, we’re going to put people in one of two conditions: You’re either going to be in a serial condition, where you iteratively create six prototypes from start to finish; or in a parallel condition, where you create three alternatives, get feedback, create two more, get feedback, and then make a final one.
I should clarify that the amount of time that was available was exactly the same in both conditions, and in both conditions people got exactly the same amount of feedback.
The only difference is when and how they got it.
And, again, people come up with all sorts of stuff: Creative ideas and crummy ideas, well executed and poorly executed, and, overall, we’re able to measure, using web analytics, the click-through rate that people clicked on these advertisements.
And so, over the past several years, we’ve run millions of advertisements out on the web.
And what we see, in aggregate, is that participants who got a parallel design medicine — who were forced to create multiple alternatives in parallel — had a higher click-through rate: The ads they created were clicked on more than ads in the serial condition.
And not only that, but the people who clicked on those ads and then went to the site subsequently spent a whole lot more time on that site and what this is telling us is that we’re getting the right people through to those ads.
We also had experts — both advertising professionals and clients for this website — rate the quality of the advertisements and the experts also rated the quality of the parallel ads to be higher.
And then we had the ads rated by a crowd online for the diversity of the ads.
And what we see is that the ads in the parallel condition are also more diverse. And so why does a parallel approach yield better results?
I think one of the important things that creating multiple alternatives in parallel does, is it separates your ego from the thing that you make.
If I have only one idea and you critique it, I’m going to treat that as feedback about me; whereas if I have multiple different ideas and I get critique about them, I can see that its feedback about the ideas and not a referendum on me as a person,
Also, automatically, by creating multiple alternatives, people are inspired to compare what they’ve created and try and transfer what they’ve learned from one design as they go forward in the future.
And we see this transfer across a wide variety of domains.
For example, in Dedre Gentner’s research on business negotiation, she had participants read business school cases, and she either had people read the cases one at a time and think about each individually, or she had people read them two at a time and compare them.
And what she found was that having people compare two cases — to be able to contrast them and see similarities — yielded to a three-fold increase in the amount of wisdom that they were able to get out of those cases and transfer to a new negotiation task.
So, what we got out of this is that maybe there’s some big benefits of creating multiple alternatives, especially for design teams and not just for individual design.
So the next experiment we ran looked at sharing multiple alternatives.
Same basic idea — we have a new client this time.
And we’re going to have people either create and share multiple, create multiple and share their best, or create and share only one.
Participants came up with lots of different designs.
And [what] you can see is that the “share multiple” condition drastically outperforms the other two conditions. So being able to create and share multiple designs has especially significant benefits for teams. And there are a number of reasons for this.
I’d like to point out one in particular, which is the increase in group rapport. When we asked people how they felt about their teammate, both before and after the task, in the create- and share-one conditions, people felt worse about their teammate afterwards — the single design approach can create enmity between teammates, and hostility — whereas, when creating and sharing multiple designs, people felt better about their teammates afterwards.
One important benefit of sharing multiple designs, both with users and with designers, is that alternatives provide a vocabulary for talking about the space of possible designs.
As Tohidi and colleagues showed, this could be especially valuable for users because users don’t know what the space of possible designs is.
And so having multiple alternatives gives this vocabulary.
I hope that today’s lecture has provided you with the conceptual tools for why it’s valuable to create many different alternatives.
And I hope that this will be really useful for you as you go about your design projects.
I’ll see you next time.