Google tests: what a difference a few pixels make
Stephen Shankland | Sunday, 31 August 2008
In the old days, designers often had little more than gut checks and rules of thumb to determine the efficacy of creations such as advertisements or newspaper layouts. Later came expensive eye-tracking tests that showed how people scanned pages or computer screens. Now, though, Google, has the benefit of millions of users using its Web site to get things right.
The company, which in May began describing the split A/B tests it uses to see which of two alternative Google interfaces fares better, offered more details Tuesday. The theme: seemingly imperceptible differences are in fact perceptible. "We test almost everything, even things that you would think are so small that we could not possibly care (nor could they possibly matter). In fact, small changes do matter, and we do care," said Ben Gomes, a Google distinguished engineer, in a blog post. Google runs 50 to 200 tests at any given moment, he said.
Defining "better" can be tricky. Shifting elements to emphasise one can de-emphasise another, Gomes said. In one case, Google tried two versions of a "plus" box that, when clicked, shows a company's stock performance. The slightly bolder one yielded many more clicks. But is that a distraction or an improvement? Google hasn't decided which option is better overall, Gomes said. Another example, vastly more obvious, is a test that lets users move individual search results up or down in the list.
"At this point, I can't say what we expect from this feature; we're just curious to see how it will be used," Gomes said.
The company, which in May began describing the split A/B tests it uses to see which of two alternative Google interfaces fares better, offered more details Tuesday. The theme: seemingly imperceptible differences are in fact perceptible. "We test almost everything, even things that you would think are so small that we could not possibly care (nor could they possibly matter). In fact, small changes do matter, and we do care," said Ben Gomes, a Google distinguished engineer, in a blog post. Google runs 50 to 200 tests at any given moment, he said.
Defining "better" can be tricky. Shifting elements to emphasise one can de-emphasise another, Gomes said. In one case, Google tried two versions of a "plus" box that, when clicked, shows a company's stock performance. The slightly bolder one yielded many more clicks. But is that a distraction or an improvement? Google hasn't decided which option is better overall, Gomes said. Another example, vastly more obvious, is a test that lets users move individual search results up or down in the list.
"At this point, I can't say what we expect from this feature; we're just curious to see how it will be used," Gomes said.