Wednesday, December 19, 2007

Google Way Of Testing Search Results Quality

Peter Norvig, the Directory of Search Quality at Google , in an interview with Technology Review explained how Google tests the quality of Google search engine results. Pete's answer

We test it in lots of ways. At the grossest level, we track what users are clicking on. If they click on the number-one result, and then they're done, that probably means they got what they wanted. If they're scrolling down, page after page, and reformulating the query, then we know the results aren't what they wanted. Another way we do it is to randomly select specific queries and hire people to say how good our results are. These are just contractors that we hire who give their judgment. We train them on how to identify spam and other bad sites, and then we record their judgments and track against that. It's more of a gold standard because it's someone giving a real opinion, but of course, since there's a human in the loop, we can't afford to do as much of it. We also invite people into the labs, or sometimes we go into homes and observe them as they do searches. It provides insight into what people are having difficulty with.

And when asked about Google search engine future plans for next 2 to 5 years, he replied

You'll see integration of various kinds of content. We're getting into speech recognition and all the kinds of interfaces on phones, where you have a tiny screen and awkward keyboard. You'll see that gaining in importance. You'll see integration of our various properties. We used to put the onus on the user and ask them if they wanted Web search or image search or video search. Now we're trying to solve that for them and serve up the results in a way that makes sense.

Read more interesting answers from here at Technology Review