Mobile Search Masterclass: How Google & Yahoo Really Measure Up; Is Paid Search The Path To Discovery?
In the last of a series of in-depth posts exploring the ideas and insights presented at the recent Mobile Search Masterclass, I’d like to hone in on a series of revealing, real-life case studies from Mobile Commerce Ltd. (MC), a must-watch company that – among its many other capabilities – possesses what the founders call a “piece of enablement” that gives them deep insight into the search queries passed through the operator portals in the U.K., and the results set returned to the user.
This central position, combined with the company’s prowess in search advertising, makes MC a top address for the inside track on the mobile search experience and the effectiveness of search to expose publishers/content users would likely appreciate in the top results. (MC tells me it has recently added The Times, and Tiscali to its local search customer base. It is currently “building out with MSN” and close to sealing a deal with a handset manufacturer to “provide a GPS app on the handset.” So watch this space…
Speaking of “scoops,” Colin Bates, MC CTO, treated the Masterclass participants to a fact-filled (and rather controversial) presentation of mobile search case studies that I simply must share. Is there really a mobile Web? Are users enjoying their mobile search experience? Does paid search pay? These are some of the thought-provoking questions Colin addressed in his contribution to the session.
As Colin sees it -and I share his view – the proliferation of mobile search services changes all the rules.
Search boxes are everywhere, and at the least a main feature on the vast majority of mobile operator portals. At the same time that mobile search has become more central to our content experiences (or perhaps because of it), mobile operators are moving past their walled garden approach, opting instead for a more balanced strategy that showcases their own portal content alongside a mix of content (mobile Web & Web) from off-portal sites and providers.
Some facts to back this up. MC sees a marked rise in mobile search, and estimates usage hovers at around 30 million searches per month. Operators (that is, the search box on their portals) are generating 85 percent of these searches.
Why? A big reason is convenience. Mobile operators – at least in the U.K. – have succeeded in making mobile search a no-brainer. The search box is displayed prominently on the portal home page, and the portal destination itself is often times one click from when the user turns on the phone. (This list shows the tie-ups between mobile operators and branded search providers in the U.K. It’s interesting to note that Yahoo, a company perceived to be in decline in the online space, has gained significant traction in mobile.)
Mobile search – just as its online counterpart – is well on its way to becoming the de facto interface to all things digital on mobile. However, the actual search volume per operator depends not so much on user behavior per se, as it does the precise position of the search box on the screen.
Likewise, click-through on sponsored results is dependent upon their position on the page. This is significant because paid advertising is widely regarded as a form of content discovery. It goes a long way toward taking the pain out of finding and buying content. What’s more, numerous studies report users cannot distinguish between search results and sponsored results on a page. In mobile, all links look the same…
Against this backdrop, the purpose (and pay-off) of paid search takes on a whole new meaning. In addition to being a key revenue driver for mobile operators and their partner mobile search providers, it is emerging as the one sure-fire way to get a top-notch spot on the tiny screen.
As Colin pointed out, mobile search is miles better than it was just last year, but it still has a way to go. In a word, information retrieval is poor at best. Recall is good, and we’re offered tons of results (which unfortunately is precisely what we don’t want on a small screen). Precision is the opposite. “We rarely get what we want because the stuff can’t be found [by search engines].”
To complicate matters, the mobile Web remains the rather poor cousin of the real Web. There is no cross-linking data to power PageRank algorithms, which is a major reason why universal search providers (think Google, Yahoo, Microsoft) fall down on the job. It also doesn’t help that mobile websites (some, not all of course) are a nightmare to index. Information/content is spread across multiple pages; the language is terse (so it’s tough to even identify what the site is about); and the content is dynamic, perishable, and silo-ized.
This observation is more than a confirmation of what I have said all along (retrofitted Web search solutions on mobile short change the users and the mobile ecosystem); it explains why Google and Yahoo (Colin did not road test Microsoft) ask us to indicate where we want them to search, and typically deliver relevant results for the Web and rubbish for the mobile Web.
In the screenshots below, a query for “directory enquiries” made to Web pages correctly delivers the phone book from BT. The query made to mobile sites returned a result from 2007 linking users to the National AIDS Trust. Talk about relevant results!
Yahoo results are much better. It appears that Yahoo has sharpened its focus on identifying the best quality mobile Web results (in terms of relevance and user experience) for each of the top query keywords.
Connect the dots, and paid search listings are the best and most effective way to get your content in front of the users who may appreciate it. Counting on Google, Yahoo, or any other Internet giant turned mobile to find your mobile site, no matter how relevant it may be to a search query, is a long wait. If your business model depends on being found by a search engine, then you may well go out of business before you show up in the top results. (Remember you have to show up in the top three.)
Is paid search the way to go? To test this theory, MC set up a site and an experiment. It purchased keywords and measured click-through – and the positive results speak for themselves. (BTW: Notice the difference in CTRs between Google and Yahoo.)
At a much higher level, this exercise shows that the mobile Web is not just a temporary stop on our journey to the real and true one Web. There are reasons and advantages to building a mobile site that I picked up during my recent briefing with Paul Nerger, VP, Advanced Services and Applications, with DotMobi. I’ll have the complete Q&A next week, as well as a few surprise stats, the inside track on the DotMobi roadmap, and a preview of a new DotMobi site set to launch soon where designers can meet, learn, and exchange best practice.
The message: PageRank doesn’t work but that is no reason not to build a mobile site. Indeed, mobile evangelist and independent consultant Tomi Ahonen has often and expertly argued that mobile is different and our approach to this medium must begin with a rethink. Success in mobile depends on our ability to “think mobile” – and it’s not about moving chunks of Internet content to mobile.
On a final note, Colin also provided us invaluable insight into queries and what users really want from mobile search. Since MC sits in the middle of the conversation that takes place, the company has a first-hand view of top queries and first-hand evidence of a Long Tail of search queries.
What do users look for?
Social networking/chat/dating services lead the pack, followed by adult, other search engines, and Internet brands (Google Yahoo, eBay etc…). Much further down, and accounting for a much smaller percentage of searches, are entertainment (music, content, downloads etc…), sports, games, and local. If we examine the top terms in social networking, we find the list includes: Facebook, Bebo, chat, dating, YouTube, e-buddy, faceparty, friends united, peperonity, and mobile community (in that order).
Ironically, Facebook has experienced a huge drop in the number of queries since about March/April. This is because Facebook now has a mobile site, and users don’t have to look for it anymore. Again, having a mobile destination makes business sense.
A final observation from Colin: “Users are grazing, not researching. They are looking for time-fillers rather than facts, and they are using search boxes for site-finding rather than data-finding.” Read between the lines and this also plays in favor of Taptu’s business model and belief that mobile search is entertainment. It’s a new way of thinking that MSG examined here.
My take: It’s a fact-filled presentation and speaks volumes about mobile search and search advertising. The proliferation of mobile search boosts usage and turns up the pressure on publishers to go mobile. Should they have mobile websites or keep with their Internet websites? It’s not either/or – it’s both. It’s really a question of business models. Generally speaking, sites that want to sell something should have a made-for-mobile presence. And once you have that covered, how do you get customers? Sure you can submit your site maps to Google & Co. – but you should also invest in advertising as a means to achieve your goal. Advertising is emerging as a more reliable means of discovery than mobile search, so include banner ads, text links, and sponsored mobile results in your strategy. WAP and Web discovery (alone) is a risky business at best.
July 31st, 2008 at 8:42 am
Hi Peggy, I’ll pick up the conversation with Colin directly but though the Google vs Yahoo! comparison looks compelling there are a couple of points worth highlighting:
(1) The results are to some extent dependent on the search used. Yahoo! responds better (by its own admission) to requests for downloads and related material rather than more general searches and this may have skewed the result.
but more importantly,
(2) The service is only as good as the depth that it offers. If you are trying to build a decent sized business watch out. There is no depth in the Yahoo! base. I would argue for those serious about building a meaningful audience not to lose focus on Google and other market places like AdMob. Searching for a “better quality” CTR is not necessarily the primary driver.
August 4th, 2008 at 4:40 pm
[...] this jives with the mobile search road test conducted by Mobile Commerce Ltd, which MSG reported here. Once again we can see for ourselves that retrofitted Web search solutions (think Google, Yahoo, [...]
August 18th, 2008 at 1:05 pm
[...] mark over the visibility – hence effectiveness — of the ad, and lends credence to this must-read post as well as the startling conclusion that Internet search engines (think Google) are playing up [...]
March 5th, 2009 at 5:56 pm
[...] real Web. There is no cross-linking data to power PageRank algorithms, which is a major reason why regular road tests performed by companies such as Mobile Commerce Ltd. – a U.K. company that, among its many other [...]
March 9th, 2009 at 10:06 pm
[...] While I may have my issues with some of the Bernstein report findings, there’s no arguing the fact that Google controls a sizeable share of the U.S. mobile search market, and that despite the fact the search giant consistently delivers a poorer user experience (an observation based on MSPR findings as well as those reported by Mobile Commerce during a recent Mobile Search Master Class). [...]
April 6th, 2009 at 9:30 pm
[...] of sources quoted on MSG over the last months, including Mobile Commerce (which revealed data during a recent Mobile Search Masterclass that proves Google mobile search results tend to be less than satisfactory) and a recent report from [...]