The Straight Story on Search Engines
Want unbiased, accurate results? Choose your search site carefully. From the big guys to the undiscovered gems, we find the sites you can trust.Let's say you're interested in digital cameras. So you search for exactly that--"digital cameras"--on MSN. But when the results come back, you find yourself saying, "What is all this stuff?" One set of matches at the top of the page is called Featured Sites. Below that is a set of links labeled Sponsored Sites. Below those is still another group called Web Directory Sites.
So what's going on? Many of these results are links that advertisers have paid MSN to put in front of you, and many others are links to sites that haven't paid a dime but that provide good information on digital cameras. Which ones are which? Even if you study MSN's explanatory boxes closely, you can't know for sure.
Featured Sites, according to MSN, are a mix of Web pages from three sources: advertisers who have paid to be there; other sites that, like MSN, are owned by Microsoft; and sites that are just plain useful. MSN doesn't identify which links come from which category. Sponsored Sites consist entirely of spots paid for by advertisers. Once you get to Web Directory Sites you might think, "Finally--I'm past the ads and down to the links that will help me." Not so fast. The first link, labeled "10 Most Popular Sites for Digital Cameras," leads you to a page from Ask Jeeves. The first ten spots on that results page are also bought and paid for. Talk about an overwhelming response--and a lot of confusion.
The fact that search engines have put parts of their results pages up for sale can shake your trust in these gateways to the Internet. But it starts to get even more troubling when you examine the listings themselves. We believe that some sites alter the "real results." (Note: In this story, we use the term real results to refer to search engines' main body of listings that--in theory--advertisers don't pay for in any way.) Plus, some observers are concerned that small sites and nonprofit groups might get bumped off the listings altogether because they can't afford to pay to be included.
We went behind the scenes to find out how search sites operate. We put the engines through their paces and found that while some sites clearly label paid links, others, like MSN, do a poor job. We also suspect that some sites adjust the results to improve their corporate bottom line--the sites deny this, however. And even though it's becoming more common for companies to pay search engines a fee in return for cataloging their Web pages, small sites and nonprofit groups aren't going to be left in the dust anytime soon. But the practice could pose a future threat.
Despite our misgivings, the situation is not completely hopeless. There's always Google. Not only does Google deliver exceptionally relevant matches, but it's also the best of the bunch at identifying ads. In fact, we like Google so much, we gave it two awards this year. Turn to "PC World's 20th World Class Awards" to read Google's accolades.
In this article, we've got the best advice on how to decipher the search results at major sites. We also offer some handy search tricks (see "Super Search Tips"). On top of that, we've discovered some useful sites you may not have stumbled on before (see "Sites for Special Searches").
Avalanche of Advertisers
The way that search sites currently handle sponsored links has caught the attention of Commercial Alert, a consumer group founded by Ralph Nader. The organization filed a complaint last July with the Federal Trade Commission alleging that major search sites disguise the links that are paid for. "So many Web searches have been hijacked by advertisers," says Gary Ruskin, Commercial Alert's executive director. "The FTC should require search sites to tell us when ads are ads." At press time, the FTC was still investigating the complaint.
According to recent survey results published by Consumer WebWatch, a group affiliated with the nonprofit organization Consumers Union, 60 percent of Internet users interviewed did not know that search sites take fees to list some Web sites more prominently than others. And 80 percent of these users felt that search engines should disclose advertising deals.
Not too long ago, Web searching was all about cool technology. Search sites competed fiercely for your clicks. Sites like Lycos and AltaVista, for instance, vied for your attention by constantly updating their search tools. But the battle among search engines has slowed dramatically--at least where innovation is concerned.
The look and feel of each popular search site may still be different, but now a lot of sites are built on the same foundation. Rather than create their own unique search tools, some of the big search sites use technology by Inktomi to conduct searches. The search business is also more focused on making money. Without a profit, search sites won't be around. Banner ads haven't been as lucrative as the sites had hoped, so the search sites are now acting more like the Yellow Pages: They're doing their best to match advertisers with potential shoppers.
Paying to Appear at the Top
Here's where paid placement links enter the scene. These sponsored links show up along with the real results when you run a query on specific keywords. Lots of search sites have deals with Overture (formerly GoTo.com), a company that specializes in distributing sponsored search results. Overture-based links currently appear on results pages of nearly all major search sites, except Google. However, at the time of this writing, AOL announced it would not renew its contract with Overture. It will start to display ads purchased through Google this summer. To advertise on Overture's search engine partners, advertisers bid an amount they will pay per click when a searcher opens a sponsored link; the highest bidder wins top placement. Note: PC World participates in Overture's paid placement program and appears as an advertiser on Overture's network of search sites.
Earning money is fine--no one wants the search engines to go out of business. But as Nader's group insists, it's up to the sites to clearly identify ads. Some do not, and here lies the problem.
Each search site writes its own rules on how to label paid placement links. How can you tell which links are ads and which are not? On some sites it's easy, and on others it's anybody's guess.
There's no question about Google's ads. They don't look like regular listings. The ads are highlighted in a different color, and each is labeled as a sponsored link. They're always separate on the results page, appearing as shaded boxes to the side of the listings or right at the top. (Full disclosure: PC World is an advertiser on Google and PC World pages appear as sponsored links.)
Most of the other search sites show their Overture-powered results before the real results, usually at the top of the page. Yahoo calls its paid listings Sponsor Matches, and the distinction is clear (even if the results page is a bit cluttered). The site presents you with shopping links first, followed by Category Matches, and then Sponsor Matches. Yahoo provides some details about how its ad system works when you click the link "What are Sponsor Matches?" Lycos calls its ads Sponsored Search Listings, but the label, with its small gray font, doesn't exactly jump out at you, and there's no link to explain the term. Ask Jeeves also offers its "sponsored links" before its real results, but it doesn't spell out what the label means.
MSN, on the other hand, has decided that it's okay to mix paid and unpaid results in its top category, Featured Sites, and not tell you which is which. Click the adjacent About icon and a screen pops up to tell you that Featured Sites are picked by editors who strive to choose the most relevant paid and unpaid results. But you still can't tell which links are paid for and which are not.
At AltaVista, you see paid links in the top spot that are labeled Products and Services--not very revealing. When you click the label for an explanation, AltaVista first says that the sites have been reviewed by editors for relevancy to the search term, then identifies the listings as ads. "We feel like we're making it pretty clear," says AltaVista spokesperson Krista Thomas.
Critics like Danny Sullivan, editor of UK-based SearchEngineWatch.com, believe that labels like AltaVista's are misleading. "I think 'sponsored' should become an industry-standard label for paid links," says Sullivan. "People want to know what sponsored links are. It doesn't mean they won't use them."
Thumbs-Down on Metasearches
Many metasearch sites--sites that combine results from several search sources in one results screen--do the worst job of labeling paid links, Sullivan says. "People assume metasearch engines are delivering the best editorial goods. The reality is, they query paid listings, so they become meta-yellow pages," he says.
For instance, MetaCrawler calls upon search engines such as AltaVista and other sites like Sprinks, which are full of paid placements. For most queries at MetaCrawler, the top category in the listings is peppered with paid links, mysteriously labeled Featured Search Results. You also see a sponsored link called MetaCrawler Suggests that often presents completely irrelevant links. For example, when we ran a query on the term "802.11b specifications," the MetaCrawler Suggests ad pointed to "Dating & Romance for over 30s"--a link to Udate.com, a matchmaking site. Hello? Relevancy, in this case, had gone out the window.
MetaCrawler's paid links are supposed to be relevant to the search at hand, says Steve Stratz, senior public relations manager at InfoSpace. (InfoSpace's technology powers the searches at Excite, WebCrawler, and MetaCrawler.) However, in recent months, the ad spots have been filled by the company's big network advertisers and they often appear on multiple sites, reports Stratz. InfoSpace recently launched its new metasearch product at Excite and WebCrawler. The company hasn't decided whether these changes will roll out on MetaCrawler.
Unless you really want to pore through tons of ads, we recommend that you kick off your Web searching anywhere but metasearch sites. Try Google or Yahoo. However, if you like the fact that metasearch sites scour numerous engines at once, check out Ixquick, a different kind of metasearch site that forgoes the slew of ads (see additional information about Ixquick in "Sites for Special Searches").
Rigged Results?
Paid links that aren't clearly labeled are bad enough, but at some sites, PC World found evidence that you can't always trust the real results. In some cases, results at AOL and MSN seem to serve the companies' corporate needs as much as searchers' interests. (Note: PC World regularly provides content for AOL's Computer Center channel.)
In our tests, we found that some portal sites plug their own products and services every chance they get. For example, when we searched for "travel" on AOL and on MSN, the companies ranked their own affiliated e-commerce sites high on the first page. Though you might expect such listings in the sponsored links, we noticed the phenomenon in the real results as well.
When you run a search for "travel" on AOL, four of the first ten results that AOL delivers are affiliated with AOL--the number one recommended site is AOL Travel. AOL ranks Expedia, Microsoft's travel site, at number 14. Plug the same keyword into MSN's search field and you'll find no mention of any AOL travel sites within the first 50 results. But Expedia appears in the top spot in two categories, Featured Sites and Sponsored Sites (go to PC World's Analysis for more details). Expedia is the second-most-visited travel site on the Web, and AOL Travel is number seven, according to recent statistics from the research firm Jupiter Media Metrix.
Why are results so different on the two portals? It's hard to find an innocent explanation. AOL and MSN both use Inktomi's technology for searches. MSN also uses Overture-powered sponsored links. When we carried out our search tests, AOL was still using Overture's technology.
Inktomi and Overture say that their customers can manipulate results and prevent competitors' sites from showing up. But both AOL and MSN deny manipulating Inktomi's search results. Neither company, though, could explain how they produce different results for identical searches.
"Every AOL search [result] features the most relevant content for AOL users, whether it is a partner or nonpartner site," says AOL spokesperson Andrew Weinstein. With the search results that MSN generates, "everything is editorially managed," says Brian Gluth, MSN senior product manager. "It's all relevancy."
Not Always Commercial-Free
AOL and MSN aren't the only sites where commercial interests seem to affect the real results. Try using MetaCrawler to search for a topic like "Abraham Lincoln." Even once we'd looked past the paid links, we were presented with a remarkable list of links to sites selling Lincoln T-shirts, vases, and other memorabilia. Of the first ten real results, only three weren't trying to sell us something. And when we ran a search for "digital cameras" on MetaCrawler, the last handful of links appearing on each page were identical--all pointing to shopping sites.
InfoSpace's Steve Stratz says MetaCrawler has no policy to place commercial sites above real results. "The search providers on MetaCrawler currently are heavy on the paid side," says Stratz. "We are working hard to even this out by partnering with pure search providers." Stratz points to recent agreements with Ask Jeeves, FAST, and LookSmart. Search engine AllTheWeb is based on FAST's search technology.
Even computer scientists can't prove or disprove whether funny business goes on within relevancy rankings. "Many search engines really don't publish reliable scientific information about the quality of their results," says Amanda Spink, an associate professor of information sciences and technology at Pennsylvania State University who recently completed a five-year study of the search habits of Excite users. "Most search sites don't like to give out this kind of competitive information," she adds.
Search companies provided only basic information about how they rank results. Google employs a complex algorithm, or mathematical formula, to rank pages. Yahoo employs a team of editors to create useful categories of Web sites that they've reviewed for inclusion in the directory. Beyond that, however, search sites refuse to explain the ins and outs of their formulas.
Spink's study showed that about 70 percent of people don't scroll beyond the first or second page of search results. That could prove troublesome if results pages become front-loaded with sponsors or paid inclusion listings--links that Web site owners pay a yearly fee to have included in a search engine's results.
Paying to Be Seen
A site owner who pays a search engine to be included receives a guarantee that the site's pages will be frequently revisited (say, every couple of days) by the search engine's spider--a tool that trolls the Web for new or updated pages. This scheme differs from paid placement, in which advertisers are guaranteed a top position in the results whenever someone searches on certain words. Paid inclusion sites pay to be looked at by the spider, but aren't guaranteed a spot at the top of the results. Sites that buy into Yahoo's plan fork over $299 per year. Lycos customers pay $18 per year, plus $12 for every URL they submit.
Search companies promise that this payment feature won't fiddle with results. "This does not affect the relevancy ranking in any way, shape, or form," says Bryan Burdick, vice president of portal services for TerraLycos, the company that owns the Lycos search site. Lycos is another company that uses the FAST search engine technology. So, according to Lycos, users will see ranked results based on what the FAST formula decides is most relevant, regardless of who has paid to be included.
Google, however, says it firmly opposes paid inclusion. "The best thing for a search engine to do is crawl as many URLs as it can, as often as possible," says Google software engineer Matt Cutts. "Treating a smaller list of sites differently isn't fair."
Search industry veterans like Sullivan foresee possible problems down the road. "There is the potential for abuse," he says. "Pages that consumers want could be ignored by spiders."
Are certain types of nonprofit sites being left behind by search engines as paid inclusion becomes widespread? Not yet, Sullivan contends. "Search engines are probably realizing that they need to do a good job of picking nonprofit and research sites," he says. "They don't want to look bad." LookSmart, for instance, offers a special site, Zeal.com, that lets nonprofit organizations sign up for consideration for free.
Are Ads All Bad?
Sponsored links and paid inclusion practices aren't all gloom and doom for users. Ads can be useful, depending on what you're searching for. If you're shopping for something specific, sponsored links can often lead you to reputable e-commerce sites. For example, if you search for "Palm M130" on any of the major sites, you'll get links to several stores, and often to ads or sponsored links with the latest prices.
That said, the huge focus on commercialism can leave you feeling like the once cool and democratic Web is becoming too much like the Yellow Pages. But if you don't like the way that search sites are making money through paid placement and paid inclusion programs, then consider the alternatives. You might find them even more distasteful.
AltaVista now lets Web sites pay to add small graphics and tag lines to the search results for a company name or product. When we searched for "oatmeal" on AltaVista, for example, AltaVista ranked the Quaker Oatmeal site number one. We saw the familiar Quaker-man logo next to the listing. In this case, the food company paid to have its logo appear--it did not pay to appear at the top. Still, imagine if the whole results page became cluttered with logos and marketing spiels.
Another alternative: those increasingly aggressive pop-up, pop-under, and "floating" ads. Somill Hwang, a San Francisco public relations account executive, has a strong reaction to such bandwidth-intensive ads. Hwang, whose friends call her the office search engine guru, says she'll take sponsored links over pop-up ads any day. "If you don't want a sponsored link, you just glance past it," she says. Annoying ads are the quickest way for search companies to lose her loyalty.
Pundits like Danny Sullivan agree that paid links will stay around. But searchers should insist that these links be called "sponsored" links. If you see labels on search sites like "special results," "featured links," or other euphemisms, your antennae should go up quickly. The good news is that you can still find results that haven't been paid for. We're impressed with Google and Yahoo, and the ways that they lead you to such listings. You can tune out the ads easily and zoom in on the links you want.
If a search engine doesn't clearly mark its ads, vote with your mouse. You wouldn't watch a TV channel that disguised infomercials as shows, and you wouldn't buy a magazine that allowed merchants to pay to be recommended in its articles. Why put up with confusion at search sites?
Laurianne McLaughlin is a freelance writer based in Massachusetts. Tom Spring, a PC World senior reporter, also contributed to this article.
Report Card: The Truth Behind Web Search Results
Google and Yahoo clearly identify advertisers' links; AltaVista and MetaCrawler don't. Our chart has the lowdown.
Report Card: The Truth Behind Web Search Results (chart)
| Site | Search technology | Clarity of labeling (word score) | Paid inclusion1 | Comments |
| AltaVista (http://www.altavista.com) | Own search engine. | Poor | Y | Products and Services paid links section doesn't clearly identify the links as ads. Also, site delivers lots of annoying pop-under ads during searches. |
| AOL (http://www.aol.com) | Inktomi search engine; AOL modifies the results for inappropriate material. | Good | Y | Sponsored links label is clear, along with the tagline Provided by a third party and not endorsed by AOL. But you get plenty of flashing, distracting banner ads. |
| Ask Jeeves (http://www.askjeeves.com) | Own search engine (now owns Teoma). | Good | Y | Featured listing link looks like an ad; Sponsored links label stands out but has no explanation. Delivers lots of annoying pop-under ads. |
| Google (http://www.google.com) | Own search engine. | Excellent | N | Does a great job all round: Clear Sponsored Link labels, bold fonts, and the page design all help to distinguish between paid and unpaid links. |
| Lycos (http://www.lycos.com) | FAST's search engine. | Average | Y | Sponsored search listings label is clear, but it's a little hard to read. Start here link is confusing; it leads you to advertisers. Company may discontinue it. |
| MetaCrawler (http://www.metacrawler.com) | Combs multiple search engines and combines results on one page. | Poor | Y | Featured Search Results label on paid links is quite confusing; MetaCrawler suggests ad contained within the search results list often delivers unrelated paid links. |
| MSN (http://www.msn.com) | Both LookSmart and Inktomi search engines; employs its own ranking formula. | Average | Y | Handy buttons explain Featured Sites and Sponsored Sites. Still, we'd like a way to find out which results inside Featured Sites are paid for. |
| Yahoo (http://www.yahoo.com) | Editors review sites and categorize them; employs Google engine for full search. | Excellent | Y | Clearly separates its Sponsor Matches paid links from the unpaid ones. But Yahoo makes heavy use of banner ads. |
Can You Spot the Ads?
Iit' not always easy to recognize paid links in the search engines' results. Google makes it obvious; MetaCrawler does not.
Portals Plug Their Sites
When we searched for "travel" on AOL and MSN, we found that these portal sites promote their own interests above others.
Search Gems: Sites for Special Searches
These 21 specialty (and less-well-known) search sites are strong alternatives to the big guys. They let you focus your searches on specific topics.
Message board central. Find postings from all over the Web on topics such as arts and antiques, computers, outdoor recreation, sports, and travel.
Truly nerdy tech specs. When you need details on a technology or a component down to the geek level, search here and you'll get plenty of info.
Deep links. The site says that its public directory gives you access to over 100,000 databases and search engines.
A librarian's eyes. Here you'll get a librarian's view of the best links for both mainstream and offbeat research. It's also useful for academic topics.
Completely commercial-free. This volunteer-led directory categorizes sites in much the way that Yahoo's directory does, but without all the sponsored links and banner ads.
Gov granddaddy. This catalog of U.S. government publications might overwhelm you with information. The Web site has copies of all federal agency reports from 1994.
Public records sleuth. This site delivers URLs for some off-the-beaten-path databases of public records. It covers a range of areas such as business, legal, medical, technical, and other specialized data.
Word wizard. You get to search among research topics and language tools at this site. You can try the translating tools for 15 languages, legal dictionaries, a computing terms dictionary, and even a thesaurus tool that can help you solve crossword puzzles.
Metasearch master. Carry out metasearches using combinations of popular sites that you pick and choose. You can search the Web, MP3s, news, and pictures--and avoid the paid sites that most other metasearch tools favor.
Find and map it. Run queries for hotels, auto shops, and other businesses anywhere in the world and watch a map pop up displaying the locale.
Magazine mania. You can search through a huge archive for specific magazine articles, or browse topics like "small business" or "recruiting."
Search on search. This site offers useful lists of search engines and directories that handle queries on images, news, shopping, online radio, discussion groups, and more.
News in context. This one-stop shop lets you search breaking news from wire services and newspapers. You can also tap into research sites for medical, education, and government topics.
Going global. Here you'll find links to regional search engines in 196 countries (and counting). You can search by topic, as well--art, Buddhism, kids, music, and wine are a few.
Multimedia jackpot. Search international sources for audio and video clips, news stories, and more. Where else will you find the Moulin Rouge soundtrack in different languages on the first try?
Statistical Resources on the Web
Stats in a sec. The site gives you a quick way to search for everything from A to, er, W. Hunt for statistics on everything from aging and ancestry to watersheds and wildlife.
People and places. Want to call the plumber you used in the next town over, but can't remember the name of his business? Search here for "plumber" by town name.
International people and places. Here you'll find links to Yellow Pages, White Pages, e-mail addresses, and fax numbers from more than 170 countries around the world.
Hooked on organization. This unusual, ad-free, and highly useful search tool queries multiple search sites and organizes the results by type of document--message boards, maps, or images, for example--not just by site name.
MP3s and more. This Canadian metasearch site lets you search news topics, online auctions, MP3s, and books. You can use its friendly MP3 search tool without viewing any music ads.
Bit o' Brit. This British site gives you everything from art history facts to academic topics to music terms to famous quotations, using a collection of reference books.
Search Help: Super Search Tips
Seek and ye shall find: We asked search experts for their best
advice. Try out these top ten searching tricks.
1. Play Favorites If you choose two or three search sites to use most often, familiarize yourself with their advanced search rules. The more you use them, the better your results.
2. Specify What You Want When you're researching a product, for example, use a query that helps the search site know what you want. Try entering "Sony Mavica reviews" instead of just "Sony Mavica." The words "compare" and "buy" help, too.
3. Quote Me Putting quotation marks around a search phrase often works magic. For example, if you include quotation marks when you search for the historian "Studs Terkel," you will avoid getting listings for cufflinks or building materials.
4. Be a Task Master You can often locate what you want by entering a task into the search field. Try typing in "update my social security" or "File my taxes," say.
5. Brush Up on Boolean Try the Boolean command AND first, to see links with all search terms, as in Intel AND memory. Go to Danny Sullivan's Boolean tips.
6. Make a Date If you want links that relate to a particular time, include the date or year in quotation marks. Example: "Olympics and 2002."
7. Learn Your Lingo If you're searching for specialized material, make a note of the specific phrases that others use in the field. For example, a fundraiser who often researches potential donors' biographies tells us that the quickest search is often "John Smith" combined with "honorary degree."
8. Think Before You Click Avoid wasting time on irrelevant sites and pages. Scan the search results blurb for the context in which your terms were used, the URL, the identity of the publisher, and the date (if available).
9. Ask the Expert Web Savvy columnist Brad Grimes says that you can often save time by going to an expert. Got a question about wine? Head to Wine Spectator, for example, instead of going to Google or Yahoo.
10. Quit It, Already It's important to know when to stop Web searching, says pundit Danny Sullivan. Depending on your query, sometimes it might be faster to pick up the phone.