Overlap among major web search engines

Amanda Spink, Bernard Jansen, Vinish Kathuria, Sherry Koshman

Research output: Contribution to journalArticle

19 Citations (Scopus)

Abstract

Purpose - This paper reports the findings of a major study examining the overlap among results retrieved by three major web search engines. The goal of the research was to: measure the overlap across three major web search engines on the first results page overlap (i.e. share the same results) and the differences across a wide range of user defined search terms; determine the differences in the first page of search results and their rankings (each web search engine's view of the most relevant content) across single-source web search engines, including both sponsored and non-sponsored results; and measure the degree to which a meta-search web engine, such as Dogpile.com, provides searchers with the most highly ranked search results from three major single source web search engines. Design/methodology/approach - The authors collected 10,316 random Dogpile.com queries and ran an overlap algorithm using the URL for each result by query. The overlap of first result page search for each query was then summarized across all 10,316 to determine the overall overlap metrics. For a given query, the URL of each result for each engine was retrieved from the database. Findings - The percent of total results unique retrieved by only one of the three major web search engines was 85 percent, retrieved by two web search engines was 12 percent, and retrieved by all three web search engines was 3 percent. This small level of overlap reflects major differences in web search engines retrieval and ranking results. Research limitations/implications - This study provides an important contribution to the web research literature. The findings point to the value of meta-search engines in web retrieval to overcome the biases of single search engines. Practical implications - The results of this research can inform people and organizations that seek to use the web as part of their information seeking efforts, and the design of web search engines. Originality/value - This research is a large investigation into web search engine overlap using real data from a major web meta-search engine and single web search engines that sheds light on the uniqueness of top results retrieved by web search engines.

Original languageEnglish
Pages (from-to)419-426
Number of pages8
JournalInternet Research
Volume16
Issue number4
DOIs
Publication statusPublished - 2006
Externally publishedYes

Fingerprint

Search engines
search engine
Search engine
Web search
World Wide Web
ranking
Websites
Values

Keywords

  • Search engines
  • World wide web

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Information Systems
  • Business, Management and Accounting(all)

Cite this

Overlap among major web search engines. / Spink, Amanda; Jansen, Bernard; Kathuria, Vinish; Koshman, Sherry.

In: Internet Research, Vol. 16, No. 4, 2006, p. 419-426.

Research output: Contribution to journalArticle

Spink, A, Jansen, B, Kathuria, V & Koshman, S 2006, 'Overlap among major web search engines', Internet Research, vol. 16, no. 4, pp. 419-426. https://doi.org/10.1108/10662240610690034
Spink, Amanda ; Jansen, Bernard ; Kathuria, Vinish ; Koshman, Sherry. / Overlap among major web search engines. In: Internet Research. 2006 ; Vol. 16, No. 4. pp. 419-426.
@article{d182e3777e404ea98be5322509c047f1,
title = "Overlap among major web search engines",
abstract = "Purpose - This paper reports the findings of a major study examining the overlap among results retrieved by three major web search engines. The goal of the research was to: measure the overlap across three major web search engines on the first results page overlap (i.e. share the same results) and the differences across a wide range of user defined search terms; determine the differences in the first page of search results and their rankings (each web search engine's view of the most relevant content) across single-source web search engines, including both sponsored and non-sponsored results; and measure the degree to which a meta-search web engine, such as Dogpile.com, provides searchers with the most highly ranked search results from three major single source web search engines. Design/methodology/approach - The authors collected 10,316 random Dogpile.com queries and ran an overlap algorithm using the URL for each result by query. The overlap of first result page search for each query was then summarized across all 10,316 to determine the overall overlap metrics. For a given query, the URL of each result for each engine was retrieved from the database. Findings - The percent of total results unique retrieved by only one of the three major web search engines was 85 percent, retrieved by two web search engines was 12 percent, and retrieved by all three web search engines was 3 percent. This small level of overlap reflects major differences in web search engines retrieval and ranking results. Research limitations/implications - This study provides an important contribution to the web research literature. The findings point to the value of meta-search engines in web retrieval to overcome the biases of single search engines. Practical implications - The results of this research can inform people and organizations that seek to use the web as part of their information seeking efforts, and the design of web search engines. Originality/value - This research is a large investigation into web search engine overlap using real data from a major web meta-search engine and single web search engines that sheds light on the uniqueness of top results retrieved by web search engines.",
keywords = "Search engines, World wide web",
author = "Amanda Spink and Bernard Jansen and Vinish Kathuria and Sherry Koshman",
year = "2006",
doi = "10.1108/10662240610690034",
language = "English",
volume = "16",
pages = "419--426",
journal = "Internet Research",
issn = "1066-2243",
publisher = "Emerald Group Publishing Ltd.",
number = "4",

}

TY - JOUR

T1 - Overlap among major web search engines

AU - Spink, Amanda

AU - Jansen, Bernard

AU - Kathuria, Vinish

AU - Koshman, Sherry

PY - 2006

Y1 - 2006

N2 - Purpose - This paper reports the findings of a major study examining the overlap among results retrieved by three major web search engines. The goal of the research was to: measure the overlap across three major web search engines on the first results page overlap (i.e. share the same results) and the differences across a wide range of user defined search terms; determine the differences in the first page of search results and their rankings (each web search engine's view of the most relevant content) across single-source web search engines, including both sponsored and non-sponsored results; and measure the degree to which a meta-search web engine, such as Dogpile.com, provides searchers with the most highly ranked search results from three major single source web search engines. Design/methodology/approach - The authors collected 10,316 random Dogpile.com queries and ran an overlap algorithm using the URL for each result by query. The overlap of first result page search for each query was then summarized across all 10,316 to determine the overall overlap metrics. For a given query, the URL of each result for each engine was retrieved from the database. Findings - The percent of total results unique retrieved by only one of the three major web search engines was 85 percent, retrieved by two web search engines was 12 percent, and retrieved by all three web search engines was 3 percent. This small level of overlap reflects major differences in web search engines retrieval and ranking results. Research limitations/implications - This study provides an important contribution to the web research literature. The findings point to the value of meta-search engines in web retrieval to overcome the biases of single search engines. Practical implications - The results of this research can inform people and organizations that seek to use the web as part of their information seeking efforts, and the design of web search engines. Originality/value - This research is a large investigation into web search engine overlap using real data from a major web meta-search engine and single web search engines that sheds light on the uniqueness of top results retrieved by web search engines.

AB - Purpose - This paper reports the findings of a major study examining the overlap among results retrieved by three major web search engines. The goal of the research was to: measure the overlap across three major web search engines on the first results page overlap (i.e. share the same results) and the differences across a wide range of user defined search terms; determine the differences in the first page of search results and their rankings (each web search engine's view of the most relevant content) across single-source web search engines, including both sponsored and non-sponsored results; and measure the degree to which a meta-search web engine, such as Dogpile.com, provides searchers with the most highly ranked search results from three major single source web search engines. Design/methodology/approach - The authors collected 10,316 random Dogpile.com queries and ran an overlap algorithm using the URL for each result by query. The overlap of first result page search for each query was then summarized across all 10,316 to determine the overall overlap metrics. For a given query, the URL of each result for each engine was retrieved from the database. Findings - The percent of total results unique retrieved by only one of the three major web search engines was 85 percent, retrieved by two web search engines was 12 percent, and retrieved by all three web search engines was 3 percent. This small level of overlap reflects major differences in web search engines retrieval and ranking results. Research limitations/implications - This study provides an important contribution to the web research literature. The findings point to the value of meta-search engines in web retrieval to overcome the biases of single search engines. Practical implications - The results of this research can inform people and organizations that seek to use the web as part of their information seeking efforts, and the design of web search engines. Originality/value - This research is a large investigation into web search engine overlap using real data from a major web meta-search engine and single web search engines that sheds light on the uniqueness of top results retrieved by web search engines.

KW - Search engines

KW - World wide web

UR - http://www.scopus.com/inward/record.url?scp=33747511445&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33747511445&partnerID=8YFLogxK

U2 - 10.1108/10662240610690034

DO - 10.1108/10662240610690034

M3 - Article

VL - 16

SP - 419

EP - 426

JO - Internet Research

JF - Internet Research

SN - 1066-2243

IS - 4

ER -