Abstract
Research has reported that about 10% of Web searchers utilize advanced query operators, with the other 90% using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE (" "). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.
| Original language | English |
|---|---|
| Pages (from-to) | 383-411 |
| Number of pages | 29 |
| Journal | ACM Transactions on Information Systems |
| Volume | 21 |
| Issue number | 4 |
| DOIs | |
| Publication status | Published - Oct 2003 |
| Externally published | Yes |
Fingerprint
Keywords
- Boolean operators
- Coverage
- Query operators
- Ranking
- Relative precision
- Search engines
- Web results
ASJC Scopus subject areas
- Information Systems
Cite this
Coverage, relevance, and ranking : The impact of query operators on Web search engine results. / Eastman, Caroline M.; Jansen, Bernard.
In: ACM Transactions on Information Systems, Vol. 21, No. 4, 10.2003, p. 383-411.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Coverage, relevance, and ranking
T2 - The impact of query operators on Web search engine results
AU - Eastman, Caroline M.
AU - Jansen, Bernard
PY - 2003/10
Y1 - 2003/10
N2 - Research has reported that about 10% of Web searchers utilize advanced query operators, with the other 90% using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE (" "). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.
AB - Research has reported that about 10% of Web searchers utilize advanced query operators, with the other 90% using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE (" "). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.
KW - Boolean operators
KW - Coverage
KW - Query operators
KW - Ranking
KW - Relative precision
KW - Search engines
KW - Web results
UR - http://www.scopus.com/inward/record.url?scp=2442457056&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=2442457056&partnerID=8YFLogxK
U2 - 10.1145/944012.944015
DO - 10.1145/944012.944015
M3 - Article
AN - SCOPUS:2442457056
VL - 21
SP - 383
EP - 411
JO - ACM Transactions on Information Systems
JF - ACM Transactions on Information Systems
SN - 1046-8188
IS - 4
ER -