Search log analysis: What it is, what's been done, how to do it

Research output: Contribution to journalArticle

166 Citations (Scopus)

Abstract

The use of data stored in transaction logs of Web search engines, Intranets, and Web sites can provide valuable insight into understanding the information-searching process of online searchers. This understanding can enlighten information system design, interface development, and devising the information architecture for content collections. This article presents a review and foundation for conducting Web search transaction log analysis. A methodology is outlined consisting of three stages, which are collection, preparation, and analysis. The three stages of the methodology are presented in detail with discussions of goals, metrics, and processes at each stage. Critical terms in transaction log analysis for Web searching are defined. The strengths and limitations of transaction log analysis as a research method are presented. An application to log client-side interactions that supplements transaction logs is reported on, and the application is made available for use by the research community. Suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction log analysis for Web-searching research. Finally, a complete flat text transaction log from a commercial search engine is available as supplementary material with this manuscript.

Original languageEnglish
Pages (from-to)407-432
Number of pages26
JournalLibrary and Information Science Research
Volume28
Issue number3
DOIs
Publication statusPublished - Sep 2006
Externally publishedYes

Fingerprint

transaction
Search engines
Intranets
Interfaces (computer)
Websites
Information systems
search engine
Systems analysis
Intranet
information process
methodology
supplement
research method
information system
interaction
community

ASJC Scopus subject areas

  • Library and Information Sciences

Cite this

Search log analysis : What it is, what's been done, how to do it. / Jansen, Bernard.

In: Library and Information Science Research, Vol. 28, No. 3, 09.2006, p. 407-432.

Research output: Contribution to journalArticle

@article{7a177cce4d5246e29ffff43b2d729885,
title = "Search log analysis: What it is, what's been done, how to do it",
abstract = "The use of data stored in transaction logs of Web search engines, Intranets, and Web sites can provide valuable insight into understanding the information-searching process of online searchers. This understanding can enlighten information system design, interface development, and devising the information architecture for content collections. This article presents a review and foundation for conducting Web search transaction log analysis. A methodology is outlined consisting of three stages, which are collection, preparation, and analysis. The three stages of the methodology are presented in detail with discussions of goals, metrics, and processes at each stage. Critical terms in transaction log analysis for Web searching are defined. The strengths and limitations of transaction log analysis as a research method are presented. An application to log client-side interactions that supplements transaction logs is reported on, and the application is made available for use by the research community. Suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction log analysis for Web-searching research. Finally, a complete flat text transaction log from a commercial search engine is available as supplementary material with this manuscript.",
author = "Bernard Jansen",
year = "2006",
month = "9",
doi = "10.1016/j.lisr.2006.06.005",
language = "English",
volume = "28",
pages = "407--432",
journal = "Library and Information Science Research",
issn = "0740-8188",
publisher = "Elsevier BV",
number = "3",

}

TY - JOUR

T1 - Search log analysis

T2 - What it is, what's been done, how to do it

AU - Jansen, Bernard

PY - 2006/9

Y1 - 2006/9

N2 - The use of data stored in transaction logs of Web search engines, Intranets, and Web sites can provide valuable insight into understanding the information-searching process of online searchers. This understanding can enlighten information system design, interface development, and devising the information architecture for content collections. This article presents a review and foundation for conducting Web search transaction log analysis. A methodology is outlined consisting of three stages, which are collection, preparation, and analysis. The three stages of the methodology are presented in detail with discussions of goals, metrics, and processes at each stage. Critical terms in transaction log analysis for Web searching are defined. The strengths and limitations of transaction log analysis as a research method are presented. An application to log client-side interactions that supplements transaction logs is reported on, and the application is made available for use by the research community. Suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction log analysis for Web-searching research. Finally, a complete flat text transaction log from a commercial search engine is available as supplementary material with this manuscript.

AB - The use of data stored in transaction logs of Web search engines, Intranets, and Web sites can provide valuable insight into understanding the information-searching process of online searchers. This understanding can enlighten information system design, interface development, and devising the information architecture for content collections. This article presents a review and foundation for conducting Web search transaction log analysis. A methodology is outlined consisting of three stages, which are collection, preparation, and analysis. The three stages of the methodology are presented in detail with discussions of goals, metrics, and processes at each stage. Critical terms in transaction log analysis for Web searching are defined. The strengths and limitations of transaction log analysis as a research method are presented. An application to log client-side interactions that supplements transaction logs is reported on, and the application is made available for use by the research community. Suggestions are provided on ways to leverage the strengths of, while addressing the limitations of, transaction log analysis for Web-searching research. Finally, a complete flat text transaction log from a commercial search engine is available as supplementary material with this manuscript.

UR - http://www.scopus.com/inward/record.url?scp=33748976562&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748976562&partnerID=8YFLogxK

U2 - 10.1016/j.lisr.2006.06.005

DO - 10.1016/j.lisr.2006.06.005

M3 - Article

AN - SCOPUS:33748976562

VL - 28

SP - 407

EP - 432

JO - Library and Information Science Research

JF - Library and Information Science Research

SN - 0740-8188

IS - 3

ER -