ACM SIGIR Forum· 2017
A Study of Smoothing Methods for Language Models Applied to Ad Hoc Information Retrieval
- 1,571citations
- 2017year
Short summary
This paper demonstrates that smoothing techniques significantly improve language model-based ad hoc information retrieval, with performance sensitive to smoothing parameters.
AI-generated from the title and abstract; the full text is not read.
Key points
- Language modeling for retrieval ranks documents by query likelihood from estimated document language models.
- Smoothing is essential to correct maximum likelihood estimates due to data sparseness.
- Retrieval performance is sensitive to the choice of smoothing parameters.
- Several popular smoothing methods were compared on different test collections.
AI-generated from the title and abstract; the full text is not read.
Abstract
Language modeling approaches to information retrieval are attractive and promising because they connect the problem of retrieval with that of language model estimation, which has been studied extensively in other application areas such as speech recognition. The basic idea of these approaches is to estimate a language model for each document, and then rank documents by the likelihood of the query according to the estimated language model. A core problem in language model estimation is smoothing, which adjusts the maximum likelihood estimator so as to correct the inaccuracy due to data sparseness. In this paper, we study the problem of language model smoothing and its influence on retrieval performance. We examine the sensitivity of retrieval performance to the smoothing parameters and compare several popular smoothing methods on different test collection.
The authors' abstract, as published at the source. ACM SIGIR Forum, 2017 · DOI ↗
Continue with a free account
Ask the paper: 3 free questions a day about this paper; save it, get its citation, new summaries every day for your field. Takeaways are Premium.
Continue free on the webSign in with Google or Apple; no card needed. You come back to this paper.
On your phone:
Field: Information Systems
Information SystemsComputer Science