Thomas Roelleke

Information Retrieval (IR) models are a core component of IR research and IR systems. The past decade brought a consolidation of the family of IR models, which by 2000 existed of relatively isolated views on TF IDF, vector-space model, probabilistic relevance framework and BM25, and language modelling (LM). Though TF-IDF is the model best known outside of IR, in the IR research community, there is the view that "TF-IDF is not a model." Moreover, TF-IDF is referred to as "purely heuristic," and BM25 and LM are viewed as instantiations of the PRF. BM25 is viewed as being a mostly heuristic composition of well-defined (since PRF-based) parameters. LM is viewed as an overall clean and surprisingly simple and effective approach to IR, but it does not come near to TF-IDF regarding the strong intuition that made TF-IDF so popular. The BIR model and the Poisson model are models that feed into BM25. Whereas the BIR model is relatively well known and understood, the Poisson model and its role in showing the probabilistic roots of F-IDF and BM25 feel less known. This book takes a horizontal approach in gathering the foundations of TF-IDF, PRF, BIR, Poisson, BM25, LM, probabilistic inference networks (PIN's) and divergence-based models. Also, we include precision and recall as a "model" since conditional probability and total probability are a link between retrieval models and evaluation models. After the model chapter, this book focuses on relationships between models, starting with the frameworks that allow for modeling or exploring the relationships. We review the role of the vector-space "model" being a framework to implement IR models. Moreover, we reconsider the PRF as the binding element between BIR (BM25), TF- IDF and LM, thereby pointing out that an LM-based relevance model "BM25-LM" is a research challenge. When pairing the models for discussing the relationships, we choose TF-IDF as a centre point, and it turns out that the most popular

Foundations and Relationships

Information Retrieval Models

Cathal Gurrin

Yulan He

Gabriella Kazai

Udo Kruschwitz

Suzanne Little

Stefan Rüger

Keith van Rijsbergen

These proceedings contain the papers presented at ECIR 2010, the 32nd Eu- pean Conference on Information Retrieval. The conference was organizedby the Knowledge Media Institute (KMi), the Open University, in co-operation with Dublin City University and the University of Essex, and was supported by the Information Retrieval Specialist Group of the British Computer Society (BCS- IRSG) and the Special Interest Group on Information Retrieval (ACM SIGIR). It was held during March 28-31, 2010 in Milton Keynes, UK. ECIR 2010 received a total of 202 full-paper submissions from Continental Europe (40%), UK (14%), North and South America (15%), Asia and Australia (28%), Middle East and Africa (3%). All submitted papers were reviewed by at leastthreemembersoftheinternationalProgramCommittee.Outofthe202- pers 44 were selected asfull researchpapers. ECIR has alwaysbeen a conference with a strong student focus. To allow as much interaction between delegates as possible and to keep in the spirit of the conference we decided to run ECIR 2010 as a single-track event. As a result we decided to have two presentation formats for full papers. Some of them were presented orally, the others in poster format. The presentation format does not represent any di?erence in quality. Instead, the presentation format was decided after the full papers had been accepted at the Program Committee meeting held at the University of Essex. The views of the reviewers were then taken into consideration to select the most appropriate presentation format for each paper.

32nd European Conference on IR Research, ECIR 2010, Milton Keynes, UK, March 28-31, 2010. Proceedings

Advances in Information Retrieval

Information Retrieval (IR) models are a core component of IR research and IR systems. The past decade brought a consolidation of the family of IR models, which by 2000 consisted of relatively isolated views on TF-IDF (Term-Frequency times Inverse-Document-Frequency) as the weighting scheme in the vector-space model (VSM), the probabilistic relevance framework (PRF), the binary independence retrieval (BIR) model, BM25 (Best-Match Version 25, the main instantiation of the PRF/BIR), and language modelling (LM). Also, the early 2000s saw the arrival of divergence from randomness (DFR). Regarding intuition and simplicity, though LM is clear from a probabilistic point of view, several people stated: "It is easy to understand TF-IDF and BM25. For LM, however, we understand the math, but we do not fully understand why it works." This book takes a horizontal approach gathering the foundations of TF-IDF, PRF, BIR, Poisson, BM25, LM, probabilistic inference networks (PIN's), and divergence-based models. The aim is to create a consolidated and balanced view on the main models. A particular focus of this book is on the "relationships between models." This includes an overview over the main frameworks (PRF, logical IR, VSM, generalized VSM) and a pairing of TF-IDF with other models. It becomes evident that TF-IDF and LM measure the same, namely the dependence (overlap) between document and query. The Poisson probability helps to establish probabilistic, non-heuristic roots for TF-IDF, and the Poisson parameter, average term frequency, is a binding link between several retrieval models and model parameters. Table of Contents: List of Figures / Preface / Acknowledgments / Introduction / Foundations of IR Models / Relationships Between IR Models / Summary & Research Outlook / Bibliography / Author's Biography / Index

Thomas Roelleke

About

Books

Join the club