Advanced methods for diversification of results in general-purpose and specialized search engines

Download
2020-12-28
Yiğit Sert, Sevgi
Diversifying search results is a common mechanism in information retrieval to satisfy more users by surfacing documents that address different possible intentions of users. It aims to generate a result list that is both relevant and diverse when ambiguous and/or broad queries appear. Such queries have different underlying subtopics (a.k.a., aspects or interpretations) that search result diversification algorithms should consider. In this thesis, we first address search result diversification as a useful method to support search as learning, since diversification ensures to cover all possible aspects of the query in the final ranking. We argue that, in a search engine for the education domain, it is appropriate to diversify results across multiple dimensions, including the suitability of the content for different education levels and the type of the document in addition to topical ambiguity. We introduce a framework that extends the probabilistic and supervised methods for diversification that can consider the aspects of multiple independent dimensions during ranking, and demonstrate its effectiveness on a newly developed test collection. As our second contribution, we propose three different frameworks that exploit supervised learning methods to improve the effectiveness of explicit search result diversification, which presumes that query aspects are known during diversification. We also, for the first time in the literature, propose to learn the importance of aspects by leveraging query performance predictors (QPPs). We conduct our exhaustive experiments on a commonly used benchmark dataset and show that explicit diversification performance can be considerably improved using supervised learning methods without requiring large training sets or high computing capabilities. As a third contribution of this thesis, we examine the impact of static index pruning on diversification performance. We introduce two novel strategies that take into account the topical diversity of documents and preserve documents relevant to different aspects while pruning the index. We show that our proposed pruning strategies outperform the existing approaches in terms of various diversification measures.

Suggestions

Supervised learning for image search result diversification
Göynük, Burak; Altıngövde, İsmail Sengör; Department of Computer Engineering (2019)
Due to ambiguity of user queries and growing size of data living on the internet, methods for diversifying search results have gained more importance lately. While earlier works mostly focus on text search, a similar need also exists for image data, which grows rapidly as people produce and share image data via their smartphones and social media applications such as Instagram, Snapchat, and Facebook. Therefore, in this thesis, we focus on the result diversification problem for image search. To this end, as o...
Effective & efficient methods for web search result diversification
Özdemiray, Ahmet Murat; Altıngövde, İsmail Sengör; Department of Computer Engineering (2015)
Search result diversification is one of the key techniques to cope with the ambiguous and/or underspecified information needs of the web users. In this study we first extensively evaluate the performance of a state-of-the-art explicit diversification strategy and pin-point its weaknesses. We propose basic yet novel optimizations to remedy these weaknesses and boost the performance of this algorithm. Secondly, we cast the diversification problem to the problem of ranking aggregation and propose to materializ...
Explicit Search Result Diversification Using Score and Rank Aggregation Methods
Ozdemiray, Ahmet Murat; Altıngövde, İsmail Sengör (2015-06-01)
Search result diversification is one of the key techniques to cope with the ambiguous and underspecified information needs of web users. In the last few years, strategies that are based on the explicit knowledge of query aspects emerged as highly effective ways of diversifying search results. Our contributions in this article are two-fold. First, we extensively evaluate the performance of a state-of-the-art explicit diversification strategy and pin-point its potential weaknesses. We propose basic yet novel ...
Efficient processing of category-restricted queries for Web directories
Altıngövde, İsmail Sengör; Ulusoy, Oezguer (2008-01-01)
We show that a cluster-skipping inverted index (CS-IIS) is a practical and efficient file structure to support category-restricted queries for searching Web directories. The query processing strategy with CS-IIS improves CPU time efficiency without imposing any limitations on the directory size.
Second Chance: A Hybrid Approach for Dynamic Result Caching in Search Engines
Altıngövde, İsmail Sengör; Barla Cambazoglu, B.; Ulusoy, Ozgur (2011-01-01)
Result caches are vital for efficiency of search engines. In this work, we propose a novel caching strategy in which a dynamic result cache is split into two layers: an HTML cache and a docID cache. The HTML cache in the first layer stores the result pages computed for queries. The docID cache in the second layer stores ids of documents in search results. Experiments under various scenarios show that, in terms of average query processing time, this hybrid caching approach outperforms the traditional approac...
Citation Formats
S. Yiğit Sert, “Advanced methods for diversification of results in general-purpose and specialized search engines,” Ph.D. - Doctoral Program, Middle East Technical University, 2020.