Smart Miner: A new framework for mining large scale web usage data

Date

2009-12-01

Author

Bayir, Murat Ali
Toroslu, İsmail Hakkı
Coşar, Ahmet
Fidan, Guven

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

87
views

0
downloads

In this paper, we propose a novel framework called Smart-Miner for web usage mining problem which uses link information for producing accurate user sessions and frequent navigation patterns. Unlike the simple session concepts in the time and navigation based approaches, where sessions are sequences of web pages requested from the server or viewed in the browser, Smart Miner sessions are set of paths traversed in the web graph that corresponds to users' navigations among web pages. We have modeled session construction as a new graph problem and utilized a new algorithm, Smart-SRA, to solve this problem efficiently. For the pattern discovery phase, we have developed an efficient version of the Apriori-All technique which uses the structure of web graph to increase the performance. From the experiments that we have performed on both real and simulated data, we have observed that Smart-Miner produces at least 30% more accurate web usage patterns than other approaches including previous session construction methods. We have also studied the effect of having the referrer information in the web server logs to show that different versions of Smart-SRA produce similar results. Our another contribution is that we have implemented distributed version of the Smart Miner framework by employing Map/Reduce Paradigm. We conclude that we can efficiently process terabytes of web server logs belonging to multiple web sites by our scalable framework. Copyright is held by the International World Wide Web Conference Committee (IW3C2).

URI

https://hdl.handle.net/11511/56492

DOI

https://doi.org/10.1145/1526709.1526732

Collections

Department of Computer Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

Performance-based parametric design explorations: A method for generating appropriate building components Ercan, Burak; Elias Özkan, Soofia Tahira (2015-05-01) Performance-based parametric design explorations depend on formulating custom-designed workflows that require reading, writing, interpreting and manipulating databases, as part of the design process. The possibilities of customization and parameterization offered by the user-friendly interfaces of advanced building-performance simulation software and digital design tools have now enabled architects to carry out performance-based design explorations without the help of simulation experts. This paper presents...
Advanced methods for result and score caching in web search engines Yafay, Erman.; Altıngövde, İsmail Sengör; Department of Computer Engineering (2019) Search engines employ caching techniques in main memory to improve system efficiency and scalability. In this thesis, we focus on improving the cache performance for web search engines where our contributions can be separated into two main parts. Firstly, we investigate the impact of the sample size for frequency statistics for most popular cache eviction strategies in the literature, and show that cache performance improves with larger samples, i.e., by storing the frequencies of all (or, most of) the quer...
WaPUPS: Web access pattern extraction under user-defined pattern scoring Alkan, Oznur Kirmemis; Karagöz, Pınar (2016-04-01) Extracting patterns from web usage data helps to facilitate better web personalization and web structure readjustment. The classical frequency-based sequence mining techniques consider only the binary occurrences of web pages in sessions that result in the extraction of many patterns that are not informative for users. To handle this problem, utility-based mining technique has emerged, which assigns non-binary values, called utilities, to web pages and calculates pattern utilities accordingly. However, the ...
Exploiting Navigational Queries for Result Presentation and Caching in Web Search Engines Ozcan, Rifat; Altıngövde, İsmail Sengör; Ulusoy, Ozgur (Wiley, 2011-04-01) Caching of query results is an important mechanism for efficiency and scalability of web search engines. Query results are cached and presented in terms of pages, which typically include 10 results each. In navigational queries, users seek a particular website, which would be typically listed at the top ranks (maybe, first or second) by the search engine, if found. For this type of query, caching and presenting results in the 10-per-page manner may waste cache space and network bandwidth. In this article, w...
Cluster searching strategies for collaborative recommendation systems Altıngövde, İsmail Sengör; Ulusoy, Ozgur (2013-05-01) In-memory nearest neighbor computation is a typical collaborative filtering approach for high recommendation accuracy. However, this approach is not scalable given the huge number of customers and items in typical commercial applications. Cluster-based collaborative filtering techniques can be a remedy for the efficiency problem, but they usually provide relatively lower accuracy figures, since they may become over-generalized and produce less-personalized recommendations. Our research explores an individua...

Citation Formats

M. A. Bayir, İ. H. Toroslu, A. Coşar, and G. Fidan, “Smart Miner: A new framework for mining large scale web usage data,” 2009, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/56492.