Date

2023-1-18

Author

Zubaroğlu, Alaettin

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

82
views

121
downloads

Cite This

Number of connected devices is steadily increasing and this trend is expected to continue in the near future. Connected devices continuously generate data streams and the data streams may often be high dimensional and contain concept drift. Real-time processing of data streams is arousing interest despite many challenges. When limited information is available about the data and its labels, unsupervised learning and particularly clustering becomes an important method of analysis. However, most clustering algorithms require the number of clusters to be known a priori and to be given as an input to the algorithm. Moreover, data stream clustering differs from traditional clustering in many aspects and it has several challenging issues. The number of clusters even changes due to the fact that data streams evolve over time. Therefore, not only the initial number of clusters but the change in the number of clusters should also be predicted throughout the stream. Also, data embedding makes the visualization of high dimensional data possible and may simplify clustering process. There exist several data stream clustering algorithms in the literature, however no data stream embedding method exists. Uniform Manifold Approximation and Projection (UMAP) is a data embedding algorithm that is suitable to be applied on stationary (stable) data streams, though it cannot adapt concept drift. In this study, we describe two novel methods, NoCStream that predicts the number of clusters continuously; and EmCStream, to apply UMAP on evolving (non-stationary) data streams, to detect and adapt concept drift and to cluster embedded data instances using a distance or partitioning based clustering algorithm. NoCStream determines the optimal number of clusters and EmCStream embeds and clusters high dimensional evolving data streams continuously in real-time. We have evaluated EmCStream against the state-of-the-art stream clustering algorithms using both synthetic and real data streams containing concept drift. EmCStream outperforms DenStream and CluStream, in terms of clustering quality, on both synthetic and real evolving data streams. We have also evaluated NoCStream and compared its performance with other methods in terms of the prediction of number of clusters, clustering quality and its genericity. NoCStream outperforms other methods on both synthetic and real evolving data streams.

Subject Keywords

Data streams, Evolving data streams, Stream clustering, Concept drift, Drift detection, Drift adaptation, Number of clusters, K prediction

URI

https://hdl.handle.net/11511/102023

Collections

Graduate School of Natural and Applied Sciences, Thesis