Generating an explainable time-series forecasting using heterogeneous mixture of experts with large language models

Date

2024-9-05

Author

Balaban, İsmail

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

189
views

1065
downloads

Cite This

Time series forecasting involves three distinct methodologies: statistical models, machine learning (ML) models, and deep learning (DL) models. When working with time series forecasting, it is crucial to understand and include features such as trend, seasonality, and auto-correlation of the data into the models. Statistical models incorporate these characteristics in order to enable the forecasting of future values. ML models offer robust capabilities for capturing complex correlations in complicated data structures. However, DL models work as black boxes. Statistical models utilize previous values and error terms to make predictions, allowing for a clear explanation of the impact of each parameter. ML models include methods that explain how the model functions internally, clarifying how it makes predictions or decisions. However, DL models are the least transparent among them. Our approach involves developing a Mixture of Experts (MoE) framework that uses DL models as experts while also enhancing the transparency of their decision-making processes. A novel aspect of this approach is the use of a Large Language Model (LLM) as the gating network. However, the LLM backbone is frozen in the gating network. The prompts and embeddings are modified to ensure their suitability for time series forecasting. The system outputs prediction, explainability which assigns confidence to prediction or weight assignment to expert models, and reasoning. The results demonstrate that by incorporating DL models into a MoE framework, we were able to extract insights into the reasons behind their predictions that the models could not explain on their own.

Subject Keywords

Time-series forecasting, Mixture-of-experts, Large language models, Deep learning

URI

https://hdl.handle.net/11511/111442

Collections

Graduate School of Natural and Applied Sciences, Thesis