A new approach to multivariate adaptive regression splines by using Tikhonov regularization and continuous optimization

2010-12-01
TAYLAN, PAKİZE
Weber, Gerhard Wilhelm
Ozkurt, Fatma Yerlikaya
Show full item record
Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
178
views
0
downloads
This paper introduces a model-based approach to the important data mining tool Multivariate adaptive regression splines (MARS), which has originally been organized in a more model-free way. Indeed, MARS denotes a modern methodology from statistical learning which is important in both classification and regression, with an increasing number of applications in many areas of science, economy and technology. It is very useful for high-dimensional problems and shows a great promise for fitting nonlinear multivariate functions. The MARS algorithm for estimating the model function consists of two algorithms, these are the forward and the backward stepwise algorithm. In our paper, we propose not to use the backward stepwise algorithm. Instead, we construct a penalized residual sum of squares for MARS as a Tikhonov regularization problem which is also known as ridge regression. We treat this problem using continuous optimization techniques which we consider to become an important complementary technology and model-based alternative to the concept of the backward stepwise algorithm. In particular, we apply the elegant framework of conic quadratic programming. This is an area of convex optimization which is very well-structured, herewith, resembling linear programming and, hence, permitting the use of powerful interior point methods. Based on these theoretical and algorithmical studies, this paper also contains an application to diabetes data. We evaluate and compare the performance of the established MARS and our new CMARS in classifying diabetic persons, where CMARS turns out to be very competitive and promising.
Regression, Statistical Learning, MARS, Clustering, Curvature, Penalty Methods, Classification, Continuous Optimization, Conic Quadratic Programming, Well-Structured Convex Problems, Interior Point Methods
https://hdl.handle.net/11511/56537
Graduate School of Applied Mathematics, Article

Suggestions

Continuous optimization applied in MARS for modern applications in finance, science and technology
Taylan, Pakize; Weber, Gerhard Wilhelm; Yerlikaya, Fatma (2008-05-23)
Multivariate adaptive regression spline (MARS) denotes a tool from statistics, important in classification and regression, with applicability in many areas of finance, science and technology. It is very useful in high dimensions and shows a great promise for fitting nonlinear multivariate functions. The MARS algorithm for estimating the model function consists of two subalgorithms. We propose not to use the second one (backward stepwise algorithm), but we construct a penalized residual sum of squares for a ...
Efficient adaptive regression spline algorithms based on mapping approach with a case study on finance
Koc, Elcin Kartal; İyigün, Cem; Batmaz, İnci; Weber, Gerhard-Wilhelm (2014-09-01)
Multivariate adaptive regression splines (MARS) has become a popular data mining (DM) tool due to its flexible model building strategy for high dimensional data. Compared to well-known others, it performs better in many areas such as finance, informatics, technology and science. Many studies have been conducted on improving its performance. For this purpose, an alternative backward stepwise algorithm is proposed through Conic-MARS (CMARS) method which uses a penalized residual sum of squares for MARS as a T...
EVALUATING THE CMARS PERFORMANCE FOR MODELING NONLINEARITIES
Batmaz, İnci; Kartal-Koc, Elcin; Köksal, Gülser (2010-02-04)
Multivariate Adaptive Regression Splines (MARS) is a very popular nonparametric regression method particularly useful for modeling nonlinear relationships that may exist among the variables. Recently, we developed CMARS method as an alternative to backward stepwise part of the MARS algorithm. Comparative studies have indicated that CMARS performs better than MARS for modeling nonlinear relationships. In those studies, however, only main and two-factor interaction effects were sufficient to model the nonline...
Refinements, extensions and modern applications of conic multivariate adaptive regression splines
Yerlikaya Özkurt, Fatma; Weber, Gerhard Wilhelm; Department of Scientific Computing (2013)
Conic Multivariate Adaptive Regression Splines (CMARS) which has been developed at the Institute of Applied Mathematics, METU, as an alternative approach to the well-known data mining tool Multivariate Adaptive Regression Splines (MARS). CMARS is based on given data and a penalized residual sum of squares for MARS, interpreted as a Tikhonov Regularization problem. CMARS treats this problem by a continuous optimization technique called Conic Quadratic Programming (CQP). This doctoral thesis adapts the CMARS ...
Privacy-preserving horizontal federated learning methodology through a novel boosting-based federated random forest algorithm
Gençtürk, Mert; Çiçekli, Fehime Nihan; Department of Computer Engineering (2023-1-04)
In this thesis, a novel federated ensemble classification algorithm for horizontally partitioned data called Boosting-based Federated Random Forest (BOFRF) is proposed, which not only increases the predictive power of all participating sites, but also provides significantly high improvement on the predictive power of sites having unsuccessful local models. In this regard, a federated version of random forest, which is a well-known bagging algorithm, is implemented by adapting the idea of boosting to it. In ...
Citation Formats
P. TAYLAN, G. W. Weber, and F. Y. Ozkurt, “A new approach to multivariate adaptive regression splines by using Tikhonov regularization and continuous optimization,” TOP, pp. 377–395, 2010, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/56537.
METU IIS - Integrated Information Service open@metu.edu.tr