A new approach to multivariate adaptive regression splines by using Tikhonov regularization and continuous optimization

2010-12-01
TAYLAN, PAKİZE
Weber, Gerhard Wilhelm
Ozkurt, Fatma Yerlikaya
This paper introduces a model-based approach to the important data mining tool Multivariate adaptive regression splines (MARS), which has originally been organized in a more model-free way. Indeed, MARS denotes a modern methodology from statistical learning which is important in both classification and regression, with an increasing number of applications in many areas of science, economy and technology. It is very useful for high-dimensional problems and shows a great promise for fitting nonlinear multivariate functions. The MARS algorithm for estimating the model function consists of two algorithms, these are the forward and the backward stepwise algorithm. In our paper, we propose not to use the backward stepwise algorithm. Instead, we construct a penalized residual sum of squares for MARS as a Tikhonov regularization problem which is also known as ridge regression. We treat this problem using continuous optimization techniques which we consider to become an important complementary technology and model-based alternative to the concept of the backward stepwise algorithm. In particular, we apply the elegant framework of conic quadratic programming. This is an area of convex optimization which is very well-structured, herewith, resembling linear programming and, hence, permitting the use of powerful interior point methods. Based on these theoretical and algorithmical studies, this paper also contains an application to diabetes data. We evaluate and compare the performance of the established MARS and our new CMARS in classifying diabetic persons, where CMARS turns out to be very competitive and promising.

Suggestions

Continuous optimization applied in MARS for modern applications in finance, science and technology
Taylan, Pakize; Weber, Gerhard Wilhelm; Yerlikaya, Fatma (2008-05-23)
Multivariate adaptive regression spline (MARS) denotes a tool from statistics, important in classification and regression, with applicability in many areas of finance, science and technology. It is very useful in high dimensions and shows a great promise for fitting nonlinear multivariate functions. The MARS algorithm for estimating the model function consists of two subalgorithms. We propose not to use the second one (backward stepwise algorithm), but we construct a penalized residual sum of squares for a ...
Efficient adaptive regression spline algorithms based on mapping approach with a case study on finance
Koc, Elcin Kartal; İyigün, Cem; Batmaz, İnci; Weber, Gerhard-Wilhelm (2014-09-01)
Multivariate adaptive regression splines (MARS) has become a popular data mining (DM) tool due to its flexible model building strategy for high dimensional data. Compared to well-known others, it performs better in many areas such as finance, informatics, technology and science. Many studies have been conducted on improving its performance. For this purpose, an alternative backward stepwise algorithm is proposed through Conic-MARS (CMARS) method which uses a penalized residual sum of squares for MARS as a T...
EVALUATING THE CMARS PERFORMANCE FOR MODELING NONLINEARITIES
Batmaz, İnci; Kartal-Koc, Elcin; Köksal, Gülser (2010-02-04)
Multivariate Adaptive Regression Splines (MARS) is a very popular nonparametric regression method particularly useful for modeling nonlinear relationships that may exist among the variables. Recently, we developed CMARS method as an alternative to backward stepwise part of the MARS algorithm. Comparative studies have indicated that CMARS performs better than MARS for modeling nonlinear relationships. In those studies, however, only main and two-factor interaction effects were sufficient to model the nonline...
Refinements, extensions and modern applications of conic multivariate adaptive regression splines
Yerlikaya Özkurt, Fatma; Weber, Gerhard Wilhelm; Department of Scientific Computing (2013)
Conic Multivariate Adaptive Regression Splines (CMARS) which has been developed at the Institute of Applied Mathematics, METU, as an alternative approach to the well-known data mining tool Multivariate Adaptive Regression Splines (MARS). CMARS is based on given data and a penalized residual sum of squares for MARS, interpreted as a Tikhonov Regularization problem. CMARS treats this problem by a continuous optimization technique called Conic Quadratic Programming (CQP). This doctoral thesis adapts the CMARS ...
A novel method for electromagnetic target classification using the music algorithm: Applied to small-scale aircraft targets
Secmen, Mustafa; Sayan, Gönül (2006-01-01)
This paper introduces a novel target classification method based on the extraction of target features by using natural response related late-time electromagnetic scattered field data. In the feature extraction stage, the use of multiple signal classification (MUSIC) algorithm together with a simple but effective feature fusion approach leads to a significant reduction in the sensitivity of classification accuracy to both aspect angle variations and the signal-to-noise ratio (SNR) levels of the data. Another...
Citation Formats
P. TAYLAN, G. W. Weber, and F. Y. Ozkurt, “A new approach to multivariate adaptive regression splines by using Tikhonov regularization and continuous optimization,” TOP, pp. 377–395, 2010, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/56537.