TY - JOUR
T1 - Fast knot optimization for multivariate adaptive regression splines using hill climbing methods
AU - Ju, Xinglong
AU - Chen, Victoria C.P.
AU - Rosenberger, Jay M.
AU - Liu, Feng
N1 - Publisher Copyright:
© 2021 Elsevier Ltd
PY - 2021/6/1
Y1 - 2021/6/1
N2 - Multivariate adaptive regression splines (MARS) is a statistical modeling approach with wide-ranging real-world applications. In the MARS model building process, knot positioning is a critical step that potentially affects the accuracy of the final MARS model. Identifying well-positioned knots entails assessing the quality of many knots in each model building iteration, which requires intensive computational effort. By exploring the change in the residual sum of squares (RSS) within MARS, we find that local optima from previous iterations can be very close to those of the current iteration. In our approach, the prior change in RSS information is used to “warm start” an optimal knot positioning. We propose two methods for MARS knot positioning. The first method is a hill climbing method (HCM), which ignores prior change in RSS information. The second method is a hill climbing method using prior change in RSS information (PHCM). Numerical experiments are conducted on data with up to 30 dimensions. Our results show that both versions of hill climbing methods outperform a standard MARS knot selection method on datasets with different noise levels. Further, PHCM using prior change in RSS information performs best in both accuracy and computational speed. In addition, an open source Python code will be available upon acceptance of the paper on GitHub ( https://github.com/JuXinglong/MARSHC).
AB - Multivariate adaptive regression splines (MARS) is a statistical modeling approach with wide-ranging real-world applications. In the MARS model building process, knot positioning is a critical step that potentially affects the accuracy of the final MARS model. Identifying well-positioned knots entails assessing the quality of many knots in each model building iteration, which requires intensive computational effort. By exploring the change in the residual sum of squares (RSS) within MARS, we find that local optima from previous iterations can be very close to those of the current iteration. In our approach, the prior change in RSS information is used to “warm start” an optimal knot positioning. We propose two methods for MARS knot positioning. The first method is a hill climbing method (HCM), which ignores prior change in RSS information. The second method is a hill climbing method using prior change in RSS information (PHCM). Numerical experiments are conducted on data with up to 30 dimensions. Our results show that both versions of hill climbing methods outperform a standard MARS knot selection method on datasets with different noise levels. Further, PHCM using prior change in RSS information performs best in both accuracy and computational speed. In addition, an open source Python code will be available upon acceptance of the paper on GitHub ( https://github.com/JuXinglong/MARSHC).
KW - Hill climbing
KW - Knot optimization
KW - Knot positioning
KW - MARS
KW - Regression
UR - http://www.scopus.com/inward/record.url?scp=85100436110&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85100436110&partnerID=8YFLogxK
U2 - 10.1016/j.eswa.2021.114565
DO - 10.1016/j.eswa.2021.114565
M3 - Article
AN - SCOPUS:85100436110
SN - 0957-4174
VL - 171
JO - Expert Systems with Applications
JF - Expert Systems with Applications
M1 - 114565
ER -