TY - GEN
T1 - Knot optimization for multivariate adaptive regression splines
AU - Ju, Xinglong
AU - Chen, Victoria C.P.
AU - Rosenberger, Jay M.
AU - Liu, Feng
N1 - Publisher Copyright:
© 2019 IISE Annual Conference and Expo 2019. All rights reserved.
PY - 2019
Y1 - 2019
N2 - Multivariate adaptive regression splines (MARS) is a popular statistical modeling method. In the MARS model building process, knot positioning is a critical aspect that potentially affects the accuracy of the final MARS model. However, identifying well-positioned knots requires assessing the quality of many knots in each model building iteration, creating a time-consuming process. By exploring the change in residual sum of squares (?RSS) function within MARS, we find that local optima from previous model building iterations are very close to those of the current iteration. In our approach, this prior ?RSS information is used to "warm start" an optimization algorithm for knot positioning. We propose two methods for MARS knot positioning. The first method is a fixed step hill climbing method (FSHC), which assumes a fixed step size in the search algorithm and ignores prior ?RSS information. The second method is FSHC using prior ?RSS information. Numerical experiments are conducted on data with up to 30 dimensions. Our results show that both versions of FSHC outperform the original MARS knot selection method. Further, FSHC using prior ?RSS information performs best in both accuracy and computational speed.
AB - Multivariate adaptive regression splines (MARS) is a popular statistical modeling method. In the MARS model building process, knot positioning is a critical aspect that potentially affects the accuracy of the final MARS model. However, identifying well-positioned knots requires assessing the quality of many knots in each model building iteration, creating a time-consuming process. By exploring the change in residual sum of squares (?RSS) function within MARS, we find that local optima from previous model building iterations are very close to those of the current iteration. In our approach, this prior ?RSS information is used to "warm start" an optimization algorithm for knot positioning. We propose two methods for MARS knot positioning. The first method is a fixed step hill climbing method (FSHC), which assumes a fixed step size in the search algorithm and ignores prior ?RSS information. The second method is FSHC using prior ?RSS information. Numerical experiments are conducted on data with up to 30 dimensions. Our results show that both versions of FSHC outperform the original MARS knot selection method. Further, FSHC using prior ?RSS information performs best in both accuracy and computational speed.
KW - Hill Climbing
KW - Knot Optimization
KW - Knot Positioning
KW - MARS
KW - Regression
UR - http://www.scopus.com/inward/record.url?scp=85095440849&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85095440849&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85095440849
T3 - IISE Annual Conference and Expo 2019
BT - IISE Annual Conference and Expo 2019
T2 - 2019 Institute of Industrial and Systems Engineers Annual Conference and Expo, IISE 2019
Y2 - 18 May 2019 through 21 May 2019
ER -