Fast knot optimization for multivariate adaptive regression splines using hill climbing methods

Xinglong Ju, Victoria C.P. Chen, Jay M. Rosenberger, Feng Liu

    Research output: Contribution to journalArticlepeer-review

    30 Scopus citations

    Abstract

    Multivariate adaptive regression splines (MARS) is a statistical modeling approach with wide-ranging real-world applications. In the MARS model building process, knot positioning is a critical step that potentially affects the accuracy of the final MARS model. Identifying well-positioned knots entails assessing the quality of many knots in each model building iteration, which requires intensive computational effort. By exploring the change in the residual sum of squares (RSS) within MARS, we find that local optima from previous iterations can be very close to those of the current iteration. In our approach, the prior change in RSS information is used to “warm start” an optimal knot positioning. We propose two methods for MARS knot positioning. The first method is a hill climbing method (HCM), which ignores prior change in RSS information. The second method is a hill climbing method using prior change in RSS information (PHCM). Numerical experiments are conducted on data with up to 30 dimensions. Our results show that both versions of hill climbing methods outperform a standard MARS knot selection method on datasets with different noise levels. Further, PHCM using prior change in RSS information performs best in both accuracy and computational speed. In addition, an open source Python code will be available upon acceptance of the paper on GitHub ( https://github.com/JuXinglong/MARSHC).

    Original languageEnglish
    Article number114565
    JournalExpert Systems with Applications
    Volume171
    DOIs
    StatePublished - 1 Jun 2021

    Keywords

    • Hill climbing
    • Knot optimization
    • Knot positioning
    • MARS
    • Regression

    Fingerprint

    Dive into the research topics of 'Fast knot optimization for multivariate adaptive regression splines using hill climbing methods'. Together they form a unique fingerprint.

    Cite this