TY - JOUR
T1 - BIAS REDUCTION IN SAMPLE-BASED OPTIMIZATION
AU - Dentcheva, Darinka
AU - Lin, Yang
N1 - Publisher Copyright:
© 2022 Society for Industrial and Applied Mathematics
PY - 2022
Y1 - 2022
N2 - We consider the stochastic optimization problems which use observed data to estimate essential characteristics of the random quantities involved. Sample average approximation (SAA) or empirical (plug-in) estimation are very popular ways to use data in optimization. It is well known that SAA suffers from downward bias. Our proposal is to use smooth estimators rather than empirical ones in the optimization problems. We establish consistency results for the optimal value and the set of optimal solutions of the new problem formulation. The performance of the proposed approach is compared to SAA theoretically and numerically. We analyze the bias of the new problems and identify sufficient conditions for ensuring less biased estimation of the optimal value of the true problem. At the same time, the error of the new estimator remains controlled. Those conditions are satisfied for many popular statistical problems such as regression models, classification problems, and optimization problems with average (conditional) value at risk. We have proved that smoothing the least-squares objective in a regression problem by a normal kernel leads to a ridge regression. Our numerical experience shows that the new estimators also frequently exhibit smaller variance and smaller mean-square error than those of SAA.
AB - We consider the stochastic optimization problems which use observed data to estimate essential characteristics of the random quantities involved. Sample average approximation (SAA) or empirical (plug-in) estimation are very popular ways to use data in optimization. It is well known that SAA suffers from downward bias. Our proposal is to use smooth estimators rather than empirical ones in the optimization problems. We establish consistency results for the optimal value and the set of optimal solutions of the new problem formulation. The performance of the proposed approach is compared to SAA theoretically and numerically. We analyze the bias of the new problems and identify sufficient conditions for ensuring less biased estimation of the optimal value of the true problem. At the same time, the error of the new estimator remains controlled. Those conditions are satisfied for many popular statistical problems such as regression models, classification problems, and optimization problems with average (conditional) value at risk. We have proved that smoothing the least-squares objective in a regression problem by a normal kernel leads to a ridge regression. Our numerical experience shows that the new estimators also frequently exhibit smaller variance and smaller mean-square error than those of SAA.
KW - kernel estimators
KW - regularized regression
KW - sample average approximation
KW - smoothing
KW - stochastic programming
KW - strong law of large numbers
UR - http://www.scopus.com/inward/record.url?scp=85131255446&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85131255446&partnerID=8YFLogxK
U2 - 10.1137/20M1326428
DO - 10.1137/20M1326428
M3 - Article
AN - SCOPUS:85131255446
SN - 1052-6234
VL - 32
SP - 130
EP - 151
JO - SIAM Journal on Optimization
JF - SIAM Journal on Optimization
IS - 1
ER -