TY - JOUR
T1 - Combining Fourier and lagged k-nearest neighbor imputation for biomedical time series data
AU - Rahman, Shah Atiqur
AU - Huang, Yuxiao
AU - Claassen, Jan
AU - Heintzman, Nathaniel
AU - Kleinberg, Samantha
N1 - Publisher Copyright:
© 2015 Elsevier Inc.
PY - 2015/12/1
Y1 - 2015/12/1
N2 - Most clinical and biomedical data contain missing values. A patient's record may be split across multiple institutions, devices may fail, and sensors may not be worn at all times. While these missing values are often ignored, this can lead to bias and error when the data are mined. Further, the data are not simply missing at random. Instead the measurement of a variable such as blood glucose may depend on its prior values as well as that of other variables. These dependencies exist across time as well, but current methods have yet to incorporate these temporal relationships as well as multiple types of missingness. To address this, we propose an imputation method (FL. k-NN) that incorporates time lagged correlations both within and across variables by combining two imputation methods, based on an extension to k-NN and the Fourier transform. This enables imputation of missing values even when all data at a time point is missing and when there are different types of missingness both within and across variables. In comparison to other approaches on three biological datasets (simulated and actual Type 1 diabetes datasets, and multi-modality neurological ICU monitoring) the proposed method has the highest imputation accuracy. This was true for up to half the data being missing and when consecutive missing values are a significant fraction of the overall time series length.
AB - Most clinical and biomedical data contain missing values. A patient's record may be split across multiple institutions, devices may fail, and sensors may not be worn at all times. While these missing values are often ignored, this can lead to bias and error when the data are mined. Further, the data are not simply missing at random. Instead the measurement of a variable such as blood glucose may depend on its prior values as well as that of other variables. These dependencies exist across time as well, but current methods have yet to incorporate these temporal relationships as well as multiple types of missingness. To address this, we propose an imputation method (FL. k-NN) that incorporates time lagged correlations both within and across variables by combining two imputation methods, based on an extension to k-NN and the Fourier transform. This enables imputation of missing values even when all data at a time point is missing and when there are different types of missingness both within and across variables. In comparison to other approaches on three biological datasets (simulated and actual Type 1 diabetes datasets, and multi-modality neurological ICU monitoring) the proposed method has the highest imputation accuracy. This was true for up to half the data being missing and when consecutive missing values are a significant fraction of the overall time series length.
KW - Biomedical data
KW - Imputation
KW - Missing data
KW - Time series
UR - http://www.scopus.com/inward/record.url?scp=84947942678&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84947942678&partnerID=8YFLogxK
U2 - 10.1016/j.jbi.2015.10.004
DO - 10.1016/j.jbi.2015.10.004
M3 - Article
C2 - 26477633
AN - SCOPUS:84947942678
SN - 1532-0464
VL - 58
SP - 198
EP - 207
JO - Journal of Biomedical Informatics
JF - Journal of Biomedical Informatics
ER -