TY - GEN
T1 - Causal inference with rare events in large-scale time-series data
AU - Kleinberg, Samantha
PY - 2013
Y1 - 2013
N2 - Large-scale observational datasets are prevalent in many areas of research, including biomedical informatics, computational social science, and finance. However, our ability to use these data for decision-making lags behind our ability to collect and mine them. One reason for this is the lack of methods for inferring the causal impact of rare events. In cases such as the monitoring of continuous data streams from intensive care patients, social media, or finance, though, rare events may in fact be the most important ones - signaling critical changes in a patient's status or trading volume. While prior data mining approaches can identify or predict rare events, they cannot determine their impact, and probabilistic causal inference methods fail to handle inference with infrequent events. Instead, we develop a new approach to finding the causal impact of rare events that leverages the large amount of data available to infer a model of a system's functioning and evaluates how rare events explain deviations from usual behavior. Using simulated data, we evaluate the approach and compare it against others, demonstrating that it can accurately infer the effects of rare events.
AB - Large-scale observational datasets are prevalent in many areas of research, including biomedical informatics, computational social science, and finance. However, our ability to use these data for decision-making lags behind our ability to collect and mine them. One reason for this is the lack of methods for inferring the causal impact of rare events. In cases such as the monitoring of continuous data streams from intensive care patients, social media, or finance, though, rare events may in fact be the most important ones - signaling critical changes in a patient's status or trading volume. While prior data mining approaches can identify or predict rare events, they cannot determine their impact, and probabilistic causal inference methods fail to handle inference with infrequent events. Instead, we develop a new approach to finding the causal impact of rare events that leverages the large amount of data available to infer a model of a system's functioning and evaluates how rare events explain deviations from usual behavior. Using simulated data, we evaluate the approach and compare it against others, demonstrating that it can accurately infer the effects of rare events.
UR - http://www.scopus.com/inward/record.url?scp=84896062175&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84896062175&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:84896062175
SN - 9781577356332
T3 - IJCAI International Joint Conference on Artificial Intelligence
SP - 1444
EP - 1450
BT - IJCAI 2013 - Proceedings of the 23rd International Joint Conference on Artificial Intelligence
T2 - 23rd International Joint Conference on Artificial Intelligence, IJCAI 2013
Y2 - 3 August 2013 through 9 August 2013
ER -