FastREI: Fast Rare Event Identification on X-ray Data with Cross-Stage Optimizations

High-Energy Diffraction Microscopy (HEDM) is a powerful technique for in-situ characterization of metallic microstructures, but traditional analysis methods are too computationally intensive for real-time experimental steering, often taking hours for a single scan. While machine learning frameworks like Rare Event Indicator (REI) achieve significant speedups, they face critical performance bottlenecks that render them insufficient for next-generation detectors or light sources like the APS Upgrade, which will increase data rates by over 100-fold. The primary challenge for accelerating the REI framework stems from a fundamental data scale disparity: the workflow must ingest massive, I/O-intensive images (e.g., 2048 × 2048) from disk while processing tiny, computationally inefficient patches (e.g., 15 × 15) on the GPU. This mismatch leads to high I/O costs and severe CPU-GPU load imbalance. To address these limitations, we present FastREI, a high-performance framework designed for CPU-GPU systems. FastREI implements a suite of optimizations, including strategic device placement based on workload profiling, parallel data processing via array partitioning, advanced loadbalancing techniques to saturate the GPU, and kernel fusion to reduce launch overhead. Our integrated approach reduces the end-to-end workflow time by approximately 90%, achieving a 10-fold speedup over the baseline REI framework. This acceleration enables real-time data analysis at the extreme data rates of modern light sources, paving the way for adaptive, highthroughput materials science experiments.