TY - GEN
T1 - Conditional Diffusion Model Optimization for Real-Time Point-of-Care Image-Quality Enhancement on Nvidia Jetson Orin Nano
AU - Jayaprakash, Tharun Kumar
AU - Tian, Ye
AU - Golebka, Jedrzej
AU - Chen, Royce W.S.
AU - Gan, Yu
AU - Thakoor, Kaveri A.
N1 - Publisher Copyright:
© 2026 IEEE.
PY - 2026
Y1 - 2026
N2 - High-resolution optical coherence tomography (OCT) is often inaccessible in low-resource settings. While diffusion models can enhance low-cost OCT scans, their computational cost prohibits embedded deployment. We solve this problem by presenting the first real-time deployment of a diffusion-based OCT enhancement model on an Nvidia Jetson Orin Nano. Our systematic pipeline achieves a 33 ms inference time by first pruning the model from 224 M (million) to 42.9 M parameters (80.8% reduction) with minimal fidelity loss (no statistically significant difference in quality (p>0.3) ). We then apply a novel hybrid INT8 quantization approach, strategically offloading unstable Softmax operations to a lightweight custom CUDA kernel. Expert evaluation shows that downstream diagnostic utility of the resulting images is also not impacted. This work provides a concrete blueprint for translating large generative models into practical, affordable medical diagnostic tools.
AB - High-resolution optical coherence tomography (OCT) is often inaccessible in low-resource settings. While diffusion models can enhance low-cost OCT scans, their computational cost prohibits embedded deployment. We solve this problem by presenting the first real-time deployment of a diffusion-based OCT enhancement model on an Nvidia Jetson Orin Nano. Our systematic pipeline achieves a 33 ms inference time by first pruning the model from 224 M (million) to 42.9 M parameters (80.8% reduction) with minimal fidelity loss (no statistically significant difference in quality (p>0.3) ). We then apply a novel hybrid INT8 quantization approach, strategically offloading unstable Softmax operations to a lightweight custom CUDA kernel. Expert evaluation shows that downstream diagnostic utility of the resulting images is also not impacted. This work provides a concrete blueprint for translating large generative models into practical, affordable medical diagnostic tools.
KW - Diffusion Models
KW - Embedded AI
KW - Medical Imaging
KW - Model Compression
KW - Optical Coherence Tomography
KW - Targeted Graph Surgery
KW - TensorRT
UR - https://www.scopus.com/pages/publications/105041624066
UR - https://www.scopus.com/pages/publications/105041624066#tab=citedBy
U2 - 10.1109/ISBI61048.2026.11516048
DO - 10.1109/ISBI61048.2026.11516048
M3 - Conference contribution
AN - SCOPUS:105041624066
T3 - Proceedings - International Symposium on Biomedical Imaging
BT - ISBI 2026 - 23rd IEEE International Symposium on Biomedical Imaging
T2 - 23rd IEEE International Symposium on Biomedical Imaging, ISBI 2026
Y2 - 8 April 2026 through 11 April 2026
ER -