An enhanced Mamba-based YOLOv12 model featuring dual-stage sparse attention and enhanced interlayer feature correlation for automated lung nodule detection in CT images

Lung cancer is one of the leading causes of cancer-related mortality worldwide, and early detection of pulmonary nodules is critical for improving patient outcomes. In this study, we propose DSEM-YOLO, an enhanced deep learning framework based on YOLOv12 for automatic pulmonary nodule detection in chest CT images. The proposed framework improves the baseline model in three aspects. First, a Dual-stage Sparse Attention (DSSA) module is introduced into the backbone to enable efficient global feature extraction through a two-level sparse selection mechanism at the region and pixel levels. Second, an Enhanced Interlayer Feature Correlation (EFC) strategy is incorporated into the neck to recover fine-grained information lost during downsampling and to strengthen semantic alignment across hierarchical features. Third, a Mamba × Self-Attention (MXSA) hybrid detection head is designed to combine the linear-complexity sequence modeling of Mamba with the global contextual reasoning capability of self-attention, thereby improving background suppression and discrimination of low-contrast nodules. Extensive experiments were conducted on three datasets to evaluate the effectiveness of the proposed method. The proposed DSEM-YOLO framework achieves high-precision detection results on the LUNA16 and LUNA25 datasets, with precision scores of 94.61 % and 90.84 %, and inference speeds reaching 28.84 FPS and 30.33 FPS, respectively. In addition, DSEM-YOLO achieves outstanding adaptive performance on the independent clinical validation dataset. These results indicate that DSEM-YOLO achieves a favorable balance between detection accuracy and real-time performance, highlighting its potential for clinical application in early lung cancer screening.