TY - JOUR
T1 - Multi-modal data semantic localization with relationship dependencies for efficient signal processing in EH CRNs
AU - Chen, Sijia
AU - Song, Bin
AU - Fan, Luhai
AU - Du, Xiaojiang
AU - Guizani, Mohsen
N1 - Publisher Copyright:
© 2015 IEEE.
PY - 2019/6
Y1 - 2019/6
N2 - Due to spectrum scarcity and energy consumption caused by processing and transmitting multimodal data signals in cognitive radio networks (CRNs), locating key information in the signal for further energy management in EH CRNs is necessary. Therefore, to adaptively capture semantic associations of multimedia signals, we present a novel visual-semantic reasoning framework for phrases simultaneously localization. To address the preferences limitations of current algorithms caused by the independent localizing of phrases and the ignorance of inter-phrase dependencies, our framework models the phrases simultaneously followed by inter-phrase dependencies-based jointly localization. Specifically, the framework consists of two core modules, including spatial-semantic perception tensor factorization and visual-semantic relationship reasoning network which can be denoted as SSPTF and VSRN, respectively. That is, SSPTF integrates regions and phrases into a tensor so that tensor factorization can be used to capture a shared potential association for all phrases. Furthermore, based on the predefined phrases-semantic dependencies graph, VSRN explicitly exploits the conjunctions between phrases to refine the phrase-region matching scores from SSPTF to achieve jointly localization. By constructing it as an end-to-end training architecture, the strong performance of the framework over Flicker-Entities30K on accuracy and the state-of-the-art results on some categories demonstrate the effectiveness of the proposed unified framework.
AB - Due to spectrum scarcity and energy consumption caused by processing and transmitting multimodal data signals in cognitive radio networks (CRNs), locating key information in the signal for further energy management in EH CRNs is necessary. Therefore, to adaptively capture semantic associations of multimedia signals, we present a novel visual-semantic reasoning framework for phrases simultaneously localization. To address the preferences limitations of current algorithms caused by the independent localizing of phrases and the ignorance of inter-phrase dependencies, our framework models the phrases simultaneously followed by inter-phrase dependencies-based jointly localization. Specifically, the framework consists of two core modules, including spatial-semantic perception tensor factorization and visual-semantic relationship reasoning network which can be denoted as SSPTF and VSRN, respectively. That is, SSPTF integrates regions and phrases into a tensor so that tensor factorization can be used to capture a shared potential association for all phrases. Furthermore, based on the predefined phrases-semantic dependencies graph, VSRN explicitly exploits the conjunctions between phrases to refine the phrase-region matching scores from SSPTF to achieve jointly localization. By constructing it as an end-to-end training architecture, the strong performance of the framework over Flicker-Entities30K on accuracy and the state-of-the-art results on some categories demonstrate the effectiveness of the proposed unified framework.
KW - Multi-media signals processing
KW - phrase simultaneously localization
KW - spatial-semantic perception tensor factorization
KW - unified phrase localization framework
KW - visual-semantic reasoning
UR - http://www.scopus.com/inward/record.url?scp=85067017696&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85067017696&partnerID=8YFLogxK
U2 - 10.1109/TCCN.2019.2893360
DO - 10.1109/TCCN.2019.2893360
M3 - Article
AN - SCOPUS:85067017696
VL - 5
SP - 347
EP - 357
JO - IEEE Transactions on Cognitive Communications and Networking
JF - IEEE Transactions on Cognitive Communications and Networking
IS - 2
M1 - 8612944
ER -