TY - JOUR
T1 - Large Multi-Modal Models (LMMs) as Universal Foundation Models for AI-Native Wireless Systems
AU - Xu, Shengzhe
AU - Kurisummoottil Thomas, Christo
AU - Hashash, Omar
AU - Muralidhar, Nikhil
AU - Saad, Walid
AU - Ramakrishnan, Naren
N1 - Publisher Copyright:
© 1986-2012 IEEE.
PY - 2024
Y1 - 2024
N2 - Large language models (LLMs) and foundation models have been recently touted as a game-changer for 6 G systems. However, recent efforts on LLMs for wireless networks are limited to a direct application of existing language models that were designed for natural language processing (NLP) applications. To address this challenge and create wireless-centric foundation models, this paper presents a comprehensive vision on how to design universal foundation models that are tailored towards the unique needs of next-generation wireless systems, thereby paving the way towards the deployment of artificial intelligence (AI)-native networks. Diverging from NLP-based foundation models, the proposed framework promotes the design of large multi-modal models (LMMs) fostered by three key capabilities: 1) processing of multi-modal sensing data, 2) grounding of physical symbol representations in real-world wireless systems using causal reasoning and retrieval-augmented generation (RAG), and 3) enabling instructibility from the wireless environment feedback to facilitate dynamic network adaptation thanks to logical and mathematical reasoning facilitated by neuro-symbolic AI. In essence, these properties enable the proposed LMM framework to build universal capabilities that cater to various cross-layer networking tasks and alignment of intents across different domains. Preliminary results from experimental evaluation demonstrate the efficacy of grounding using RAG in LMMs, and showcase the alignment of LMMs with wireless system designs. Furthermore, the enhanced rationale exhibited in the responses to mathematical questions by LMMs, compared to vanilla LLMs, demonstrates the logical and mathematical reasoning capabilities inherent in LMMs. Building on those results, we present a sequel of open questions and challenges for LMMs. We then conclude with a set of recommendations that ignite the path towards LMM-empowered AI-native systems.
AB - Large language models (LLMs) and foundation models have been recently touted as a game-changer for 6 G systems. However, recent efforts on LLMs for wireless networks are limited to a direct application of existing language models that were designed for natural language processing (NLP) applications. To address this challenge and create wireless-centric foundation models, this paper presents a comprehensive vision on how to design universal foundation models that are tailored towards the unique needs of next-generation wireless systems, thereby paving the way towards the deployment of artificial intelligence (AI)-native networks. Diverging from NLP-based foundation models, the proposed framework promotes the design of large multi-modal models (LMMs) fostered by three key capabilities: 1) processing of multi-modal sensing data, 2) grounding of physical symbol representations in real-world wireless systems using causal reasoning and retrieval-augmented generation (RAG), and 3) enabling instructibility from the wireless environment feedback to facilitate dynamic network adaptation thanks to logical and mathematical reasoning facilitated by neuro-symbolic AI. In essence, these properties enable the proposed LMM framework to build universal capabilities that cater to various cross-layer networking tasks and alignment of intents across different domains. Preliminary results from experimental evaluation demonstrate the efficacy of grounding using RAG in LMMs, and showcase the alignment of LMMs with wireless system designs. Furthermore, the enhanced rationale exhibited in the responses to mathematical questions by LMMs, compared to vanilla LLMs, demonstrates the logical and mathematical reasoning capabilities inherent in LMMs. Building on those results, we present a sequel of open questions and challenges for LMMs. We then conclude with a set of recommendations that ignite the path towards LMM-empowered AI-native systems.
KW - AI-native
KW - Alignment
KW - Grounding
KW - Instructibility
KW - Large multi-modal models
KW - Universal foundation model
UR - http://www.scopus.com/inward/record.url?scp=85199115312&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85199115312&partnerID=8YFLogxK
U2 - 10.1109/MNET.2024.3427313
DO - 10.1109/MNET.2024.3427313
M3 - Article
AN - SCOPUS:85199115312
SN - 0890-8044
VL - 38
SP - 10
EP - 20
JO - IEEE Network
JF - IEEE Network
IS - 5
ER -