TY - GEN
T1 - ConceptX
T2 - 37th AAAI Conference on Artificial Intelligence, AAAI 2023
AU - Alam, Firoj
AU - Dalvi, Fahim
AU - Durrani, Nadir
AU - Sajjad, Hassan
AU - Khan, Abdul Rafae
AU - Xu, Jia
N1 - Publisher Copyright:
Copyright © 2023, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved.
PY - 2023/6/27
Y1 - 2023/6/27
N2 - The opacity of deep neural networks remains a challenge in deploying solutions where explanation is as important as precision. We present ConceptX, a human-in-the-loop framework for interpreting and annotating latent representational space in pre-trained Language Models (pLMs). We use an unsupervised method to discover concepts learned in these models and enable a graphical interface for humans to generate explanations for the concepts. To facilitate the process, we provide auto-annotations of the concepts (based on traditional linguistic ontologies). Such annotations enable development of a linguistic resource that directly represents latent concepts learned within deep NLP models. These include not just traditional linguistic concepts, but also task-specific or sensitive concepts (words grouped based on gender or religious connotation) that helps the annotators to mark bias in the model. The framework consists of two parts (i) concept discovery and (ii) annotation platform.
AB - The opacity of deep neural networks remains a challenge in deploying solutions where explanation is as important as precision. We present ConceptX, a human-in-the-loop framework for interpreting and annotating latent representational space in pre-trained Language Models (pLMs). We use an unsupervised method to discover concepts learned in these models and enable a graphical interface for humans to generate explanations for the concepts. To facilitate the process, we provide auto-annotations of the concepts (based on traditional linguistic ontologies). Such annotations enable development of a linguistic resource that directly represents latent concepts learned within deep NLP models. These include not just traditional linguistic concepts, but also task-specific or sensitive concepts (words grouped based on gender or religious connotation) that helps the annotators to mark bias in the model. The framework consists of two parts (i) concept discovery and (ii) annotation platform.
UR - http://www.scopus.com/inward/record.url?scp=85159861519&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85159861519&partnerID=8YFLogxK
U2 - 10.1609/aaai.v37i13.27057
DO - 10.1609/aaai.v37i13.27057
M3 - Conference contribution
AN - SCOPUS:85159861519
T3 - Proceedings of the 37th AAAI Conference on Artificial Intelligence, AAAI 2023
SP - 16395
EP - 16397
BT - AAAI-23 Special Programs, IAAI-23, EAAI-23, Student Papers and Demonstrations
A2 - Williams, Brian
A2 - Chen, Yiling
A2 - Neville, Jennifer
Y2 - 7 February 2023 through 14 February 2023
ER -