Bei Wang Phillips

PDF
Collaborative Research: SCH: Geometry and Topology for Interpretable and Reliable Deep Learning in Medical Imaging

Award Number and Duration

NSF IIS 2205418 (University of Utah)

NSF IIS 2205417 (University of Virginia)

September 1, 2022 to August 31, 2026 (Estimated)

PI and Point of Contact

Bei Wang (Utah PI)
Associate Professor
School of Computing and Scientific Computing and Imaging Institute
University of Utah
beiwang AT sci.utah.edu
homepage

Tom Preston Fletcher (UVA PI)
Associate Professor
Department of Electrical and Computer Engineering and Department of Computer Science
Associate director, Center for Engineering in Medicine
University of Virgina
ptf8v AT virginia.edu
homepage
Companion collaborative project NSF IIS 2205417.

Collaborators

Jonathan C. Garneau, MD (UVA, Senior Personnel)
Assistant Professor
Division of Head and Neck Oncologic and Microvascular Surgery
University of Virgina
homepage

Overview

Deep learning models are being developed for safety-critical applications, such as health care, autonomous vehicles, and security. Their impressive performance has the potential to make profound impacts on human lives. For example, deep neural networks (DNNs) in medical imaging have been shown to have impressive diagnostic capabilities, often near that of expert radiologists. However, deep learning has not made it into standard clinical care, primarily due to a lack of understanding of why a model works and why it fails. The goal of this project is to develop methods for making machine learning models interpretable and reliable, and thus bridge the trust gap to make machine learning translatable to the clinic. This project achieves this goal through investigation of the mathematical foundations -- specifically the geometry and topology -- of DNNs. Based on these mathematical foundations, this project will develop computational tools that will improve the interpretability and reliability of DNNs. The methods developed in this project will be broadly applicable wherever deep learning is used, including health care, security, computer vision, natural language processing, etc.

The power of a deep neural network lies in its hidden layers, where the network learns internal representations of input data. This research project centers around the hypothesis that geometry and topology provide critical tools for analyzing the internal representations of DNNs. The first goal of this project is to develop a rigorous mathematical and algorithmic foundation for describing the geometry and topology of a neural network's internal representations and then design efficient algorithms for geometric and topological computations necessary to explore these spaces. The next aim of this project is to apply these tools to improve the interpretability of deep learning. This will be done by linking a model's internal representation with interpretable and trusted features and by interactive visualization that explores the landscape of a model's internal representation. The next goal of this project focuses on model reliability, where geometry and topology will be used for failure identification, mitigation, and prevention. Finally, this project will test the developed techniques for reliable and interpretable neural networks in a real-world setting to aid expert oncologists in predicting patient outcomes in head and neck cancers, e.g., whether a tumor will metastasize.

Publications and Manuscripts

Year 2 (2023 - 2024)
	Position: Topological Deep Learning is the New Frontier for Relational Learning. Theodore Papamarkou, Tolga Birdal, Michael Bronstein, Gunnar Carlsson, Justin Curry, Yue Gao, Mustafa Hajij, Roland Kwitt, Pietro Lio, Paolo Di Lorenzo, Vasileios Maroulas, Nina Miolane, Farzana Nasrin, Karthikeyan Natesan Ramamurthy, Bastian Rieck, Simone Scardapane, Michael T. Schaub, Petar Velickovic, Bei Wang, Yusu Wang, Guo-Wei Wei, Ghada Zamzmi. Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. PMLR online arXiv:2402.08871
	Interpreting and generalizing deep learning in physics-based problems with functional linear models. Amirhossein Arzani, Lingxiao Yuan, Pania Newell, Bei Wang. Engineering with Computers, 2024. arXiv:2307.04569 DOI:10.1007/s00366-024-01987-z
	In-Context Example Ordering Guided by Label Distributions. Zhichao Xu, Daniel Cohen, Bei Wang, Vivek Srikumar. Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), 2024. arXiv:2402.11447.
	Comparing Mapper Graphs of Artificial Neuron Activations. Youjia Zhou, Helen Jenne, Davis Brown, Madelyn Shapiro, Brett Jefferson, Cliff Joslyn, Gregory Henselman-Petrusek, Brenda Praggastis, Emilie Purvine, Bei Wang. IEEE Workshop on Topological Data Analysis and Visualization (TopoInVis) at IEEE VIS, pages 41-50, 2023. DOI:10.1109/TopoInVis60193.2023.00011
	Visualizing and Analyzing the Topology of Neuron Activations in Deep Adversarial Training. Youjia Zhou, Yi Zhou, Jie Ding, Bei Wang. Topology, Algebra, and Geometry in Machine Learning (TAGML) Workshop at ICML, 2023. OpenReview:Q692Q3dPMe.
	A Lightweight Constrained Generation Alternative for Query focused Summarization. Zhichao Xu, Daniel Cohen. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2023. arXiv:2304.11721 DOI: 10.1145/3539618.3591936
	A Reusable Model-agnostic Framework for Faithfully Explainable Recommendation and System Scrutability. Zhichao Xu, Hansi Zeng, Jintao Tan, Zuohui Fu, Yongfeng Zhang and Qingyao Ai. ACM Transactions on Information Systems (TOIS), 42(1), Pages 1 - 29, 2023. DOI: 10.1145/3605357
	Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression. Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham and Vivek Srikumar. Manuscript, 2024. arXiv:2407.04965
	Measuring Feature Dependency of Neural Networks by Collapsing Feature Dimensions in the Data Manifold. Yinzhu Jin, Matthew B. Dwyer, P. Thomas Fletcher. International Symposium in Biomedical Imaging (ISBI), 2024. arXiv:2404.12341.
	Quantifying Hippocampal Shape Asymmetry in Alzheimer’s Disease Using Optimal Shape Correspondences. Shen Zhu, Ifrah Zawar, Jaideep Kapur, P Thomas Fletcher. International Symposium in Biomedical Imaging (ISBI), 2024. arXiv:2312.01043.
	Learning spatially-continuous fiber orientation functions. Tyler Spears, P Thomas Fletcher. International Symposium in Biomedical Imaging (ISBI), 2024. arXiv:2312.05721.
	Implications of data topology for deep generative models. Yinzhu Jin, Rory McDaniel, N Joseph Tatro, Michael J Catanzaro, Abraham D Smith, Paul Bendich, Matthew B Dwyer, P Thomas Fletcher. Frontiers in Computer Science, vol 6, 2024. Download online
Year 1 (2022 - 2023)
	Experimental Observations of the Topology of Convolutional Neural Network Activations. Emilie Purvine, Davis Brown, Brett Jefferson, Cliff Joslyn, Brenda Praggastis, Archit Rathore, Madelyn Shapiro, Bei Wang, Youjia Zhou. Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI), 2023. DOI: 10.1609/aaai.v37i8.26134 arXiv:2212.00222
	VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations. Archit Rathore, Sunipa Dev, Jeff M. Phillips, Vivek Srikumar, Yan Zheng, Chin-Chia Michael Yeh, Junpeng Wang, Wei Zhang, Bei Wang. ACM Transactions on Interactive Intelligent Systems, 2023. DOI: 10.1145/3604433 arXiv:2104.02797.
	TopoBERT: Exploring the Topology of Fine-Tuned Word Representations. Archit Rathore, Yichu Zhou, Vivek Srikumar, Bei Wang. Information Visualization, 22(3), pages 186-208, 2023. DOI: 10.1177/14738716231168671
	NASDM: Nuclei-Aware Semantic Histopathology Image Generation Using Diffusion Models. Aman Shrivastava and P. Thomas Fletcher. International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), to appear, 2023. arXiv:2303.11477

Presentations, Educational Development and Broader Impacts

Year 2 (2023 - 2024)

Bei Wang Guest Lecture: Biases in Word Embeddings. ScaDS.AI Leipzig (Center for Scalable Data Analytics and Artificial Intelligence), Leipzig, Germany, June 3, 2024.
Bei Wang Invited Talk: Topology of Artificial Neuron Activations in Deep Learning. School of Physical and Mathematical Sciences, Nanyang Technological University, Singapore, Oct 19, 2023.
Tom Preston Fletcher Invited Talk: Applications Riemannian Geometry in Deep Neural Networks. University of Stavanger, Norway, June 2024.
Tom Preston Fletcher Invited Talk: Implicit Neural Networks for Learning Spatially-Continuous Fiber Orientation Functions from Diffusion MRI. NeuroConnect Workshop 2024, Blowing Rock, NC, August 2024.

Year 1 (2022 - 2023)

Bei Wang Invited Talk, International Forum at ChinaVis, July 21, 2023.
Tom Preston Fletcher Invited Talk, Institute for Mathematical and Statistical Innovation (IMSI), Workshop on Object Oriented Data Analysis in Health Sciences, July 10-14, 2023.
Bei Wang Invited Talk, Oxford Applied Topology Seminar, Centre for Topological Data Analysis (Oxford, Liverpool, and Durham), UK, June 16, 2023.
Bei Wang, Session Chairs, full paper session and Young Researchers Forum at the International Symposium on Computational Geometry (SOCG), June 10-15, 2023.
Workshop: Dagstuhl Seminar: Topological Data Analysis and Applications, May 7-12, 2023.
Organizers: Bei Wang, Ulrich Bauer, Vijay Natarajan.
Bei Wang Invited Talk, Colorado State University Topology Seminar, April 18, 2023.
Bei Wang Invited Talk, Northeastern Topology Seminar, April 11, 2023.
Bei Wang Invited Talk, Institute for Mathematical and Statistical Innovation (IMSI), Randomness in Topology and its Applications workshop, March 21, 2023.
Bei Wang Keynote Talk, Machine Learning on Higher-Order Structured data (ML-HOS) Workshop at ICDM 2022. Hypergraph Co-Optimal Transport, November 28, 2022.
Bei Wang Invited Talk, Stochastic Seminar, Department of Mathematics, University of Utah, November 4, 2022.
Bei Wang Invited Talk, Mini Symposium on Statistics and Machine Learning in Topological and Geometric Data Analysis at SIAM Conference on Mathematics of Data Science (MDS22), September 29, 2022.

Students

Aman Shrivistava (CS PhD), University of Virginia.

Yinzhu Jin (CS PhD), University of Virginia.

Zhichao Xu (CS PhD), University of Utah.

Dhruv Meduri (CS PhD), University of Utah.

Acknowledgement

This material is based upon work supported or partially supported by the National Science Foundation.

Any opinions, findings, and conclusions or recommendations expressed in this project are those of author(s) and do not necessarily reflect the views of the National Science Foundation.

Web page last update: September 7, 2024.