Naiyu Yin
Postdoctoral Research Associate
Lehigh University
Research Interests
About
I am a machine learning researcher working at the intersection of causality, Bayesian inference, and the interpretability of large language models. Currently a Postdoctoral Research Associate in the Department of Mathematics at Lehigh University, I build methods that make AI systems more interpretable, robust, and trustworthy.
My research spans several connected threads: causal discovery and inference, causal representation learning for domain generalization and counterfactual image generation, mechanistic interpretability of large language models, and Bayesian inference with uncertainty quantification. More recently, I have been developing foundation models and language models for scientific applications, including causal graph learning and polymer design.
I completed my Ph.D. in Electrical, Computer, and Systems Engineering and an M.S. in Applied Mathematics at Rensselaer Polytechnic Institute, following an M.S. in Electrical and Computer Engineering from Duke University. During my doctoral studies, I collaborated closely with IBM Research across multiple summers, translating research ideas into methods for LLM interpretability, causal discovery, and robust computer vision.
My work has been published at venues including CVPR, AAAI, ECCV, ICPR (Oral, Best Industrial Paper nomination), ICML, IJCAI, and NeurIPS, along with workshops on mechanistic interpretability and machine learning for the physical sciences. I also serve as a reviewer for leading conferences and journals in machine learning, computer vision, and statistics. I am always glad to connect with others working on causal machine learning, interpretability, and foundation models for science. Feel free to reach out.
News
Two papers, "Scalable Circuit Learning for Interpreting Large Language Models" and "Rational Sparse Autoencoder," are accepted to the Mechanistic Interpretability Workshop at ICML 2026!
Our paper "CGU-Bayes: Causal Graph Uncertainty-Guided Bayesian Inference for Domain Generalization" has been accepted to CVPR 2026!
Invited to organize and host the minisymposium "Foundation Models: From Theory to Practice" at SIAM-NNP Annual Meeting!
Paper "Fake It Till You Make It: Multi-Physics Synthesis Breaks the Data Barrier in Chemical Language Models" is accepted to NeurIPS 2025 Machine Learning and the Physical Sciences Workshop.
Invited talk at the 18th U.S. National Congress on Computational Mechanics (USNCCM18).
Our paper "Learning Causal Graphs at Scale: A Foundation Model Approach" is now available on arXiv!
Started as a Postdoctoral Research Associate at Lehigh University.
Paper "Integrating Markov Blanket Discovery into Causal Representation Learning for Domain Generalization" is accepted to ECCV 2024.
Paper "Efficient Nonlinear DAG Learning Under Projection Framework" is accepted to ICPR 2024 with an Oral presentation. Nominated for Best Industrial Paper Award!
Started research internship at IBM Research, Yorktown Heights.
Paper "Effective Causal Discovery under Identifiable Heteroscedastic Noise Model" is accepted to AAAI 2024.
Selected Publications
View All →CGU-Bayes: Causal Graph Uncertainty-Guided Bayesian Inference for Domain Generalization
Naiyu Yin, Hangjing Wang, Yue Yu, Tian Gao, Amit Dhurandhar, Chung-Hao Lee, Qiang Ji
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
A Bayesian framework leveraging causal graph uncertainty for robust domain generalization.
Scalable Circuit Learning for Interpreting Large Language Models
Naiyu Yin, Dennis Wei, Tian Gao, Amit Dhurandhar, Karthikeyan Natesan Ramamurthy, Yue Yu
Mechanistic Interpretability Workshop at ICML
A scalable circuit learning method that uses sparse linear regression over sparse-autoencoder features to recover interpretable circuits, matching the structural accuracy of intervention-based methods at a fraction of the computational cost.
Rational Sparse Autoencoder
Naiyu Yin, Yue Yu
Mechanistic Interpretability Workshop at ICML
Learning Causal Graphs at Scale: A Foundation Model Approach
Naiyu Yin, Tian Gao, Yue Yu
arXiv preprint
A foundation model for causal discovery that uses an attention-based architecture (Attention-DAG) to learn multiple linear SEMs, enabling efficient zero-shot DAG inference with improved accuracy in small-sample regimes.
Fake It Till You Make It: Multi-Physics Synthesis Breaks the Data Barrier in Chemical Language Models
Naiyu Yin, Ning Liu, Brian Y. Lattimer, Jim Lua, Yue Yu
Machine Learning and the Physical Sciences Workshop at NeurIPS
A physics-augmented chemical language model for polymer property prediction and targeted discovery.