My research is in Human–Computer Interaction with a focus on interactive and accessible AI agents. I explore how such agents can robustly perceive multimodal context, collaborate with users proactively to preserve user agency, and adapt continually to support creative work in design, education, and beyond.
I am currently a Ph.D. student at CMU HCI Institute, advised by professors Jeff Bigham and Amy Pavel
For more details, check out my: CV | Google Scholar | Linkedin
Updates
-
Ph.D. Application Review (Fall '25): Over the past few years, I have reviewed Ph.D. applications for more than thirty students worldwide. If you would like feedback on your application materials, feel free to email me.
-
Call for Speakers for CMU Accessibility Seminar (Fall '25): We are hosting a seminar series on machine learning for accessibility and related areas at CMU. If you would like to give a talk on campus or online, please fill out the sign‑up form and we will contact you with the details.
Preprints
-
Morae: Proactively Pausing UI Agents for User Choices Yi-Hao Peng, Dingzeyu Li, Jeff Bigham, Amy Pavel To Appear at ACM Symposium on User Interface Software and Technology (UIST 2025) project website | paper
-
StepWrite: Adaptive Planning for Speech-Driven Text Generation Hamza El Alaoui, Atieh Taheri, Yi-Hao Peng, Jeff Bigham To Appear at ACM Symposium on User Interface Software and Technology (UIST 2025) project website | paper
-
Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions Hua Shen, Tiffany Knearem, Reshmi Ghosh, Kenan Alkiek*, Kundan Krishna*, Yachuan Liu*, Ziqiao Ma*, Savvas Petridis*, Yi-Hao Peng*, Qiwei Li*, Sushrita Rakshit*, Chenglei Si*, Yutong Xie*, Jeff Bigham, Frank Bentley, Joyce Chai, Zachary Lipton, Qiaozhu Mei, Rada Mihalcea, Michael Terry, Diyi Yang, Meredith Ringel Morris, Paul Resnick, David Jurgens To Appear at Advances in Neural Information Processing Systems (NeurIPS 2025), Position Paper Track project website | paper
-
Tempura: Temporal Event Masked Prediction and Understanding for Reasoning in Action Jen-Hao Cheng, Huapeng Zhou, Vivian Wang, Yi-Hao Peng, Hsiang-Wei Huang, Wenhao Chai, Huayu Wang, Hou-I Liu, Kuang-Ming Chen, Cheng-Yen Yang, Yipeng Wang, Yi-Ling Chen, Vibhav Vineet, Qin Cai, Jenq-Neng Hwang arXiv.2025.05 (under submission) project website | paper
Publications
-
CodeA11y: Making AI Coding Assistants Useful for Accessible Web Development Peya Mowar, Yi-Hao Peng, Jason Wu, Aaron Steinfeld, Jeff Bigham ACM Conference on Human Factors in Computing Systems (CHI 2025) project website | paper
-
AutoPresent: Designing Structured Visuals From Scratch Jiaxin Ge*, Zora Wang*, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten Sap, Alane Suhr, Daniel Fried, Graham Neubig, Trevor Darrell IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025) project website | paper
-
DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation Yi-Hao Peng, Faria Huq, Yue Jiang, Jason Wu, Amanda Xin Yue Li, Jeff Bigham, Amy Pavel European Conference on Computer Vision (ECCV 2024) project website | paper
-
UIClip: A Data-driven Model for Assessing User Interface Design Jason Wu, Yi-Hao Peng, Amanda Xin Yue Li, Amanda Swearngin, Jeff Bigham, Jeff Nichols ACM Symposium on User Interface Software and Technology (UIST 2024) project website | paper | video preview
-
Long-form Answers to Visual Questions Asked by Blind and Low Vision People Mina Huh, Fangyuan Xu, Yi-Hao Peng, Chongyan Chen, Hansika Murugu, Danna Gurari, Eunsol Choi, Amy Pavel Conference on Language Modeling (COLM 2024) project website | paper | video preview
-
AdvCAPTCHA: Creating Usable and Secure Audio CAPTCHA with Adversarial Machine Learning Hao-Ping (Hank) Lee, Wei-Lun Kao, Hung-Jui Wang, Ruei-Che Chang, Yi-Hao Peng, Fu-Yin Cherng, Shang-Tse Chen NDSS Symposium on Usable Security and Privacy (USEC 2024) paper | video preview
-
Designing a conversational telepresence robot for homebound older adults Yaxin Hu, Laura Stegner, Yasmine Kotturi, Caroline Zhang, Yi-Hao Peng, Faria Huq, Yuhang Zhao, Jeff Bigham, Bilge Mutlu ACM Conference on Designing Interactive Systems (DIS 2024) paper | video preview
-
GenAssist: Making Image Generation Accessible Mina Huh, Yi-Hao Peng, Amy Pavel ACM Symposium on User Interface Software and Technology (UIST 2023) project website | paper | video preview
-
Slide Gestalt: Automatic Structure Extraction in Slide Decks for Non-Visual Access Yi-Hao Peng, Peggy Chi, Anjuli Kannan, Meredith Ringel Morris, Irfan Essa ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics Jason Wu, Siyan Wang, Siman Shen, Yi-Hao Peng, Jeff Nichols, Jeff Bigham ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
Reality Rifts: Wonder-ful Interfaces by Disrupting Perceptual Causality Lung-Pan Cheng, Yi Chen, Yi-Hao Peng, Christian Holtz ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
AVScript: Accessible Video Editing with Audio-Visual Scripts Mina Huh, Saelyne Yang, Yi-Hao Peng, Xiang Anthony Chen, Young-Ho Kim, Amy Pavel ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
Diffscriber: Describing Visual Design Changes to Support Mixed-Ability Presentation Authoring Yi-Hao Peng, Jason Wu, Jeff Bigham, Amy Pavel ACM Symposium on User Interface Software and Technology (UIST 2022) project website | paper | video preview
-
ImageExplorer: Multi-Layered Touch Exploration to Encourage Skepticism for Imperfect AI-Generated Image Captions Jaewook Lee, Jaylin Herskovitz, Yi-Hao Peng, Anhong Guo ACM Conference on Human Factors in Computing Systems (CHI 2022) paper | video preview | talk
-
Slidecho: Flexible Non-Visual Exploration of Presentation Videos Yi-Hao Peng, Jeff Bigham, Amy Pavel ACM Conference on Computers and Accessibility (ASSETS 2021) project website | paper | talk
-
Say It All: Feedback for Improving Non-Visual Presentation Accessibility Yi-Hao Peng, JiWoong Jang, Jeff Bigham, Amy Pavel ACM Conference on Human Factors in Computing Systems (CHI 2021) project website | paper | video preview | talk | slides
-
StrengthGaming: Enabling Dynamic Repetition Tempo in Strength Training-based Exergame Design Sih-Pin Lai, Cheng-An Hsieh, Yu-Hsin Lin, Teepob Harutaipree, Shih-Chin Lin, Yi-Hao Peng, Lung-Pan Cheng, Mike Y. Chen ACM Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2020) paper | video preview | talk | demo recording
-
WalkingVibe: Reducing Virtual Reality Sickness and Improving Realism while Walking in VR using Unobtrusive Head-mounted Vibrotactile Feedback Yi-Hao Peng, Carolyn Yu, Shi-Hong Liu, Chung-Wei Wang, Paul Taele, Neng-Hao Yu, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2020) paper | video preview | talk
-
PersonalTouch: Improving Touchscreen Usability by Personalizing Accessibility Settings based on Individual User’s Touchscreen Interaction Yi-Hao Peng, Muh-Tarng Lin, Yi Chen, Tzu-Chuan Chen, Pin-Sung Ku, Paul Taele, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2019) paper | official apple research kit documentation
-
PhantomLegs: Reducing Virtual Reality Sickness using Head-Worn Haptic Devices Shi-Hong Liu, Neng-Hao Yu, Liwei Chan, Yi-Hao Peng, Wei-Zen Sun, Mike Y. Chen IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2019) paper | video preview | talk
-
PeriText: Utilizing Peripheral Vision for Reading Text on Augmented Reality Smart Glasses Pin-Sung Ku, Yu-Chih Lin, Yi-Hao Peng, Mike Y. Chen IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2019) paper | video preview | talk
-
SpeechBubbles: Enhancing Captioning Experiences for Deaf and Hard-of-Hearing People in Group Conversations Yi-Hao Peng, Ming-Wei Hsu, Paul Taele, Ting-Yu Lin, Po-En Lai, Leon Hsu, Tzu-Chuan Chen, Te-Yen Wu, Yu-An Chen, Hsien-Hui Tang, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2018) paper | video preview | talk
Press Coverage
-
Researchers Say Head-mounted Haptics Can Combat Smooth Locomotion Discomfort in VR (covered Project WalkingVibe), Road to VR, May, 2020
-
ResearchKit and CareKit Reimagined (covered Project PersonalTouch), Apple Worldwide Developers Conference — session 217 (02:57), June, 2019
-
The Radical Frontier of Inclusive Design (covered Project SpeechBubbles), Fast Company, May, 2018