I develop multimodal interactive agents that unify semantic understanding, proactive collaboration, and iterative self-improvement, enabling accessible user experiences and delivering robust, adaptive outcomes across design, education, and beyond.
I am currently a Ph.D. student at CMU HCI Institute, advised by professors Jeff Bigham and Amy Pavel
For more details, check out my: CV | Google Scholar | Linkedin
Preprints
-
AutoPresent: Designing Structured Visuals From Scratch Jiaxin Ge*, Zora Wang*, Xuhui Zhou, Yi-Hao Peng, Sanjay Subramanian, Qinyue Tan, Maarten Sap, Alane Suhr, Daniel Fried, Graham Neubig, Trevor Darrell To Appear at IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2025) project website | paper
-
CodeA11y: Making AI Coding Assistants Useful for Accessible Web Development Peya Mowar, Yi-Hao Peng, Jason Wu, Aaron Steinfeld, Jeffrey Bigham To Appear at ACM Conference on Human Factors in Computing Systems (CHI 2025) project website | paper
-
Towards Bidirectional Human-AI Alignment: A Systematic Review for Clarifications, Framework, and Future Directions Hua Shen et al. (In collaboration between UMich, CMU, Stanford, Google, and Microsoft) To Appear at the Workshop at International Conference on Learning Representations (ICLR 2025) project website | paper
Publications
-
DreamStruct: Understanding Slides and User Interfaces via Synthetic Data Generation Yi-Hao Peng, Faria Huq, Yue Jiang, Jason Wu, Amanda Xin Yue Li, Jeffrey Bigham, Amy Pavel European Conference on Computer Vision (ECCV 2024) project website | paper
-
UIClip: A Data-driven Model for Assessing User Interface Design Jason Wu, Yi-Hao Peng, Amanda Xin Yue Li, Amanda Swearngin, Jeffrey Bigham, Jeffrey Nichols ACM Symposium on User Interface Software and Technology (UIST 2024) project website | paper | video preview
-
Long-form Answers to Visual Questions Asked by Blind and Low Vision People Mina Huh, Fangyuan Xu, Yi-Hao Peng, Chongyan Chen, Hansika Murugu, Danna Gurari, Eunsol Choi, Amy Pavel Conference on Language Modeling (COLM 2024) project website | paper | video preview
-
AdvCAPTCHA: Creating Usable and Secure Audio CAPTCHA with Adversarial Machine Learning Hao-Ping (Hank) Lee, Wei-Lun Kao, Hung-Jui Wang, Ruei-Che Chang, Yi-Hao Peng, Fu-Yin Cherng, Shang-Tse Chen NDSS Symposium on Usable Security and Privacy (USEC 2024) paper | video preview
-
Designing a conversational telepresence robot for homebound older adults Yaxin Hu, Laura Stegner, Yasmine Kotturi, Caroline Zhang, Yi-Hao Peng, Faria Huq, Yuhang Zhao, Jeffrey Bigham, Bilge Mutlu ACM Conference on Designing Interactive Systems (DIS 2024) paper | video preview
-
GenAssist: Making Image Generation Accessible Mina Huh, Yi-Hao Peng, Amy Pavel ACM Symposium on User Interface Software and Technology (UIST 2023) project website | paper | video preview
-
Slide Gestalt: Automatic Structure Extraction in Slide Decks for Non-Visual Access Yi-Hao Peng, Peggy Chi, Anjuli Kannan, Meredith Ringel Morris, Irfan Essa ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
WebUI: A Dataset for Enhancing Visual UI Understanding with Web Semantics Jason Wu, Siyan Wang, Siman Shen, Yi-Hao Peng, Jeffrey Nichols, Jeffrey Bigham ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
Reality Rifts: Wonder-ful Interfaces by Disrupting Perceptual Causality Lung-Pan Cheng, Yi Chen, Yi-Hao Peng, Christian Holtz ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
AVScript: Accessible Video Editing with Audio-Visual Scripts Mina Huh, Saelyne Yang, Yi-Hao Peng, Xiang Anthony Chen, Young-Ho Kim, Amy Pavel ACM Conference on Human Factors in Computing Systems (CHI 2023) paper | video preview
-
Diffscriber: Describing Visual Design Changes to Support Mixed-Ability Presentation Authoring Yi-Hao Peng, Jason Wu, Jeffrey P. Bigham, Amy Pavel ACM Symposium on User Interface Software and Technology (UIST 2022) project website | paper | video preview
-
ImageExplorer: Multi-Layered Touch Exploration to Encourage Skepticism for Imperfect AI-Generated Image Captions Jaewook Lee, Jaylin Herskovitz, Yi-Hao Peng, Anhong Guo ACM Conference on Human Factors in Computing Systems (CHI 2022) paper | video preview | talk
-
Slidecho: Flexible Non-Visual Exploration of Presentation Videos Yi-Hao Peng, Jeffrey P. Bigham, Amy Pavel ACM Conference on Computers and Accessibility (ASSETS 2021) project website | paper | talk
-
Say It All: Feedback for Improving Non-Visual Presentation Accessibility Yi-Hao Peng, JiWoong Jang, Jeffrey P. Bigham, Amy Pavel ACM Conference on Human Factors in Computing Systems (CHI 2021) project website | paper | video preview | talk | slides
-
StrengthGaming: Enabling Dynamic Repetition Tempo in Strength Training-based Exergame Design Sih-Pin Lai, Cheng-An Hsieh, Yu-Hsin Lin, Teepob Harutaipree, Shih-Chin Lin, Yi-Hao Peng, Lung-Pan Cheng, Mike Y. Chen ACM Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2020) paper | video preview | talk | demo recording
-
WalkingVibe: Reducing Virtual Reality Sickness and Improving Realism while Walking in VR using Unobtrusive Head-mounted Vibrotactile Feedback Yi-Hao Peng, Carolyn Yu, Shi-Hong Liu, Chung-Wei Wang, Paul Taele, Neng-Hao Yu, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2020) paper | video preview | talk
-
PersonalTouch: Improving Touchscreen Usability by Personalizing Accessibility Settings based on Individual User’s Touchscreen Interaction Yi-Hao Peng, Muh-Tarng Lin, Yi Chen, Tzu-Chuan Chen, Pin-Sung Ku, Paul Taele, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2019) paper | official apple research kit documentation
-
PhantomLegs: Reducing Virtual Reality Sickness using Head-Worn Haptic Devices Shi-Hong Liu, Neng-Hao Yu, Liwei Chan, Yi-Hao Peng, Wei-Zen Sun, Mike Y. Chen IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2019) paper | video preview | talk
-
PeriText: Utilizing Peripheral Vision for Reading Text on Augmented Reality Smart Glasses Pin-Sung Ku, Yu-Chih Lin, Yi-Hao Peng, Mike Y. Chen IEEE Conference on Virtual Reality and 3D User Interfaces (VR 2019) paper | video preview | talk
-
SpeechBubbles: Enhancing Captioning Experiences for Deaf and Hard-of-Hearing People in Group Conversations Yi-Hao Peng, Ming-Wei Hsu, Paul Taele, Ting-Yu Lin, Po-En Lai, Leon Hsu, Tzu-Chuan Chen, Te-Yen Wu, Yu-An Chen, Hsien-Hui Tang, Mike Y. Chen ACM Conference on Human Factors in Computing Systems (CHI 2018) paper | video preview | talk
Press Coverage
-
Researchers Say Head-mounted Haptics Can Combat Smooth Locomotion Discomfort in VR (covered Project WalkingVibe), Road to VR, May, 2020
-
ResearchKit and CareKit Reimagined (covered Project PersonalTouch), Apple Worldwide Developers Conference — session 217 (02:57), June, 2019
-
The Radical Frontier of Inclusive Design (covered Project SpeechBubbles), Fast Company, May, 2018