I'm a Staff Research Scientist at Google DeepMind, conducting research and product launches. I develop interactive systems that support users' creativity activities, including video creation, image sharing, and programming. I received my Ph.D. in Computer Science from UC Berkeley and an M.S. from the MIT Media Lab. My research has received a Best Paper Award at ACM CHI, a Google PhD Fellowship in Human-Computer Interaction, a Berkeley Fellowship for Graduate Study, and an MIT Media Lab Fellowship. I have published in top HCI venues and served on program committees, including CHI and UIST.
Outside Google, I'm a Visiting Associate Professor at National Taiwan University's Department of Computer Science and Information Engineering, where I teach a graduate level course.
My main research projects include:
Creativity and Navigation Tools





Automatic Instructional Video Creation from a Markdown-Formatted Tutorial
ACM Symposium on User Interface Software and Technology (UIST 2021)
HowToCut is an automatic approach that converts a Markdown-formatted tutorial into an instructional video by presenting the visual instructions with a synthesized voiceover.

HelpViz: Automatic Generation of Contextual Visual Mobile Tutorials from Text-Based Instructions
ACM Symposium on User Interface Software and Technology (UIST 2021)
HelpViz is a tool for generating contextual visual mobile tutorials from text-based instructions that are abundant on the web.

Automatic Non-Linear Video Editing Transfer
AI for Content Creation (AICC) Workshop at CVPR 2021
We propose Computer Vision based techniques that extract editing styles in a source video and apply the edits to matched footage for video creation.
[Paper]

Automatic Generation of Two-Level Hierarchical Tutorials from Instructional Makeup Videos
ACM Conference on Human Factors in Computing Systems (CHI 2021)
We present a multi-modal approach for automatically generating hierarchical tutorials from instructional makeup videos.

URL2Video: Automatic Video Creation From a Web Page
ACM Symposium on User Interface Software and Technology (UIST 2020)
URL2Video captures quality materials and design styles extracted from a web page, including fonts, colors, and layouts. Using constraint programming, URL2Video's design engine organizes the visual assets into a sequence of shots and renders to a video with user-specified aspect ratio and duration.
[Paper] [Google AI Blog Post] [Video] [DOI]

Interactive Visual Description of a Web Page for Smart Speakers
Conversational User Interface Workshop at ACM Conference on Human Factors in Computing Systems (CHI 2020)

Crowdsourcing Images for Global Diversity
ACM Conference on Mobile Human-Computer Interaction (MobileHCI 2019)

DemoDraw: Authoring Illustrations of Human Movements by Iterative Physical Demonstration
ACM Symposium on User Interface Software and Technology (UIST 2016)
DemoDraw is a multi-modal approach to generate illustrations as the user physically demonstrates the movements.
[Paper (PDF)] [Video] [DOI]

DemoWiz: Re-Performing Software Demonstrations for a Live Presentation
ACM Conference on Human Factors in Computing Systems (CHI 2014)
DemoWiz is a system with a refined workflow that helps presenters capture software demonstrations, edit and rehearse them, and re-perform them for an engaging live presentation.
[Paper (PDF)] [Video] [DOI]

DemoCut: Generating Concise Instructional Videos for Physical Demonstrations
ACM Symposium on User Interface Software and Technology (UIST 2013)
DemoCut is a semi-automatic video editing system that improves the quality of amateur instructional videos for physical tasks.
[Paper (PDF)] [Video] [DOI]

Kinectograph: Body-Tracking Camera Control for Demonstration Videos
ACM Conference on Human Factors in Computing Systems (CHI 2013) Poster
Kinectograph is a recording device that automatically pans and tilts to follow specific body parts, e.g., hands, of a user in a video.
[Paper (PDF)] [Video] [DOI]

MixT: Automatic Generation of Step-by-Step Mixed Media Tutorials
ACM Symposium on User Interface Software and Technology (UIST 2012)
MixT is a system that automatically generates step-by-step mixed media tutorials from user demonstrations.
[Paper (PDF)] [Video] [DOI]

Raconteur: From Chat to Stories
ACM Conference on Intelligent User Interfaces (IUI 2011)
ACM Conference on Human Factors in Computing Systems (CHI 2011)
Raconteur is a system for conversational storytelling that encourages people to make coherent points. It performs natural language processing in real-time on a text chat between a storyteller and a viewer and recommends appropriate media items from a library.
[IUI Paper (PDF)] [DOI] [CHI Paper (PDF)] [DOI] [Video]
Programming Tools

Doppio: Tracking UI Flows and Code Changes for App Development
ACM Conference on Human Factors in Computing Systems (CHI 2018)
[Paper (PDF)] [DOI]

DemoScript: Enhancing Cross-Device Interaction Scripting with Interactive Illustrations
ACM Conference on Human Factors in Computing Systems (CHI 2016)
Best Paper Award
[Paper (PDF)] [Video] [DOI]

Weave: Scripting Cross-Device Wearable Interaction
ACM Conference on Human Factors in Computing Systems (CHI 2015)
[Paper (PDF)] [Video] [DOI]
Ubiquitous Computing

Enabling Calorie-Aware Cooking in a Smart Kitchen
ACM Conference on Persuasive Technology (Persuasive 2008)
[Paper (PDF)] [Video] [Cooking Video] [DOI] [DOI]

Ubicomp Technologies for Play-Based Occupational Therapy
IEEE Pervasive Computing Magazine (2009)
[Paper (PDF)] [DOI]

Playful Toothbrush: Ubicomp Technology for Teaching Tooth Brushing to Kindergarten Children
ACM Conference on Human Factors in Computing Systems (CHI 2008)
[Paper (PDF)] [DOI]

Prototypes

Burn Your Memory Away: One-Time Use Video Capture and Storage Device to Encourage Memory Appreciation
ACM Conference on Human Factors in Computing Systems (CHI 2009) Alt.chi
[Paper (PDF)] [Video] [DOI]

Stress OutSourced: A Haptic Social Network via Crowdsourcing
ACM Conference on Human Factors in Computing Systems (CHI 2009) Alt.chi
[Paper (PDF)] [Video] [DOI]


Designing Interactive Narrative for Children
Electronic Literature Organization (2010)