Loading...

Soundability Lab

Transforming the Human Experience of Sound

About us

We are a Human-Computer Interaction (HCI) research lab in the Computer Science and Engineering department at the University of Michigan. We explore generative AI for audio-focused applications in accessibility, healthcare, and entertainment. We don't just build models—we show you what they can do.

Our work has significant real-world impact. Our publicly released SoundWatch app has been downloaded over 4,000 times. Our indoor navigation system for visually impaired visitors has been used over 100,000 times in public museums in India. Our accessible commuication systems are being deployed at Michigan Medicine. A feature we developed to help people with paralysis control mobile phones is now built into all iPhones worldwide, and our work has directly influenced real-time captioning products at Google.

Recognized globally, our research has earned multiple paper awards at premier HCI venues, has been featured in leading media outlets (e.g., CNN, Forbes, New Scientist), and is included in academic curricula wordwide. Explore our work below—and reach out if you're interested in working with us.

Recent News

Apr 15: We have three new incoming PhD students in Fall 2025. Welcome!
Mar 16: Our work on improving communication in the operating rooms has been accepted to the journal of ORL Head and Neck Nursing!
Feb 22: Our initial work on using runtime generative tools to improve accessibility of virtual 3D scenes as been accepted to CHI 20205 LBW!.
Feb 19: Our initial work on enchancing communication in high-noise operating rooms has been accepted as a poster at Collaborating Across Borders IX!
Jan 16: Our proposal on dizziness diagnosis was approved for funding from William Demant Foundation! See news article.

Our Team

Headshot of Dhruv Jain
Dhruv "DJ" Jain

Dhruv "DJ" Jain

Assistant Professor, Computer Science & Engineering (Lab head)
Headshot of Xinyun Cao
Xinyun Cao

Xinyun Cao

PhD Student, Computer Science & Engineering
Headshot of Jeremy Huang
Jeremy Huang

Jeremy Huang

PhD Student, Computer Science & Engineering
Headshot of Alexander Wang
Alexander Wang

Alexander Wang

Visiting Researcher, Computer Science & Engineering
Headshot of Liang-Yuan Wu
Liang-Yuan Wu

Liang-Yuan Wu

MS Student, Computer Science & Engineering
Headshot of Hriday Chhabria
Hriday Chhabria

Hriday Chhabria

Undergraduate Student, Computer Science & Engineering
Headshot of Hanlong Liu
Hanlong Liu

Hanlong Liu

Undergraduate Student, Computer Science & Engineering
Headshot of Yuni Park
Yuni Park

Yuni Park

Undergraduate Research Assistant, Computer Science & Engineering
Headshot of Andy Jin
Andy Jin

Andy Jin

Undergraduate Student, Computer Science & Engineering
Headshot of Rue-Chei Chang
Rue-Chei Chang

Rue-Chei Chang

PhD Student, Computer Science & Engineering
Headshot of Anhong Guo
Anhong Guo

Anhong Guo

Assistant Professor, Computer Science & Engineering (Collaborator)
Headshot of Xinyue Chen
Xinyue Chen

Xinyue Chen

PhD Student, Computer Science & Engineering
Headshot of Xu Wang
Xu Wang

Xu Wang

Assistant Professor, Computer Science & Engineering (Collaborator)
Headshot of Elijah Bouma-Sims
Elijah Bouma-Sims

Elijah Bouma-Sims

PhD Student, Carnegie Mellon University (Collaborator)
Headshot of Lorrie Faith Cranor
Lorrie Cranor

Lorrie Cranor

Professor, Carnegie Mellon University (Collaborator)
Headshot of Michael M. McKee
Michael M. McKee

Michael M. McKee

Associate Professor, Michigan Medicine (Collaborator)

Alumni

Headshot of Reyna Wood
Reyna Wood

Wren "Reyna" Wood

Undergraduate Student, Computer Science & Engineering
Headshot of Emily Tsai
Emily Tsai

Emily Tsai

Masters Student, School of Information
Headshot of Mansanjam Kaur
Mansanjam Kaur

Mansanjam Kaur

Masters Student, School of Information
Headshot of Andrew Dailey
Andrew Dailey

Andrew Dailey

Undergraduate Student, Computer Science & Engineering

Publications

We publish our research work in the most prestigious human-computer interaction and accessibility venues including CHI, UIST, and ASSETS. Nine of our articles have been honored with awards.

A snapshot of a CART real-time captioning being broadcasted on an open screen.
AWARD

CARTGPT

ASSETS 2024 (Poster): PAPER
A user is wearing a smartwatch in front of water running down a sink. The smartwatch displays the identified sound as 'water running' with a classification confidence of 83%.
AWARD

SoundWatch Field Study

Real-World Feasibility of Sound Recognition
(Best paper honorable mention)
ASSETS 2023: PAPER | CODE
A close up shot of a person attending a 10-person video conference on a laptop.
AWARD

Classes Taught by DJ

EECS 495: Accessible Computing

This upper-level undergraduate class serves as an introduction to accessibility for undergraduate studdents and uses a curriculum designed by Professor Dhruv Jain. Students learn essential concepts related to accessibiity, disability theory, and user-centric design, and contribute to a studio-style team project in collaboration with clients with a disability and relevant stakeholders we recruit. This intense 14-week class requires working in teams to lead a full scale end-to-end accessibility project from its conceptualization, to design, to implementation, and evaluation. The goal is to reach a level of proficiency comparable to that of a well-launched employee team in a computing industry. Often, projects terminate in real-world deployments and app releases.

Read more →

EECS 598: Advanced Accessibility

This graduate-level class focuses on advances topics in accessibility including disabilty theory, user-research, and their impact on technology. Includes guest lectures by esteemed researchers and practioners in the field of accessibility.

Read more →

Talks

Sound Sensing for Deaf and Hard of Hearing Users

Navigating Graduate School with a Disability

Deep Learning for Sound Awareness on SmartWatches

Field Study of a Tactile Sound Awareness Device

First slide of the talk. A scene of a kitchen in the background with the talk title: Field Deployment of a Smarthome Sound Awareness System for Deaf and Hard of Hearing Users

Field Deployment of a In-Home Sound Awareness System

First slide of the talk. Shows DJ riding on a camel in a desert. The title of the talk reads: Autoethography of a Hard of Hearing Traveler

Autoethnography of a hard of hearing traveler

First slide of the talk. A person claps in front of a tablet interface that visaulizes the clapping sound using a pulsating bubble. The title reads: Exploring Sound Awareness in the Home for People who are Deaf or Hard of Hearing

Exploring sound awareness in the home

First slide of the talk with an image of an ear doning a hearing aid. The title reads: Deaf and Hard of Hearing Individuals' Preferences for Wearable and Mobile Sound Awareness Technologies​

Online Survey of Wearable Sound Awareness

First slide of the talk showing a person walking and talking with another person. The first person is wearing a HoloLens which shows ​real-time captions in Augmented Reality. Title is Towards Accessible Conversations in a Mobile Context for People who are Deaf and Hard of Hearing.

Towards accessible conversations in a mobile context

First slide of the talk showing a rocky beach with waves crashing over the beach. Talk title reads: Immersive Scuba Diving Simulator Using Virtual Reality

Immersive scuba diving simulator using virtual reality​

First slide of the talk showing a round table conversation with a person wearing a Google Glass. The directions of the active speakers in the conversation are visualized as arrows on the Glass. Talk title is Head-Mounted Display Visualizations to Support Sound Awareness for the Deaf and Hard of Hearing.

HMD Visualizations to Support Sound Awareness

Videos

Lab Openings

Prospective PhD students: Our lab has openings for upto three PhD students (beginning Fall 2025) in two areas, (1) data science and AI for acoustics and/or hearing health and (2) AR/VR interaction design for sound accessibility. Please see our research focus. If you believe you are the right fit, apply to the UMich CSE PhD program and email Prof. DJ at profdj [at] umich [dot] edu with: (1) a brief description of yourself and your skillset, supported by relevant prior experience, (2) some examples of projects you'd like to pursue in your PhD, and (3) your CV. We look forward to hearing from you!

Undergraduates/Masters students: Please complete this online intake form and we will get back to you when we have openings.

Potential postdocs: We are recruiting a postdoc in the area of HCI/accessibility with starting date of your choice this year (2025). If interested, please email Prof. DJ with your research interests, a paragraph explaining your dissertation, and your CV as well as apply to the official posting here.