01

Research

I build machines that recognize and express emotion in voice — audio–language models, controllable text‑to‑speech, and robust learning for human‑centered health.

My research lies at the intersection of affective computing, audio-language modeling, and robust machine learning. I develop methods for emotion-aware representation learning and generation, with a focus on audio large language models, controllable text-to-speech, and speech emotion recognition.

This work brings together representation learning and generative modeling, with applications in computational healthcare and human-centered AI — from expressive synthetic voice to scalable stress sensing for caregivers.

  • D1 Domain-Generalized Audio Sensing Robust speech emotion recognition under distribution shift.
  • D2 Sentiment Steering in LLMs Controllable generation of emotional tone in language models.
  • D3 Audio Stress Detection for ADRD Caregivers Mobile, scalable systems for caregiver support.
02

Prospective Students

Now recruiting Ph.D. students

I’m looking for curious, driven Ph.D. students to help build the next generation of emotionally intelligent machines.

The Galatea Lab works at the meeting point of affective computing, audio–language modeling, and robust machine learning — teaching systems to hear, understand, and express emotion in speech, and making them dependable in the noisy, shifting conditions of the real world. Recent projects span controllable emotional text-to-speech, audio large language models, speech emotion recognition under distribution shift, and on-device sensing for health and caregiving.

William & Mary is a small, research-active public university in Williamsburg, Virginia. Chartered in 1693, it is among the oldest universities in the country and is frequently counted among the nation’s “Public Ivies” — a place where doctoral students get close, hands-on mentorship alongside serious research.

Backgrounds I welcome

  • Computer Science
  • Electrical & Computer Engineering
  • Mathematics
  • Statistics
  • Physics
  • Related quantitative fields

Strong analytical and theoretical training matters most. Prior experience with machine learning, speech, or signal processing is a plus — but it isn’t a prerequisite, and I’m glad to help the right person grow into the field.

Email me A short note on your research interests and your CV is a perfect start. Prospective students should also apply to the W&M Computer Science Ph.D. program.
03

Selected Publications

2026

ICML '26

Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech

Du, H., Shi, J., Lu, S., Zhou, G., Gao, Y. · Proceedings of the International Conference on Machine Learning

InterSpeech '26

Emo-BPO: Emotion Bidirectional Preference Optimization for Diffusion-based Emotional TTS

Shi, J., Du, H., Song, X., Hong, Y. A., Zhang, Y., Gao, Y. · InterSpeech

ACL '26

Confidence-Aware Ranker Ensembles for Robust In-Context Knowledge Editing

Nair, T., Nafee, M. W., Jiang, M., Gao, Y., Chen, H., Zhang, Y. · Findings of the Association for Computational Linguistics

ACL '26

AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling

Shi, J., Du, H., Song, X., Hong, Y. A., Zhang, Y., Gao, Y. · Findings of the Association for Computational Linguistics

ICASSP '26

Emotion-Aligned Generation in Diffusion Text-to-Speech Models via Preference-Guided Optimization

Shi, J., Du, H., He, Y., Hong, Y. A., Gao, Y. · IEEE ICASSP

ICASSP '26

EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognition

Shi, J., Du, H., Hong, Y. A., Gao, Y. · IEEE ICASSP

ICASSP '26

Plug-and-Play Emotion Graphs for Compositional Prompting in Zero-Shot Speech Emotion Recognition

Shi, J., Du, H., Hong, Y. A., Gao, Y. · IEEE ICASSP

ICASSP '26

Hierarchical Convolution Multibranch Transformer for EEG Signals

Mersa, O., Jiang, M., Gao, Y., Li, Q., Zhang, Y. · IEEE ICASSP

ICASSP '26

Pairs, Not Labels: Predicting Protein-Phenotype Associations via Link Prediction

Jiang, M., Chen, Y., Liu, G., Sahu, A., Gao, Y., Zhang, Y. · IEEE ICASSP

2025

EMNLP '25

Role-Guided Annotation and Prototype-Aligned Representation Learning for Historical Literature Sentiment Classification

Du, H., Shi, J., Myerston, J., Lu, S., Zhou, G., Gao, Y. · Findings of EMNLP

InterSpeech '25

Emotion-Aware Audio Large Language Models with Dual Cross-Attention and Context-Aware Instruction Tuning

Du, H., Lu, S., Zhou, G., Gao, Y. · InterSpeech

InterSpeech '25

Contrastive Learning for Speech Emotion Domain Generalization via Soft Prompt Tuning

Shi, J., Zhang, Y., Gao, Y. · InterSpeech

CHASE '25

ElectroMeter: The Practical Electrolyte Measurement System

Clapham, J., Zhou, M., MacDonald, C., Koltermann, K., Gao, Y., Shao, H. · IEEE/ACM CHASE

JMIR '25

Challenges to Recruiting Dementia Caregiving Dyads in Community-Based Settings

Ko, E., Gao, Y., Wang, P., Wijayasingha, L., Wright, K. D., Gordon, K. C., Wang, H., Stankovic, J. A., Rose, K. M. · JMIR Formative Research

2024

TMLR '24

MiddleGAN: Generating Inter-domain Images with GANs

Macdonald, C., Chu, Z., Stankovic, J., Zhou, G., Shao, H., Gao, Y. · Transactions on Machine Learning Research

CHASE '24★ Best Paper Award

Gait-Guard: Turn-aware Freezing of Gait Detection for Non-intrusive Intervention Systems

Koltermann, K., Clapham, J., Blackwell, G., Jung, W., Burnet, E., Gao, Y., Shao, H., et al. · IEEE/ACM CHASE

2023

SmartComp '23★ Best Paper Runner-Up

E-ADDA: Unsupervised Adversarial Domain Adaptation Enhanced by a New Mahalanobis Distance Loss

Gao, Y., Baucom, B., Gordon, K., Rose, K., Wang, H., Stankovic, J. · IEEE SmartComp

2020 – 2021

ACM THealth '21

Emotion Recognition Robust to Indoor Environmental Distortions Using Out-of-Distribution Detection

Gao, Y., Salekin, A., Gordon, K., Rose, K., Wang, H., Stankovic, J. · ACM Transactions on Computing for Healthcare

PerCom '21

Out-of-the-Box Deployment to Support Research on In-Home Care of Alzheimer's Patients

Gao, Y., et al. · IEEE Pervasive Computing

JAN '21

Smarthealth Technology Study Protocol for Dementia Caregiving Relationships

Rose, K. M., et al., Gao, Y. · Journal of Advanced Nursing

SenSys '20

A Monitoring and Recommendation System for In-Home Caregivers (Demo)

Gao, Y., et al. · ACM SenSys

Full publication list on Google Scholar
04

Teaching

  • CSCI 680 Affective Computing Graduate 2026
  • CSCI 680 Deep Transfer Learning Graduate 2025
  • CSCI 516 Fundamentals of Artificial Intelligence & Machine Learning Graduate 2023 · 2024 · 2026
  • CSCI 416 Fundamentals of Artificial Intelligence & Machine Learning Undergraduate 2023 · 2024 · 2026
05

The Galatea Lab

Named for the sculpted figure given voice and feeling, the Galatea Lab studies how machines can hear, understand, and express emotion. I am grateful to work with a wonderful group of students.

Ph.D. Students

  • Jiacheng Shi First-author publications at ICASSP, ACL, InterSpeech 2024–
  • Hongfei Du First-author publications at ICML, ACL, EMNLP, InterSpeech 2024–
  • Hyo Jin Jon William & Mary 2026–
  • Siyuan Zhao William & Mary 2026–

Undergraduate Students

  • Lilly Liu William & Mary → Carnegie Mellon University 2024–26
  • Meave Meyer William & Mary 2026–
  • Ryan Hon University of Maryland 2026–
06

Service & Honors

Editorial & Panels

  • Associate Editor, Elsevier Smart Health2024–
  • Panelist, NIH2024, 2025
  • Panelist, NSF2024
  • Demo & Poster Chair, IEEE/ACM CHASE2024

Reviewer Service

  • NeurIPS2026
  • ICML, AAAI, ICLR2025
  • NeurIPS, AAAI, ICLR, ACM Health2024
  • AAAI, NeurIPS, ICLR2023
  • IEEE TAC, IMWUT/UbiComp2021–22

Honors & Awards

  • Best Paper Award, IEEE/ACM CHASE2024
  • Best Service Award, IEEE/ACM CHASE2024
  • Best Paper Runner-Up, IEEE SmartComp2023
  • Best Poster, CS Symposium, UVA2022
  • Ph.D. Fellowship, CS, UVA2019
  • Academic Excellence Fellowship, UVA2017
  • Honors with High Distinction, UCSD2016

Grants

  • Co-PI, COVA CCI — Agentic AI for Multi-Modal Perception Integrity ($39,199)2026
07

Writing

Before the models, there were stories. Alongside my research I write fiction — the other half of a life spent thinking about voice, feeling, and what it means to be given one.

  • The Tempest's Call To appear · 2026 Novel · Pegasus Publishers, United Kingdom
  • The Celestial Blade To appear · 2026 Novel · Pink Hydra Press, South Africa
  • The Purpose of a Name Short fiction · 2026 Neon Dystopia, United States
  • The Mother of Gaia Short fiction · 2025 Bewildering Stories, Canada