Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech
Research
I build machines that recognize and express emotion in voice — audio–language models, controllable text‑to‑speech, and robust learning for human‑centered health.
My research lies at the intersection of affective computing, audio-language modeling, and robust machine learning. I develop methods for emotion-aware representation learning and generation, with a focus on audio large language models, controllable text-to-speech, and speech emotion recognition.
This work brings together representation learning and generative modeling, with applications in computational healthcare and human-centered AI — from expressive synthetic voice to scalable stress sensing for caregivers.
- D1 Domain-Generalized Audio Sensing Robust speech emotion recognition under distribution shift.
- D2 Sentiment Steering in LLMs Controllable generation of emotional tone in language models.
- D3 Audio Stress Detection for ADRD Caregivers Mobile, scalable systems for caregiver support.
Prospective Students
Now recruiting Ph.D. students
I’m looking for curious, driven Ph.D. students to help build the next generation of emotionally intelligent machines.
The Galatea Lab works at the meeting point of affective computing, audio–language modeling, and robust machine learning — teaching systems to hear, understand, and express emotion in speech, and making them dependable in the noisy, shifting conditions of the real world. Recent projects span controllable emotional text-to-speech, audio large language models, speech emotion recognition under distribution shift, and on-device sensing for health and caregiving.
William & Mary is a small, research-active public university in Williamsburg, Virginia. Chartered in 1693, it is among the oldest universities in the country and is frequently counted among the nation’s “Public Ivies” — a place where doctoral students get close, hands-on mentorship alongside serious research.
Backgrounds I welcome
Strong analytical and theoretical training matters most. Prior experience with machine learning, speech, or signal processing is a plus — but it isn’t a prerequisite, and I’m glad to help the right person grow into the field.
Selected Publications
2026
Emo-BPO: Emotion Bidirectional Preference Optimization for Diffusion-based Emotional TTS
Confidence-Aware Ranker Ensembles for Robust In-Context Knowledge Editing
AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling
Emotion-Aligned Generation in Diffusion Text-to-Speech Models via Preference-Guided Optimization
EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognition
Plug-and-Play Emotion Graphs for Compositional Prompting in Zero-Shot Speech Emotion Recognition
Hierarchical Convolution Multibranch Transformer for EEG Signals
Pairs, Not Labels: Predicting Protein-Phenotype Associations via Link Prediction
2025
Role-Guided Annotation and Prototype-Aligned Representation Learning for Historical Literature Sentiment Classification
Emotion-Aware Audio Large Language Models with Dual Cross-Attention and Context-Aware Instruction Tuning
Contrastive Learning for Speech Emotion Domain Generalization via Soft Prompt Tuning
ElectroMeter: The Practical Electrolyte Measurement System
Challenges to Recruiting Dementia Caregiving Dyads in Community-Based Settings
2024
MiddleGAN: Generating Inter-domain Images with GANs
Gait-Guard: Turn-aware Freezing of Gait Detection for Non-intrusive Intervention Systems
2023
E-ADDA: Unsupervised Adversarial Domain Adaptation Enhanced by a New Mahalanobis Distance Loss
2020 – 2021
Emotion Recognition Robust to Indoor Environmental Distortions Using Out-of-Distribution Detection
Out-of-the-Box Deployment to Support Research on In-Home Care of Alzheimer's Patients
Smarthealth Technology Study Protocol for Dementia Caregiving Relationships
A Monitoring and Recommendation System for In-Home Caregivers (Demo)
Teaching
- CSCI 680 Affective Computing Graduate 2026
- CSCI 680 Deep Transfer Learning Graduate 2025
- CSCI 516 Fundamentals of Artificial Intelligence & Machine Learning Graduate 2023 · 2024 · 2026
- CSCI 416 Fundamentals of Artificial Intelligence & Machine Learning Undergraduate 2023 · 2024 · 2026
The Galatea Lab
Named for the sculpted figure given voice and feeling, the Galatea Lab studies how machines can hear, understand, and express emotion. I am grateful to work with a wonderful group of students.
Ph.D. Students
- Jiacheng Shi 2024–
- Hongfei Du 2024–
- Hyo Jin Jon 2026–
- Siyuan Zhao 2026–
Undergraduate Students
- Lilly Liu → Carnegie Mellon University 2024–26
- Meave Meyer 2026–
- Ryan Hon 2026–
Service & Honors
Editorial & Panels
- Associate Editor, Elsevier Smart Health2024–
- Panelist, NIH2024, 2025
- Panelist, NSF2024
- Demo & Poster Chair, IEEE/ACM CHASE2024
Reviewer Service
- NeurIPS2026
- ICML, AAAI, ICLR2025
- NeurIPS, AAAI, ICLR, ACM Health2024
- AAAI, NeurIPS, ICLR2023
- IEEE TAC, IMWUT/UbiComp2021–22
Honors & Awards
- Best Paper Award, IEEE/ACM CHASE2024
- Best Service Award, IEEE/ACM CHASE2024
- Best Paper Runner-Up, IEEE SmartComp2023
- Best Poster, CS Symposium, UVA2022
- Ph.D. Fellowship, CS, UVA2019
- Academic Excellence Fellowship, UVA2017
- Honors with High Distinction, UCSD2016
Grants
- Co-PI, COVA CCI — Agentic AI for Multi-Modal Perception Integrity ($39,199)2026
Writing
Before the models, there were stories. Alongside my research I write fiction — the other half of a life spent thinking about voice, feeling, and what it means to be given one.
- The Tempest's Call To appear · 2026
- The Celestial Blade To appear · 2026
- The Purpose of a Name Short fiction · 2026
- The Mother of Gaia Short fiction · 2025