Biography
Senior Research Scientist in generative audio and voice LLMs with 7+ years of experience from research to production. I have led end-to-end development of expressive TTS systems and contributed to full-duplex speech systems across data curation, codec and tokenization design, large-scale multi-GPU training, and deployment. My work focuses on turning advanced speech and audio methods into product-ready AI systems for voice generation, multimodal interaction, and intelligent audio understanding.
Download my resumé .
Interests
- Generative Audio and Voice LLMs
- Multimodal Foundation Models
- Speech and Singing Voice Generation
- Speaker Recognition and Antispoofing
- Audio Separation and Enhancement
Education
PhD in Informatics, 2024
National Institute of Informatics & SOKENDAI
MEng in Electrical Engineering and Information System, 2020
The University of Tokyo
BEng in Measurement and Control Technology and Instruments, 2016
Tianjin University
News
- 2026.01: Released A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation on arXiv.
- 2026.01: DrivingScene and PAGS accepted by ICASSP 2026.
- 2025.12: Released Towards Interactive Intelligence for Digital Humans on arXiv.
- 2025.10: Critical Information Only accepted by IEEE TDSC.
- 2025.01: SonicSim accepted by ICLR 2025.
- 2025.01: A Benchmark for Multi-Speaker Anonymization accepted by IEEE TIFS.
- 2024.12: InstructSing accepted by SLT 2024.
- 2024.12: Spoofing-Aware Speaker Verification Robust Against Domain and Channel Mismatches accepted by SLT 2024.
- 2024.03: Released HAM-TTS on arXiv.
Publications
Quickly discover relevant content by filtering publications.
(2026).
A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation.
In arXiv.
(2026).
DrivingScene: A Multi-Task Online Feed-Forward 3D Gaussian Splatting Method for Dynamic Driving Scenes.
Accepted by ICASSP 2026.
(2026).
PAGS: Priority-Adaptive Gaussian Splatting for Dynamic Driving Scenes.
Accepted by ICASSP 2026.
(2025).
Towards Interactive Intelligence for Digital Humans.
In arXiv.
(2025).
Critical Information Only: A Content Privacy-Preserving Framework for Detecting Audio Deepfakes.
In IEEE TDSC.
(2025).
SonicSim: A Customizable Simulation Platform for Speech Processing in Moving Sound Source Scenarios.
Accepted by ICLR 2025.
(2025).
A Benchmark for Multi-Speaker Anonymization.
In IEEE TIFS.
(2024).
InstructSing: High-Fidelity Singing Voice Generation via Instructing Yourself.
In SLT 2024.
(2024).
Spoofing-Aware Speaker Verification Robust Against Domain and Channel Mismatches.
In SLT 2024.
(2024).
HAM-TTS: Hierarchical Acoustic Modeling for Token-Based Zero-Shot Text-to-Speech with Model and Data Scaling.
In ArXiv.
(2024).
Joint Speaker Encoder and Neural Back-end Model for Fully End-to-End Automatic Speaker Verification with Multiple Enrollment Utterances.
In Computer Speech & Language.
(2023).
CrossSinger: A Cross-Lingual Multi-Singer High-Fidelity Singing Voice Synthesizer Trained on Monolingual Singers.
In ASRU 2023.
(2023).
HiFi-WaveGAN: Generative Adversarial Network with Auxiliary Spectrogram-Phase Loss for High-Fidelity Singing Voice Generation.
In ISNN 2024.
(2023).
Improving Generalization Ability of Countermeasures for New Mismatch Scenario by Combining Multiple Advanced Regularization Terms.
In Interspeech 2023.
(2023).
Xiaoicesing 2: A High-Fidelity Singing Voice Synthesizer Based on Generative Adversarial Network.
In Interspeech 2023.
(2023).
Beyond Universal Transformer: block reusing with adaptor in Transformer for automatic speech recognition.
In ISNN 2024.
(2023).
Cross-Modal Audio-Visual Co-Learning for Text-Independent Speaker Verification.
In ICASSP 2023.
(2023).
SSI-Net: A Multi-Stage Speech Signal Improvement System for ICASSP 2023 SSI Challenge.
In ICASSP 2023.
(2022).
Deep Spectro-temporal Artifacts for Detecting Synthesized Speech.
In DDAM 2022 Workshop.
(2022).
Data Augmentation Using McAdams-Coefficient-Based Speaker Anonymization for Fake Audio Detection.
In Interspeech 2022.
(2022).
Spoofing-Aware Attention based ASV Back-end with Multiple Enrollment Utterances and a Sampling Strategy for the SASV Challenge 2022.
In Interspeech 2022.
(2022).
Attention Back-end for Automatic Speaker Verification with Multiple Enrollment Utterances.
In ICASSP 2022.
Skills
Languages and Tools
Python, C++, Shell, Git, MySQL
Deep Learning
PyTorch, PyTorch Lightning, Hugging Face
Speech Toolkits
SpeechBrain, WeNet, WeSpeaker, Kaldi, ESPnet
Generative Audio
Expressive TTS, codec and tokenizer design, voice LLMs
Multimodal AI
Audio-language modeling, full-duplex speech systems
Communication
Chinese, English, Japanese (N2-125)
Activities
Reviewer Service
- Conferences: NeurIPS, ICLR, ICML, AISTATS, ICASSP, ICME, INTERSPEECH
- Journals: IEEE Open Journal of Signal Processing
Academic Activities
- Research Assistant at Yamagishi Lab, National Institute of Informatics
- ICASSP 2022 Oral Presentation
- Interspeech 2022 Oral Presentation
- Interspeech 2023 Poster Presentation
- Organizing Committee, Joint Workshop of VoicePersonae and ASVspoof 2023
- SLT 2024, SVDD Challenge (Invited Talk): Shared insights on singing voice generation as a guest speaker
Competitions
- 4th/77 place, VoxCeleb Speaker Recognition Challenge 2019
- 2nd/110 place, Zhijiang Cup Speech Recognition for Conversational Scenario 2021
- 4th/42 place, Audio Deep Synthesis Detection Challenge 2022 Track 1
- 5th/27 place, Audio Deep Synthesis Detection Challenge 2022 Track 2
Open Source
- WeSpeaker contributor
- ASV-Subtools contributor
- HIVE dataset contributor
Experience
Senior AI Researcher
Shanda AI Research Tokyo
Lead R&D of expressive TTS and full-stack voice-agent technologies for avatar and game products.
- Built KodamaTTS from scratch on a Qwen-based foundation model for virtual human and gaming applications
- Covered data curation, codec design, multi-node multi-GPU training, and evaluation in one pipeline
- Achieved sub-1 kbps codec quality with strong objective performance and reached top benchmark rankings in 3 languages
- Served as Co-PI on a joint project with Tsinghua University on cocktail-party speech interaction and audio separation
Multimodal Generative AI Researcher
Developed voice generation systems for Li Auto smart-space products and contributed to the multimodal foundation model MindGPT-4o.
- Proposed the GFSQ tokenizer for GPT-SoVITS to improve codebook utilization and decoding quality
- Trained a multi-timbre, multi-style voice generation model for in-car voice-blog scenarios in production
- Built synthetic-data workflows to scale accents, dialects, languages, emotions, and scenarios
- Led data production and audio-head pretraining and post-training for a full-duplex conversational model
Avatar Research Intern
Focused on high-fidelity 48kHz singing voice generation in collaboration with research and engineering teams.
- Upgraded XiaoiceSing to XiaoiceSing2 with adversarial training and achieved near-human MOS
- Developed HiFi-WaveGAN with a pulse-sequence design for stronger 48kHz singing synthesis quality
- Built CrossSinger for cross-lingual multi-singer SVS in English, Japanese, and Chinese
- Improved training efficiency with InstructSing and explored hierarchical acoustic modeling for voice LMs
Speech Recognition Researcher
Developed speech AI systems for Taobao Live compliance and broadcaster-risk control.
- Built a large-scale speaker recognition system for broadcaster identity verification in livestream scenarios
- Developed a spoken-term detection pipeline for policy-sensitive and illegal word monitoring
- Researched self-supervised speech representations and implemented an ESPnet-based end-to-end ASR system