Biography
Senior Research Scientist at Fish Audio working on generative audio and realtime voice intelligence, with 7+ years of experience from research to production. My work spans full-duplex spoken dialogue, realtime speech and voice-agent models, hyper-realistic and highly expressive TTS, music generation, and the full pipeline from data curation and codec/tokenization design to large-scale multi-GPU training and deployment. I focus on turning advanced speech and audio methods into product-ready AI systems for natural voice interaction and creative audio generation.
Download my resumé .
- Generative Audio and Voice LLMs
- Multimodal Foundation Models
- Speech and Singing Voice Generation
- Speaker Recognition and Antispoofing
- Audio Separation and Enhancement
PhD in Informatics, 2024
National Institute of Informatics & SOKENDAI
MEng in Electrical Engineering and Information System, 2020
The University of Tokyo
BEng in Measurement and Control Technology and Instruments, 2016
Tianjin University
News
- 2026.06: Speech Codec Probing from Semantic and Phonetic Perspectives accepted by Interspeech 2026.
- 2026.05: Released StepAudio 2.5 Technical Report on arXiv.
- 2026.05: A Semantically Consistent Dataset for Data-Efficient Query-Based Universal Sound Separation accepted by ICML 2026.
- 2026.03: Released BEAVER: A Training-Free Hierarchical Prompt Compression Method via Structure-Aware Page Selection on arXiv.
- 2026.01: DrivingScene and PAGS accepted by ICASSP 2026.
- 2025.12: Released Towards Interactive Intelligence for Digital Humans on arXiv.
- 2025.10: Critical Information Only accepted by IEEE TDSC.
- 2025.01: SonicSim accepted by ICLR 2025.
- 2025.01: A Benchmark for Multi-Speaker Anonymization accepted by IEEE TIFS.
- 2024.12: InstructSing accepted by SLT 2024.
- 2024.12: Spoofing-Aware Speaker Verification Robust Against Domain and Channel Mismatches accepted by SLT 2024.
- 2024.03: Released HAM-TTS on arXiv.
Publications
Skills
Python, C++, Shell, Git, MySQL
PyTorch, PyTorch Lightning, Hugging Face
SpeechBrain, WeNet, WeSpeaker, Kaldi, ESPnet
Expressive TTS, codec and tokenizer design, voice LLMs
Audio-language modeling, full-duplex speech systems
Chinese, English, Japanese
Activities
Reviewer Service
- Conferences: NeurIPS, ICLR, ICML, ACL, ICASSP, ICME, INTERSPEECH
- Journals: IEEE OJSP, IEEE TASLP
Academic Activities
- ICASSP22, Interspeech22 Oral Presentation
- Organizing Committee, Joint Workshop of VoicePersonae and ASVspoof 2023
- SLT 2024, SVDD Challenge (Invited Talk): Shared insights on singing voice generation as a guest speaker
Open Source
- WeSpeaker contributor
- ASV-Subtools contributor
- HIVE dataset contributor
Experience
Research and develop next-generation speech, audio, and voice-agent models for natural realtime interaction and creative audio generation.
- Build full-duplex dialogue systems and realtime speech models for low-latency, natural voice interaction
- Develop voice-agent models and hyper-realistic, highly expressive TTS systems
- Explore music generation models and broader generative-audio foundations
Conducted research on generative audio, voice LLMs, multimodal interaction, and intelligent audio understanding.
- Explored foundation models and data-centric methods for speech, audio, and multimodal AI
- Continued open research collaborations and publication work across generative audio and sound understanding
Led R&D of expressive TTS and full-stack voice-agent technologies for avatar and game products.
- Built KodamaTTS from scratch on a Qwen-based foundation model for virtual human and gaming applications
- Covered data curation, codec design, multi-node multi-GPU training, and evaluation in one pipeline
- Achieved sub-1 kbps codec quality with strong objective performance and reached top benchmark rankings in 3 languages
- Served as Co-PI on a joint project with Tsinghua University on cocktail-party speech interaction and audio separation
Developed voice generation systems for Li Auto smart-space products and contributed to the multimodal foundation model MindGPT-4o.
- Proposed the GFSQ tokenizer for GPT-SoVITS to improve codebook utilization and decoding quality
- Trained a multi-timbre, multi-style voice generation model for in-car voice-blog scenarios in production
- Built synthetic-data workflows to scale accents, dialects, languages, emotions, and scenarios
- Led data production and audio-head pretraining and post-training for a full-duplex conversational model
Focused on high-fidelity 48kHz singing voice generation in collaboration with research and engineering teams.
- Upgraded XiaoiceSing to XiaoiceSing2 with adversarial training and achieved near-human MOS
- Developed HiFi-WaveGAN with a pulse-sequence design for stronger 48kHz singing synthesis quality
- Built CrossSinger for cross-lingual multi-singer SVS in English, Japanese, and Chinese
- Improved training efficiency with InstructSing and explored hierarchical acoustic modeling for voice LMs
Developed speech AI systems for Taobao Live compliance and broadcaster-risk control.
- Built a large-scale speaker recognition system for broadcaster identity verification in livestream scenarios
- Developed a spoken-term detection pipeline for policy-sensitive and illegal word monitoring
- Researched self-supervised speech representations and implemented an ESPnet-based end-to-end ASR system
