About Me
I am Yaoyun Zhang, an audio researcher working on universal audio intelligence. I earned my M.Eng. in Computer Technology at X-LANCE, Shanghai Jiao Tong University, under the supervision of Prof. Mengyue Wu and Prof. Kai Yu. Before that, I received dual bachelor’s degrees in Computer Science and Technology and English from Chengdu University of Technology.
My research spans multimodal audio generation, unified audio representation learning, and trustworthy audio intelligence. I began by studying domain-invariant learning for Gaussian processes and Bayesian exploration. I then focused on synchronized video-to-audio generation, where I developed frame-level visual fusion and semantic temporal conditioning for Smooth-Foley. More recently, I have explored unified flow-matching models for audio generation from omni-modalities, as well as explainable and generalizable deepfake sound detection.
During my research internship at Meituan, I worked on neural audio codecs and tokenizers for unified representations of speech, music, and general sounds. This experience led to my current primary interest: developing unified audio codecs and generative models that remain efficient, robust, and generalizable across modalities and domains. Please feel free to contact me via email.
Education
- B.Eng. in Computer Science and Technology, School of Computer Science
- B.A. in English, College of Foreign Languages and Cultures
Work Experience
- Worked on unified neural audio codecs and tokenizers for speech, music, and general sounds.
- Exploration and optimization of efficient audio reconstruction and generation.
Publications
You can also find my publications on Google Scholar.
Smooth-Foley: Creating Continuous Sound for Video-to-Audio Generation Under Semantic Guidance
Yaoyun Zhang, Xuenan Xu, Mengyue Wu*.
ICASSP 2025.
[Project] [arXiv]
- A video-to-audio generation framework with frame-level visual fusion and semantic temporal guidance for continuous and synchronized Foley sound generation.
FakeSound2: A Benchmark for Explainable and Generalizable Deepfake Sound Detection
Zeyu Xie, Yaoyun Zhang, Xuenan Xu, Yongkang Yin, Chenxing Li, Mengyue Wu*, Yuexian Zou*.
INTERSPEECH 2026.
[Project] [arXiv]
- A benchmark for explainable and generalizable deepfake sound detection across localization, traceability, and generalization, covering 6 manipulation types and 12 diverse sources.
UniFlow-Audio: A Unified Flow Matching Framework for Audio Generation from Omni-Modalities
Xuenan Xu, Jiahao Mei, Zihao Zheng, Ye Tao, Zeyu Xie, Yaoyun Zhang, Haohe Liu, Yuning Wu, Ming Yan, Wen Wu, Chao Zhang, Mengyue Wu*.
arXiv, 2025.
[Project] [arXiv]
- The first fully open-source Flow Matching-based unified audio generation framework with a novel Dual-Fusion mechanism for both Time-Align and Non-Time-Align tasks. Supports text, audio, and video inputs across 7 tasks (TTS, TTA, V2A, etc.).
Domain Invariant Learning for Gaussian Processes and Bayesian Exploration
Zhaoxi Long, Siyuan Bian, Yaoyun Zhang, Yuliang Zhang, Qinying Gu, Xinbing Wang, Chenghu Zhou, Nanyang Ye*.
AAAI 2024.
[Code] [arXiv]
- A domain-invariant learning framework for Gaussian processes that improves out-of-distribution generalization and extends Bayesian optimization to changing environments.
* Corresponding author.
Awards
- National Encouragement Scholarship (Top 3% nationwide), 2020
- Second Prize, National Computer Game Competition, 2021
CV
My current curriculum vitae is available as a PDF.