About Guan-Ting (Daniel) Lin

Guan-Ting is currently a Ph.D. student at Speech Processing and Machine Learning Lab, National Taiwan University, under the supervision of Prof. Hung-yi Lee. His research interest includes Deep Learning for Speech Processing, Multi-modal LLM for Speech, Spoken Language Understanding.

Guan-Ting has published 10+ papers at top Speech/NLP-related conferences (ACL, EMNLP, ICASSP, INTERSPEECH, and IEEE SLT) as the first or co-first author. Notably, he won the Best Paper Award at IEEE SLT 2022 in Doha, Qatar. Additionally, he serves as an official reviewer for multiple top conferences (ICLR, ACL, EMNLP, ICASSP, etc.).

Guan-Ting has several industrial research experience. In the summer of 2022, he worked with Amazon Alexa on Acoustic Event Classification in Cambridge, USA. In the summer of 2023, he worked with the Alexa Speech Recognition LM team in Seattle, USA. He interned with Amazon’s AGI-Speech team in the summer of 2024 in Seattle, USA, working on a project to enhance end-to-end spoken language models.

He is an incoming Student Researcher @ Google, New York in 2025 Spring, and an incoming Research Scientist Intern @ Meta, Menlo Park in 2025 Fall.

For more details, please see the [CV].

Beyond academia, he enjoys singing 🎤, photography 📷, and watching MLB games ⚾️.

Recent News 🚨

Education

  • Ph.D. in Communication Engineering, EECS, National Taiwan University [2021 - Present]
    • Advisor: Prof. Hung-yi Lee
    • GPA: 4.24/4.3; Ranking: 15/158
    • Transferred from M.S. program in Feb. 2023.

Selected Publications & Preprints

(For full publication list, please see the Google Scholar).

Speech/Text Large Language Models

  • Align-SLM: Textless Spoken Language Models with Reinforcement Learning from AI Feedback
    Guan-Ting Lin, Prashanth Gurunath Shivakumar, Aditya Gourav, Yile Gu, Ankur Gandhe, Hung-yi Lee, Ivan Bulyko
    arXiv preprint
    paper
  • Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations
    Guan-Ting Lin, Cheng-Han Chiang, Hung-yi Lee
    ACL 2024
    paper / data
  • Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue
    Guan-Ting Lin, Prashanth Gurunath Shivakumar, Ankur Gandhe, Chao-Han Huck Yang, Yile Gu, Shalini Ghosh, Andreas Stolcke, Hung-yi Lee, Ivan Bulyko
    ICASSP 2024
    paper
  • Can LLMs Understand the Implication of Emphasized Sentences in Dialogue?
    Guan-Ting Lin, Hung-yi Lee
    EMNLP 2024 Findings
    paper / data

Self-supervised Speech Model

  • On the Utility of Self-supervised Models for Prosody-related Task
    Guan-Ting Lin(co-first), Chi-Luen Feng(co-first), Wei-Ping Huang, Yuan Tseng, Tzu-Han Lin, Chen-An Li, Hung-yi Lee, Nigel G. Ward
    SLT 2022 (Best Paper Award)
    paper / code
  • Introducing Semantics into Speech Encoders
    Derek Xu, Shuyan Dong, Changhan Wang, Suyoun Kim, Zhaojiang Lin, Akshat Shrivastava, Shang-Wen Li, Liang-Hsuan Tseng, Alexei Baevski, Guan-Ting Lin, Hung-yi Lee, Yizhou Sun, Wei Wang
    ACL 2023
    paper
  • SUPERB: Speech processing Universal PERformance Benchmark
    Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
    Interspeech 2021
    paper / code
  • Analyzing the Robustness of Unsupervised Speech Recognition
    Guan-Ting Lin(co-first), Chan-Jan Hsu(co-first), Da-Rong Liu, Hung-Yi Lee, Yu Tsao
    ICASSP 2022
    paper / code

Spoken Language Understanding and Spoken Question Answering

  • SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answering
    Chyi-Jiunn Lin, Guan-Ting Lin, Yung-Sung Chuang, Wei-Lun Wu, Shang-Wen Li, Abdelrahman Mohamed, Hung-yi Lee, Lin-shan Lee
    ICASSP 2024
    paper
  • Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks
    Kevin Everson, Yile Gu, Huck Yang, Prashanth Gurunath Shivakumar, Guan-Ting Lin, Jari Kolehmainen, Ivan Bulyko, Ankur Gandhe, Shalini Ghosh, Wael Hamza, Hung-yi Lee, Ariya Rastrow, Andreas Stolcke
    ICASSP 2024
    paper
  • Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target
    Guan-Wei Wu(co-first), Guan-Ting Lin(co-first), Shang-Wen Li, Hung-yi Lee
    Interspeech 2023
    paper
  • DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering
    Guan-Ting Lin, Yung-Sung Chuang, Ho-Lam Chung, Shu-wen Yang, Hsuan-Jui Chen, Shuyan Dong, Shang-Wen Li, Abdelrahman Mohamed, Hung-yi Lee, Lin-shan Lee
    Interspeech 2022 (Poster)
    paper / code

End-to-end ASR Test-time Adaptation

  • Continual Test-time Adaptation for End-to-end Speech Recognition on Noisy Speech
    Guan-Ting Lin(co-first), Wei-Ping Huang(co-first), Hung-yi Lee
    EMNLP 2024
    paper
  • Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition
    Guan-Ting Lin, Shang-Wen Li, Hung-Yi Lee
    Interspeech 2022 (Oral)
    paper / code

Audio Event Classification

  • Weight-sharing Supernet for Searching Specialized Acoustic Event Classification Networks Across Device Constraints
    Guan-Ting Lin, Qingming Tang, Chieh-Chi Kao, Viktor Rozgic, Chao Wang
    ICASSP 2023
    paper

Experience

  • Applied Scientist II Intern, Amazon AGI, Seattle, United States
    [2024/6 - 2024/9]
    • AGI-Speech Research Team
    • End-to-end Spoken Language Models.
  • Applied Scientist II Intern, Amazon Alexa, Seattle, United States
    [2023/6 - 2023/9]
    • Speech Recognition LM Research Team
    • Paralinguistics-enhanced Large Language Model on spoken dialogue.
  • Applied Scientist I Intern, Amazon Alexa, Cambridge, United States
    [2022/7 - 2022/10]
    • Manager: Chieh-Chi Kao / Mentor: Qingming Tang
    • Develop Once-for-all Network Architecture Search techniques on audio event classification.
  • Visiting Reseacher, 8th JASLT Summer Workshop, Johns Hopkins University, Baltimore, United States
    [2022/6 - 2022/7]
  • Summer Research Intern & Research Assistant, National Center for High-Performance Computing, National Applied Research Laboratories
    [2019/7 - 2020/7]
    • Advisor: Nan-You Chen
    • Low-dose CT image, denoising reconstructed images by U-NET based deep neural network.

Award

  • IEEE Signal Processing Society Travel Grant @ ICASSP 2024
  • Best paper award @ IEEE SLT 2022
  • NTU Elite Doctoral Scholarship
  • GICE Elite Doctoral Scholarship with NVIDIA
  • ISCA travel grant @ Interspeech 2022
  • Appier top-tier conference scholarship
  • Dean’s list * 3 @ NTHU
  • Phi Tau Phi Award @ NTHU
  • The Zhu Shun Yi He Qin Scholarship @ NTHU

Academic Services

  • Official Reviewer: ICLR’24’25, NeurIPS’24, ACL’24, EMNLP’24, NAACL’23’24, ICASSP’23’24, ISCSLP’22’23’24, COLING’25