Guan-Ting is currently a second-year Ph.D. student at Speech Processing and Machine Learning Lab, National Taiwan University, under the supervision of Prof. Hung-yi Lee. His research interest includes Deep Learning for Speech Processing, Multi-model Large Language model for speech and audio, Spoken Language Understanding. For more details, please see the résumé via [link].

Guan-Ting has published several conference papers at speech-related top conferences (ICASSP, INTERSPEECH, and IEEE SLT) as the first/co-first author. Notably, he won the Best Paper Award in IEEE SLT 2022, Doha, Qatar. In 2022 summer, Guan-Ting worked with Amazon Alexa for Acoustic Event Classification in Cambridge, USA. He worked with the Alexa Speech Recognition LM team in 2023 summer.

Beyond academia, he enjoys singing, photography, and watching MLB games.

Recent News

• (02/2024) A new pre-print paper Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations is released!
• (01/2024) Received IEEE Signal Processing Society Travel Grant for participating ICASSP 2024!
• (12/2023) Three papers are accepted by ICASSP 2024 (one first-author and two co-author). See you in Seoul!
• (02/2023) My internship work with Amazon Alexa is accepted by ICASSP 2023!
• (01/2023) Our paper “On the Utility of Self-supervised Models for Prosody-related Tasks”, cooperating with Prof. Nigel Ward of UTEP, won the Best Paper Award of IEEE SLT 2022!
• (09/2022) SUPERB-prosody is accepted by SLT 2022.
• (07/2022) Received ISCA Travel Grants for Interspeech 2022.
• (06/2022) Two first-author papers are accepted at Interspeech 2022.

Education

• Ph.D. in Communication Engineering, EECS College, National Taiwan University [2021 - Present]
  • Advisor: Prof. Hung-yi Lee
  • GPA: 4.24/4.3; Ranking: 15/158
  • Transferred from M.S. program in Feb. 2023.
• Advanced AI Program, National Tsing Hua University [2020 - 2021]
  • GPA: 4.3/4.3
• B.S. in Biomedical Engineering, National Tsing Hua University [2017 - 2021]
  • GPA: 4.08/4.3; Ranking: 1/45

Publications & Preprints

2024

• Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations
  Guan-Ting Lin, Cheng-Han Chiang, Hung-yi Lee
  Arxiv preprint
  paper
• Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue
  Guan-Ting Lin, Prashanth Gurunath Shivakumar, Ankur Gandhe, Chao-Han Huck Yang, Yile Gu, Shalini Ghosh, Andreas Stolcke, Hung-yi Lee, Ivan Bulyko
  ICASSP 2024
  paper
• SpeechDPR: End-to-End Spoken Passage Retrieval for Open-Domain Spoken Question Answering
  Chyi-Jiunn Lin, Guan-Ting Lin, Yung-Sung Chuang, Wei-Lun Wu, Shang-Wen Li, Abdelrahman Mohamed, Hung-yi Lee, Lin-shan Lee
  ICASSP 2024
  paper
• Towards ASR Robust Spoken Language Understanding Through In-Context Learning With Word Confusion Networks
  Kevin Everson, Yile Gu, Huck Yang, Prashanth Gurunath Shivakumar, Guan-Ting Lin, Jari Kolehmainen, Ivan Bulyko, Ankur Gandhe, Shalini Ghosh, Wael Hamza, Hung-yi Lee, Ariya Rastrow, Andreas Stolcke
  ICASSP 2024
  paper

2023

• Improving Textless Spoken Language Understanding with Discrete Units as Intermediate Target
  Guan-Wei Wu_(co-first), Guan-Ting Lin_(co-first), Shang-Wen Li, Hung-yi Lee
  Interspeech 2023
  paper
• Weight-sharing Supernet for Searching Specialized Acoustic Event Classification Networks Across Device Constraints
  Guan-Ting Lin, Qingming Tang, Chieh-Chi Kao, Viktor Rozgic, Chao Wang
  ICASSP 2023
  paper
• Introducing Semantics into Speech Encoders
  Derek Xu, Shuyan Dong, Changhan Wang, Suyoun Kim, Zhaojiang Lin, Akshat Shrivastava, Shang-Wen Li, Liang-Hsuan Tseng, Alexei Baevski, Guan-Ting Lin, Hung-yi Lee, Yizhou Sun, Wei Wang
  ACL 2023
  paper

2022

• On the Utility of Self-supervised Models for Prosody-related Task
  Guan-Ting Lin_(co-first), Chi-Luen Feng_(co-first), Wei-Ping Huang, Yuan Tseng, Tzu-Han Lin, Chen-An Li, Hung-yi Lee, Nigel G. Ward
  SLT 2022 (Best Paper Award)
  paper / code
• Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognition
  Guan-Ting Lin, Shang-Wen Li, Hung-Yi Lee
  Interspeech 2022 (Oral)
  paper / code
• DUAL: Discrete Spoken Unit Adaptive Learning for Textless Spoken Question Answering
  Guan-Ting Lin, Yung-Sung Chuang, Ho-Lam Chung, Shu-wen Yang, Hsuan-Jui Chen, Shuyan Dong, Shang-Wen Li, Abdelrahman Mohamed, Hung-yi Lee, Lin-shan Lee
  Interspeech 2022 (Poster)
  paper / code
• Analyzing the Robustness of Unsupervised Speech Recognition
  Guan-Ting Lin_(co-first), Chan-Jan Hsu_(co-first), Da-Rong Liu, Hung-Yi Lee, Yu Tsao
  ICASSP 2022
  paper / code

2021

• SUPERB: Speech processing Universal PERformance Benchmark
  Shu-wen Yang, Po-Han Chi, Yung-Sung Chuang, Cheng-I Lai, Kushal Lakhotia, Yist Y. Lin, Andy T. Liu, Jiatong Shi, Xuankai Chang, Guan-Ting Lin, Tzu-Hsien Huang, Wei-Cheng Tseng, Ko-tik Lee, Da-Rong Liu, Zili Huang, Shuyan Dong, Shang-Wen Li, Shinji Watanabe, Abdelrahman Mohamed, Hung-yi Lee
  Interspeech 2021
  paper / code
• Context-gloss Augmentation for Improving Word Sense Disambiguation
  Guan-Ting Lin, Manuel Giambi
  arXiv preprint arXiv:2110.07174
  paper

Experience

• Applied Scientist II Intern, Amazon Alexa, Seattle, United States
  [2023/6 - 2023/9]
  • Speech Recognition LM Research Team
  • Paralinguistics-enhanced Large Language Model on spoken dialogue.
• Applied Scientist I Intern, Amazon Alexa, Cambridge, United States
  [2022/7 - 2022/10]
  • Manager: Chieh-Chi Kao / Mentor: Qingming Tang
  • Develop Once-for-all Network Architecture Search techniques on audio event classification.
• Visiting Reseacher, 8th JASLT Summer Workshop, Johns Hopkins University, Baltimore, United States
  [2022/6 - 2022/7]
  • Work with Prof. Nigel Ward on self-supervised pre-training for prosody.
• Summer Research Intern & Research Assistant, National Center for High-Performance Computing, National Applied Research Laboratories
  [2019/7 - 2020/7]
  • Advisor: Nan-You Chen
  • Low-dose CT image, denoising reconstructed images by U-NET based deep neural network.

Award

• IEEE Signal Processing Society Travel Grant @ ICASSP 2024
• Best paper award @ IEEE SLT 2022
• NTU Elite Doctoral Scholarship
• GICE Elite Doctoral Scholarship with NVIDIA
• ISCA travel grant @ Interspeech 2022
• Appier top-tier conference scholarship
• Dean’s list * 3 @ NTHU
• Phi Tau Phi Award @ NTHU
• The Zhu Shun Yi He Qin Scholarship @ NTHU

Services

• Reviewer: ISCSLP 2022, ARR 2022/2023/2024, ICASSP 2023/2024, ICLR 2024