Yong Ma

01 About

I am a Postdoctoral Researcher at KTH Royal Institute of Technology, Division of Robotics, Perception and Learning (RPL), working on Brain–Robot Interfaces, Multimodal Human–Robot Interaction, and Intelligent Voice Interfaces. As part of the SSF-funded BOS (Body-centric Operating System) project, my current work fuses EEG, EMG, and voice signals with motion sensing to decode human motor intention and build multimodal brain–body–voice interfaces for real-time, anticipatory robotic control. My broader research centers on human-centered multimodal conversational and embodied intelligence, include agents and robots that understand not only what people say, but also what they intend, feel, and need, spanning speech signal processing, affective computing, and emotion-aware voice interfaces.

  • Brain–Robot Interfaces
  • EEG & EMG Sensing
  • Human–Robot Interaction
  • Voice & Speech Interfaces
  • Affective Computing

02 Work & Academic Experience

  1. 07.2026 – Present

    Postdoctoral Researcher

    KTH Royal Institute of Technology · Division of Robotics, Perception and Learning (RPL), Stockholm, Sweden

    Brain–Robot Interfaces, Multimodal Human–Robot Interaction, and Intelligent Voice Interfaces — combining EEG, EMG, and voice signals for real-time robotic interaction. Supervised by Prof. Danica Kragic Jensfelt and Prof. Mårten Björkman.

  2. 04.2023 – 05.2026

    Postdoctoral Researcher

    University of Bergen · Department of Information Science and Media Studies, Bergen, Norway

    Speech Signal Processing and Emotion-Aware Voice Interfaces: applications in dementia care and elderly healthcare. Supervised by Prof. Morten Fjeld.

  3. 02.2026 – 07.2026

    Visiting Researcher

    Technical University of Munich · Chair of Health Informatics (CHI), Munich, Germany

    Speech emotion detection and its application in emotion-aware voice interface design. Hosted by Prof. Björn Schuller.

Current Projects

In Preparation ICASSP 2027

Speech-Based Dementia Detection using Mixture-of-Experts

Detecting signs of dementia from speech using a mixture-of-experts model.

In Preparation CHI 2027

Repairing Conversational Agent Failures

Understanding and repairing conversational breakdowns in voice-based conversational agents.

Earlier Research Projects

2014 – 2015

Wearable Autism Voice Monitoring System

A wearable system to assess infantile autism by recording and analyzing speech features — fundamental, rhythm, and social-perceptual — to help detect stress in individuals with autistic disorder.

2015 – 2016

Wearable Autism Behavior Monitoring System

Analyzed behavioral differences between individuals with infantile autism and the general population to assess mental stress through speech and motion behavioral features.

03 Education

  1. 10.2017 – 11.2022

    Ph.D., Medieninformatik

    Ludwig-Maximilians-Universität München (LMU Munich), Munich, Germany

    Thesis: Emotion-Aware Voice Interface Based on Speech Signal Processing.

04 Research Publications

31 peer-reviewed publications across CHI, IUI, HRI, and related venues.

  1. Best Paper · IUI 2026Ma, Y., Zhang, Y., Andrews, P., Wu, Z., Portales, S. Z., & Fjeld, M. (2026). "Same Voice, Different Language": An Exploration of Voice-Cloned Translation to Support Non-Native Speakers in Online Meetings. Proceedings of the 31st International Conference on Intelligent User Interfaces (IUI '26), 1–17. Paphos, Cyprus: ACM.
  2. Honorable Mention · CHI 2026Ma, Y., Zhang, X., Zhang, X., Bartłomiejczyk, N., Je, S., Holzer, A., … Butz, A. (2026). Beyond Words: Measuring User Experience through Speech Analysis in Voice User Interfaces. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26), 1–25. New York, NY, USA: ACM.
  3. Best Paper · CHI 2026Dai, D., Zhang, Y., Ma, Y., & Kragic, D. (2026). AI That Moves With You: A Review of Interactive Technologies Powered by Large Foundation Models for Mobility Impairment. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26), 1–23. New York, NY, USA: ACM.
  4. Ma, Y., Zhang, Y., Fu, D., Zubicueta Portales, S., & Fjeld, M. (2026, March). Neutral by Default? Replicating User Vocal Responses to Negative Affective Cues in Conversational Agents. Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI '26), 1–5. Edinburgh, Scotland, UK: ACM.
  5. Zhang, X., Ye, Y., Ma, Y., Zhang, Z. T., & Butz, A. (2026, March). Beyond Information Amount: Rethinking Transparency through Active Reasoning Demand. Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction (HRI '26), 1–10. Edinburgh, Scotland, UK: ACM.
  6. Fu, D., Zhang, Y., Ma, Y., Kyrarini, M., Chen, C., Jirak, D., … Kragic, D. (2026). InterAI: Interactive AI for Human-Centered Robotics. Companion Proceedings of the 21st ACM/IEEE International Conference on Human-Robot Interaction, 1408–1410.
  7. Zhang, Y., Fu, D., Ma, Y., Yadollahi, E., Leite, I., & Kragic, D. (2026). The 3rd InterAI Workshop: Interactive AI for Human-Centered Robotics. Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems, 1–8.
  8. Zhang, Y., Ma, Y., Fu, D., Zubicueta Portales, S., Fjeld, M., & Kragic, D. (2026). Personalizing Emotion-Aware Conversational Agents? Exploring User Traits-Driven Conversational Strategies for Enhanced Interaction. International Journal of Human–Computer Interaction, 1–38.
  9. Ma, Y., Zhang, X., Zhang, Y., & Fjeld, M. (2025). Measuring User Experience through Speech Analysis: Insights from HCI Interviews. Extended Abstracts of the 2025 CHI Conference on Human Factors in Computing Systems, 1–4.
  10. Ma, Y., Zhang, Y., Fu, D., Portales, S. Z., Kragic, D., & Fjeld, M. (2025). Advancing User-Voice Interaction: Exploring Emotion-Aware Voice Assistants through a Role-Swapping Approach. International Conference on Human-Computer Interaction, 240–255. Springer.
  11. Ma, Y., Zhang, Y., Nordberg, O. E., Rongve, A., Bachinski, M., & Fjeld, M. (2025). State-of-the-Art HCI for Dementia Care: A Scoping Review of Recent Technological Advances. Journal of Dementia and Alzheimer's Disease, 2(4), 41.
  12. Zhang, Y., Ma, Y., Di, F., Portales, S. Z., Fjeld, M., & Kragic, D. (2025). Navigating Users in Emotion-Aware Conversational Agents: Exploring Strategies for Enhanced Interaction. Available at SSRN 5148530.
  13. Ma, Y., Nordberg, O. E., Zhang, Y., Rongve, A., Bachinski, M., & Fjeld, M. (2026, July). Bridging the Gap: Advancements in Interactive Technology to Support Dementia Care – A Scoping Review. In International Conference on Human-Computer Interaction (pp. 188–220). Cham: Springer Nature Switzerland.
  14. Ma, Y., Nordberg, O. E., Zhang, Y., Rongve, A., Bachinski, M., & Fjeld, M. (2024). Understanding Dementia Speech: Towards an Adaptive Voice Assistant for Enhanced Communication. Companion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems, 15–21.
  15. Ma, Y., Zhang, Y., Fu, D., Jahanshahikhabisi, M., Matviienko, A., Bachinski, M., … Kragic, D. (2024). HAID: Human-AI Interaction for Dementia Care. Adjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction, 1–4.
  16. Zhang, Y., Kassem, K., Gong, Z., Mo, F., Ma, Y., Kirjavainen, E., & Häkkilä, J. (2024). Human-Centered AI Technologies in Human-Robot Interaction for Social Settings. Proceedings of the International Conference on Mobile and Ubiquitous Multimedia, 501–505.
  17. Zhang, Y., Ma, Y., & Kragic, D. (2024). Vision Beyond Boundaries: An Initial Design Space of Domain-Specific Large Vision Models in Human-Robot Interaction. Adjunct Proceedings of the 26th International Conference on Mobile Human-Computer Interaction, 1–8.
  18. Zhang, Y., Rajabi, N., Taleb, F., Matviienko, A., Ma, Y., Björkman, M., & Kragic, D. (2025, October 18). Mind Meets Robots: A Review of EEG-Based Brain-Robot Interaction Systems. International Journal of Human–Computer Interaction, 41(20), 12784–12815.
  19. Zhang, Y., Yadollahi, E., Ma, Y., Fu, D., Leite, I., & Kragic, D. (2024). The 1st InterAI Workshop: Interactive AI for Human-Centered Robotics. arXiv preprint arXiv:2409.11150.
  20. Ma, Y., Zhang, Y., Bachinski, M., & Fjeld, M. (2023). Emotion-Aware Voice Assistants: Design, Implementation, and Preliminary Insights. Proceedings of the Eleventh International Symposium of Chinese CHI, 527–532.
  21. Ma, Y., Drewes, H., & Butz, A. (2022). How Should Voice Assistants Deal with Users' Emotions? arXiv preprint arXiv:2204.02212.
  22. Ma, Y., Yomna, A., Petz, B., Drewes, H., Alt, F., Hussmann, H., & Andreas, B. (2022). Enthusiasts, Pragmatists, and Skeptics: Investigating Users' Attitudes towards Emotion- and Personality-Aware Voice Assistants across Cultures. Mensch und Computer 2022, 1–15.
  23. Li, J., Ma, Y., Li, P., & Butz, A. (2021). A Journey through Nature: Exploring Virtual Restorative Environments as a Means to Relax in Confined Spaces. Creativity and Cognition, 1–9.
  24. Ma, Y., Drewes, H., & Butz, A. (2021). Fake Moods: Can Users Trick an Emotion-Aware Voice Bot? Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 1–4.
  25. Ma, Y., Li, J., Drewes, H., & Butz, A. (2021). You Sound Relaxed Now – Measuring Restorative Effects from Speech Signals. IFIP Conference on Human-Computer Interaction, 585–594. Springer.
  26. Zhang, Y., Ma, Y., Adel, O., Yadav, R., Fjeld, M., & Fratarcangeli, M. (2020). Automated Microwave Tomography (MWT) Image Segmentation: State-of-the-Art Implementation and Evaluation.
  27. Zhang, Y., Ma, Y., Omrani, A., Yadav, R., Fjeld, M., & Fratarcangeli, M. (2019). Automatic Image Segmentation for Microwave Tomography (MWT) from Implementation to Comparative Evaluation. Proceedings of the 12th International Symposium on Visual Information Communication and Interaction, 1–2.
  28. Gu, J., Gao, B., Chen, Y., Jiang, L., Gao, Z., Ma, X., … Woo, W. L. (2017). Wearable Social Sensing and its Application in Anxiety Assessment. 2017 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery (CyberC), 305–308. IEEE.
  29. Gu, J., Gao, B., Chen, Y., Jiang, L., Gao, Z., Ma, X., … Jin, J. (2017). Wearable Social Sensing: Content-Based Processing Methodology and Implementation. IEEE Sensors Journal, 17(21), 7167–7176.
  30. Sun, Y. X., Hu, J. P., Shi, K. B., Ma, Y., Zhao, Y. Y., & Zhang, Y. P. (2017). Reduced-Order Observer-Based Consensus for Multi-Agent Systems with Nonuniform Time-Varying Delays. 2017 Chinese Automation Congress (CAC), 3052–3057. IEEE.
  31. Sun, Y. X., Ma, Y., Shi, K. B., Hu, J. P., Zhao, Y. Y., & Zhang, Y. P. (2017). Unsupervised Speaker Segmentation Framework Based on Sparse Correlation Feature. 2017 Chinese Automation Congress (CAC), 3058–3063. IEEE.

05 Awards & Honors

2026

Best Paper Award — CHI 2026

AI That Moves with You: A Review of Interactive Technologies Powered by Large Foundation Models for Mobility Impairment.

2026

Honorable Mention Award — CHI 2026

Beyond Words: Measuring User Experience through Speech Analysis in Voice User Interfaces.

2026

Best Paper — IUI 2026

"Same Voice, Different Language": An Exploration of Voice-Cloned Translation to Support Non-Native Speakers in Online Meetings.

06 Academic Service & Organization

Academic Service

  • 2026 — Social Media Chair, ACM CUI 2026
  • 2026 — ConfTool Chair, Mensch und Computer 2026 (MuC)
  • 2025 — Associate Chair (AC), Mensch und Computer (MuC)
  • 2024 — Associate Chair (AC), Mensch und Computer (MuC)
  • 2024 — Associate Chair (AC), CSCW
  • Reviewer for CHI, Ro-MAN, ICMI, HCII, TEI, MuC, Frontiers, and others

Workshop Organization

  • 2026 — Co-Organizer, HRI 2026 Workshop: InterAI: Interactive AI for Human-Centered Robotics
  • 2026 — Co-Organizer, CHI 2026 Workshop: The 3rd InterAI Workshop
  • 2024 — Co-Organizer, RO-MAN 2024 Workshop: Interactive AI for Human-Centered Robotics
  • 2024 — First Organizer, NordiCHI 2024 Workshop: HAID — Human-AI Interaction for Dementia Care
  • 2024 — Co-Organizer, MuM 2024 Workshop: Human-Centered AI Technologies in Human-Robot Interaction for Social Settings

Teaching & Supervising

  • 2024 — Guest Lecturer, Voice Interfaces course, Bachelor's programme, University of Bergen
  • 2023 — Teaching Assistant, Human-Computer Interaction (HCI) course, University of Bergen
  • 2018 – 2021 — Supervised several Master's and Bachelor's thesis projects at LMU Munich

07 Get in Touch

I'm always happy to discuss research collaborations, speech & voice interfaces, human-robot interaction, or opportunities in human-centered AI.