Towards Crowd-sourced Semantic-based Multimodal User Interfaces


                Department of Systems Engineering and Engineering Management
                             The Chinese University of Hong Kong


Date: Wednesday, 11:00am - 12:30pm, March 11, 2015

Title: Towards Crowd-sourced Semantic-based Multimodal User Interfaces

Speaker: Dr. Xiaojuan Ma, Huawei Noah’s Ark Lab


Communication of information takes place in both spoken and written forms on user interfaces of desktop, laptop, and mobile devices. However, everyone may encounter occasions when receiving and expressing information via a language is difficult. Therefore, people seek alternative ways to communicate when they fail to retrieve the exact concept. One of the most common strategies is semantic paraphrasing, which requires the user interfaces to be equipped with a well-structured, dense knowledge base with semantic relations among encoded concepts. Traditionally, manual approaches for constructing knowledge bases may not scale well, while the automatic approaches via text mining may not explicitly reflect the organization of human mental lexicon. We propose a strategy that leverages collective intelligence to augment semantic network with directed, weighted, cross-part-of-speech associative links. We integrate such crowd-sourced method with machine learning and natural language processing techniques to achieve better scalability. Evaluations show that our method captures new semantic relatedness information, which can better facilitate sense making and word finding on multimodal user interfaces.


Xiaojuan Ma, Ph.D., is a researcher at Huawei Noah's Ark Lab. Before joining Noah's Ark Lab in 2012, she was a post-doctoral researcher in the Human-Computer Interaction Institute (HCII) at Carnegie Mellon University (CMU), and was awarded Computing Innovation Fellow by Computing Research Association. Dr. Ma received her Ph.D. degree from the Computer Science Department of Princeton University in 2010. She worked as a research fellow in the Department of Information Systems, National University of Singapore before joining CMU. Dr. Ma is the board member and secretary general of International Chinese Association of Computer Human Interaction (ICACHI), and member of ACM and SigCHI. Her background is in Human-Computer Interaction. She is particularly interested in human computation and crowdsourcing, multimodal interfaces for human-human, human-computer, and human-robot interactions, design, ubiquitous computing, and computational linguistics. She has served in the program committees and organizing committees of many ACM and IEEE conferences and workshops related to Human-Computer / Human-Robot Interaction, Ubiquitous Computing, and Design.


Everyone is welcome to attend the talk!

Venue: Room 513,
      William M.W. Mong Engineering Building (ERB),
      (Engineering Building Complex Phase 2)
      The Chinese University of Hong Kong.

The talk will be hosted by:
Prof. Helen Meng,
Department of Systems Engineering and Engineering Management,
The Chinese University of Hong Kong,

SEEM-5202 Website:

Wednesday, March 11, 2015 - 03:00 to 04:30