Current Projects

Cognitive Assistant that Learns and Organizes (CALO)

We are researching robust multimodal natural language and discourse understanding for use in monitoring, recording, and summarizing multi-party meetings. Research foci include automatic topic segmentation and extraction, decision detection, multimodal fusion, ontological discourse modelling, robust semantic parsing, and dialogue act detection. Funded by DARPA through SRI.

• Semlab CALO wiki
• CALO at SRI
• CALO at DARPA
• Publications

In-car Multi-agent Dialogue System

We are developing a multi-device in-car spoken dialogue system called CHAT (Conversational Handling for Automotive Tasks). This allows a driver to interact with the car's systems (stereo, navigation instructions, local information database etc) without having to read screens or be otherwise visually distracted. Research areas include device addressee detection, error handling, confirmation & clarification strategies, plug-&-play device handling, and robust dialogue management in the face of noisy input. Funded by NIST in partnership with Bosch and VW.

• Semlab NIST wiki (accessible only by project participants)
• Publications

Natural Language Interaction with Intelligent Tutoring Systems (ONR)

We are investigating the advantages of spoken dialogue in automated tutoring and coaching, through development of our Spoken Conversational Tutor (SCoT). Our Navy damage control tutor has been used in experiments at Stanford, U.S. Naval Academy and SWOS. Our tutoring system conducts an after-action review dialogue with a student who has just finished a session with the DC-Train simulator, while our coaching system currently under development will talk to a student in the midst of using DC-Train, for advice and guidance. We are also beginning development of a shiphandling coach using the COVE simulator. Our experiments have investigated how different tutoring strategies and styles affect subjects' learning and subsequent simulator performance.

• Publications
• Demo videos of SCoT (2002 version) and voice-enabled DC-Train
• Previous web page available here.

Finished Projects

Information Mapping (Infomap)

• Full description available here.

Multimodal Conversational Interface (WITAS)

• Full description available here.