Current Projects

Natural Language Interaction with Intelligent Tutoring Systems (ONR and ARI)

We are investigating the advantages of spoken dialogue in automated tutoring and coaching, through development of our Spoken Conversational Tutor (SCoT). Our current research involves an automated shiphandling coach using the COVE simulator. In conjunction with colleagues at Naval Undersea Warfare Center (NUWC), we are conducting experiments comparing the use of the automated coach ("COVE-ITS", for "intelligent tutoring system") with human instruction. We are also comparing the effects of different automated tutoring strategies with Berkeley NROTC midshipmen. Our Navy damage control tutor has been used in experiments at Stanford, U.S. Naval Academy and SWOS. Our damage control tutoring system conducts an after-action review dialogue with a student who has just finished a session with the DC-Train simulator, while our coaching system will talk to a student in the midst of using DC-Train, for advice and guidance. We also developed a spoken dialogue interface to an Army battle captain training system by Stottler-Henke. Our experiments have investigated how different tutoring strategies and styles affect subjects' learning and subsequent simulator performance.

• Publications
• Demo videos of SCoT (2002 version) and voice-enabled DC-Train
• Previous web page available here.

Cognitive Assistant that Learns and Organizes (CALO)

We are researching robust multimodal natural language and discourse understanding for use in monitoring, recording, and summarizing multi-party meetings. Research foci include automatic topic segmentation and extraction, decision detection, multimodal fusion, ontological discourse modelling, robust semantic parsing, and dialogue act detection. Funded by DARPA through SRI.

• Semlab CALO wiki
• CALO at SRI
• CALO at DARPA
• Publications

In-car Multi-agent Dialogue System

We are developing a multi-device in-car spoken dialogue system called CHAT (Conversational Handling for Automotive Tasks). This allows a driver to interact with the car's systems (stereo, navigation instructions, local information database etc) without having to read screens or be otherwise visually distracted. Research areas include device addressee detection, error handling, confirmation & clarification strategies, plug-&-play device handling, and robust dialogue management in the face of noisy input. Funded by NIST in partnership with Bosch and VW.

• Semlab NIST wiki (accessible only by project participants)
• Publications

Dialogue Systems for Human-Robot Interaction

Our newest research focus is the development and integration of information-state based spoken dialogue systems for human-robot interaction. Because real robots are expensive and temperamental, this work is being implemented within the USARSim simulation environment.

• Semlab Robot wiki (accessible only by project participants)

Dialogue system for air taxi cab with back seat driver

This project centers around the scenario of an air taxi cab with a back seat driver. Our spoken dialog system is used to enable interaction between the user (back seat driver) and the on-board computer via natural language, and planning software determines which route best satisfies the user's goals.

• Semlab Aircraft wiki (accessible only by project participants)

Finished Projects

Information Mapping (Infomap)

• Full description available here.

Multimodal Conversational Interface (WITAS)

• Full description available here.