Transparency Mechanisms to Improve Robot Failure Identification
Goal
Test if robot transparency mechanisms improve people's mental models of the robot
Research Questions
- Do transparency mechanisms help users identify the cause of the robot's failure?
- What is the effect of transparency mechanisms on the user's mental model of the robot?
Tasks of Focus
12 different failure scenarios in a simulated tabletop robot setup
Methods
Usability testing
Analysis Methods
Statistical Methods, Inductive Thematic Analysis
Population
64 participants recruited on Amazon Mechanical Turk
Select study results
- Participants were more accurate at identifying the robot's failure when they used question-answering, or both question-answering and visual mechanisms compared to when they had no access to transparency mechanisms.
- Even when participants had access to transparency mechanisms, even though their mental models of the robot were better, they were still not accurate.
Paper
Preeti Ramaraj, Saurav Sahay, Shachi H. Kumar, Walter S. Lasecki, and John E. Laird. "Towards using transparency mechanisms to build better mental models." In Advances in cognitive systems: 7th goal reasoning workshop, vol. 7, pp. 1-6. 2019.