Towards A Role Ethics Approach to Command Rejection

Ruchen Wen and Ryan Blake Jackson and Tom Williams and Qin Zhu

HRI Workshop on The Dark Side of Human-Robot Interaction: Ethical Considerations and Community Guidelines for the Field of HRI

It is crucial for robots not only to reason ethically, but also to accurately communicate their ethical intentions; a robot that erroneously communicates willingness to violate moral norms risks losing both trust and esteem, and may risk negatively impacting the moral ecosystem of its human teammates. Previous approaches to enabling moral competence in robots have primarily used norm-based language grounded in deontological ethical theories. In contrast, we present a communication strategy grounded in role-based ethical theories, such as Confucian ethics. We also present a human subjects experiment investigating the differences between the two approaches. Our preliminary results show that, while the role-based approach is equally effective at promoting trust and conveying the robot’s ethical reasoning, it may actually be less effective than the norm-based approach at encouraging certain forms of mindfulness and self-reflection.