Robot Learning to Communicate through Projected Visual Abstractions

Robots typically communicate only through physical movement, unlike humans who also use shadows and silhouettes. This work builds a robotic system with a 21-degree-of-freedom soft-skinned hand and a learned shadow self-model that maps hand configurations to projected shadow appearance. Given a target shadow image or video, the robot optimizes hand poses through gradient-based search and collision-aware simulation to produce physically feasible, expressive shadows, demonstrated on sign language, shadow puppetry, and animal imitation. Applications include expressive human-robot interaction, assistive and educational robotics, entertainment and storytelling robotics, and broader research into non-morphological robot communication channels. Authors: Danyang Yan, Boyuan Wang, Jiaxun Liu, Boyuan Chen Paper: https://arxiv.org/abs/2607.22434v1