New robot vision system grasps glass, shiny objects without depth sensors

Glass bottles slip through robotic fingers, and shiny metal parts confuse warehouse automation systems daily across factories worldwide. Now researchers at Tokyo University of Science have cracked this longstanding puzzle with a vision system called HEAPGrasp that relies solely on object outlines rather than troublesome depth sensors.

The breakthrough sidesteps the optical chaos that transparent and reflective materials create for traditional 3D sensors. Using just a standard RGB camera, the system captures silhouettes from multiple angles and reconstructs precise 3D shapes, even when dealing with the trickiest materials.

"Traditionally, transparent or mirrored objects have been unstable to detect when using depth sensors, making automatic grasping by robots difficult and ultimately leading to human intervention," explained researcher Shogo Arai.

Testing across 20 scenarios with mixed materials, HEAPGrasp achieved a 96 percent success rate while cutting camera movement by 52 percent and reducing execution time by 19 percent. The system works with existing robotic hardware, making it readily deployable in logistics, food handling, and manufacturing where robots currently struggle with everyday items like plastic containers and metal components.