Novel robot learns to play Jenga

​Boston: MIT scientists have developed a novel robot that can play a game of Jenga, combining vision and touch to carefully extract blocks without toppling the tower. In the game of Jenga, 54 rectangular blocks are stacked in 18 layers of three blocks each, with the blocks in each layer oriented perpendicular to the blocks below. The aim of the game is to carefully extract a block and place it at the top of the tower, thus building a new level, without toppling the entire structure.

The robot, described in the journal Science Robotics, is equipped with a soft-pronged gripper, a force-sensing wrist cuff, and an external camera, all of which it uses to see and feel the tower and its individual blocks. As the robot carefully pushes against a block, a computer takes in visual and tactile feedback from its camera and cuff, and compares these measurements to moves that the robot previously made. It also considers the outcomes of those moves -- specifically, whether a block, in a certain configuration and pushed with a certain amount of force, was successfully extracted or not.

In real-time, the robot then "learns" whether to keep pushing or move to a new block, in order to keep the tower from falling. "Unlike in more purely cognitive tasks or games such as chess or Go, playing the game of Jenga also requires mastery of physical skills such as probing, pushing, pulling, placing, and aligning pieces," said Alberto Rodriguez, an assistant professor at Massachusetts Institute of Technology (MIT) in the US. "It requires interactive perception and manipulation, where you have to go and touch the tower to learn how and when to move blocks," Rodriguez said.

"This is very difficult to simulate, so the robot has to learn in the real world, by interacting with the real Jenga tower. The key challenge is to learn from a relatively small number of experiments by exploiting common sense about objects and physics," he said. The tactile learning system the researchers have developed can be used in applications beyond Jenga, especially in tasks that need careful physical interaction, including separating recyclable objects from landfill trash and assembling consumer products.