WHAT THE GLOVE TOLD THE ROBOT

NEW ZEALAND and Croatian scientists have demonstrated how a gesture-capturing dive glove can communicate with a robot underwater, from opposite sides of the world.

The dive glove, developed by the Biomimetics Laboratory at the Auckland Bioengineering institute (ABI), is made with integrated wearable sensors and electronics, using electroactive polymers that are soft and stretchy smart materials.

As the diver performs certain gestures, a machine learning algorithm assesses the hand motion and recognises these in real-time. They are then interpreted as commands or messages and transmitted acoustically through the water to a buddy diver or robot.

The project, called ADRIATIC (Advancing Diver Robot Interaction Capabilities), began in 2018 as a collaboration between the University of Zagreb, Croatia, and the Biomimetics Lab at the ABI, funded through a grant from the Office of Naval Research in the US.

The researchers had initially planned to test their glove and how well it could communicate with a Croatian autonomous underwater vehicle (AUV) developed by the Croatian researchers in 2020. It was going to take place on the Adriatic coast of Croatia, but the researchers plans were stymied by Covid-19 travel restrictions.

The New Zealand- Croatian researchers developed an innovative alternative. On April 6 of this year New Zealand diver and ABI researcher, Chris Walker, donned his wetsuit at 6am and went underwater in the 5m diving pool at West Wave aquatic centre, in West Auckland. Meanwhile, at poolside, Derek Orbaugh, a PhD student with the lab, was in real-time computer contact with the Croatian researchers.

Wearing the glove Chris was able to use hand gestures to send commands to the AUV that was similarly submerged at the Laboratory for Underwater Systems and Technologies (LABUST) in Croatia, using acoustic signals, at 8pm Zagreb time.

That is, the glove talked to the robot on the other side of the world, using sound detected by a sonar receiver at poolside, and then transmitted to a server in Croatia. The signal was then converted back to sound transmitted to the AUV in the Croatian pool.

“We could watch it on the computer, and witness in real time, the robot moving in response to the gestures sent by Chris on the other side of the world,” says Professor Iain Anderson, head of the Biomimetics Lab. He describes this as the “The first Kiwi- Croat transglobal experiment of 2021. And there will be more.”

“We wanted to see if we could transcend the restrictions imposed by Covid on our collaboration, and we did it! But hopefully, come summer 2022, we’ll be able to continue our work together with our colleagues in Croatia in the same time zone and the warm Adriatic.”

The glove uses motion capture sensors made by New Zealand company StretchSense, a spin-out of the Biomimetics Lab. This experiment was a success, and more experiments are to follow as the glove and AUV are improved, says Prof. Anderson.

The research is ultimately aimed at improving diver safety. “I’m a diver, and while the underwater world is stunning, diving into it comes at some risk, especially if you’re on your own. Our research will improve communication diver-to-diver and diver-to-machine in a world where you often can’t see more than a couple of metres.”