AI Drone Training in Virtual Reality

The future of Drone Racing?

There have been articles published lately with the subject of Autonomous Drones.
This class or category of UAV’s, Quadcopter’s, or Drone’s is as if a nano drone meets the IoT (The Internet of Things) and AI (Artificial Inteligents).

The World’s Smallest Self-Flying AI Nano-Drone, Researchers Say
A tiny drone that can fit in the palm of your hand has been developed by researchers at ETH Zurich, Switzerland and the University of Bologna, Italy.

MIT engineers developed new virtual-reality training system for drones

There is one group that has put it’s drone into Virtual Reality or VR.
Why crash a drone in the real world if you don’t have to? The solution? AI Drone Training in Virtual Reality.
MIT engineers develop virtual-reality testing ground for drones

Flight Goggles

Enter Flight Goggles.
The engineers at MIT have developed a new virtual-reality system the enables the drone or vehicle to “see” a rich virtual world while flying in a empty real world physical space.
They have named their system “Flight Goggles”.
It is hoped that this system will greatly reduce the number of crashes and therefore time while developing the system. Kind of like drone racing training wheels.
Sertac Karaman, associate professor of aeronautics and astronautics at MIT “We think this is a game-changer in the development of drone technology, for drones that go fast, “If anything, the system can make autonomous vehicles more responsive, faster, and more efficient.”

Karaman was inspired by the new sport of competitive drone racing and wondered if an autonomous drone be able to compete with or even beat the human drone racing pilot?
Sertac Karaman “In the next two or three years, we want to enter a drone racing competition with an autonomous drone, and beat the best human player,”
According to Sertac on the need for a virtual enviroment for testing drone that need to go fast.
“The moment you want to do high-throughput computing and go fast, even the slightest changes you make to its environment will cause the drone to crash,” Karaman says. “You can’t learn in that environment. If you want to push boundaries on how fast you can go and compute, you need some sort of virtual-reality environment.”

Virtual Training System

The new virual training system consists of a motion capture system an image rendering program and electronics that quickly processes the images and tranmit the images to the drone.
The test space in which the drone actualy flies is a hanger/gymnasium locate in the new MIT drone testing facility in building 31. The space is lined with motion capture cameras that track the drones orientation as it flies.
The image rendering system enables Karaman and collegues to draw photo-realistic virtual reality worlds for the drone to fly in and then beam those virtual images to the drone as it is flying in an empty building’s space.
Sertac Karaman “The drone will be flying in an empty room, but will be ‘hallucinating’ a completely different environment, and will learn in that environment”.
The drone was able to process these virutal images at a rate of about 90 frames per second. That puts it at three times as fast as a human eye can see and process images. This processing speed was madw possible with custom built circuit boards that intergrate a powerful embedded supercomputer.
The system also includes an inertial measurement unit and camera.
The frame of the drone consists of a 3D printed nylon and carbon fiber reinforced frame.

VR Drone Training System Other Uses

The VR training system may also be used to test out new sensors, or specifications for existing sensors, to see how they might handle on a fast-flying drone.
It is also invisioned to be used in training drones to fly safely around humans. Kaaraman proposes splitting the test facility into two spaces.
In one space the drone would fly. In the other a person would be walking and interacting wearing a motion capture suit. The drone would see the human in virtual reality while flyiing in its own space. If a crash happens it has happened in the virtual world.
Or as the say in sports “no harm, no foul”.
“One day, when you’re really confident, you can do it in reality, and have a drone flying around a person as they’re running, in a safe way,” Karaman says.

Karaman and his colleagues will present details of their virtual training system at the IEEE International Conference on Robotics and Automation next week. Co-authors include Thomas Sayre-McCord, Winter Guerra, Amado Antonini, Jasper Arneberg, Austin Brown, Guilherme Cavalheiro, Dave McCoy, Sebastian Quilter, Fabian Riether, Ezra Tal, Yunus Terzioglu, and Luca Carlone of MIT’s Laboratory for Information and Decision Systems, along with Yajun Fang of MIT’s Computer Science and Artificial Intelligence Laboratory, and Alex Gorodetsky of Sandia National Laboratories.

Karaman says. “There are a lot of mind-bending experiments you can do in this whole virtual reality thing. Over time, we will showcase all the things you can do.”

This research was supported, in part, by U.S. Office of Naval Research, MIT Lincoln Laboratory, and the NVIDIA Corporation.

Be sure to visit the Racing Drone World YouTube channel and give us a like, comment, and or subscribe to our channel.
Follow us on Twitter

AI Drone Training in Virtual Reality