Virtual reality was imagined as a human simulation technology long before the most recent wave of innovation that brought us the Oculus RIFT and the wave of innovation that followed. Now, rendering high framerate graphics using multiple, stereoscopic points in virtual reality is matching the speed and accuracy of robotic sensors and cameras. By modeling physics, motion, and material interactions, virtual reality is poised to become a simulation tool for training automatons - robots, drones, and diagnostic gear - before they need to perform in the real world. That's one small step for robotics, but it foretells a much bigger step forward for artificial intelligence.
Recent advancements point to a potentially disruptive combination of virtual reality and artificial intelligence which will unlock a future with safe and competent intelligent machines, able to learn exponentially through self training and intelligent, realistic simulations. Ongoing academic work in machine learning and virtual reality have been migrating to corporations and startups through open source initiatives and movement of skilled people through the academic, startup, and corporate workplaces. We are beginning to see how the mingling of these people and technologies might combine VR and machine learning to create a force more disruptive than either alone.
This week, NVIDIA announced a cloud-based virtual reality simulator that uses accurate physics modeling to simulate real world environments. This "hyper reality" system is well suited to train robots to function in simulated environments. Previously, NVIDIA had demonstrated the use of VR input for training drones, using simulated visual input and testing the accuracy of navigation. Stereoscopic simulated visuals were allowed the drone to use visual 3D position algorithms to maintain accurate position and navigation. This test was early evidence that drones and self driving cars might soon learn advanced navigation with a combination of real world environments and virtual reality visuals. These virtual environments can be purposefully and increasingly challenging for critical applications, able to train a self-driving car to drive in an area full of simulated people, or a robot to respond to complex challenges and variances before being placed on a real assembly line.
OpenAI, a think tank founded by Elon Musk, announced in August that the team had developed and trained a machine learning agent - a neural network - to play Valve's real time strategy game DOTA II. This agent was trained using a view of the screen as visual input to the network, much as a human player would interact with the game. However, by hacking the game to run in the cloud and render to the vision system of a machine learning agent, the development team was able to train the agent through self play - playing itself over and over again, faster than real-time, and in the cloud. Once ready for human players, the machine player evolved from really good to "unbeatable," over the course of a week, defeating some of the best players in the world. Without the benefit of years of context in how games are played, or any notion of strategy or tactics, the agent learned only from its own successes and failures in the structured environment of the interactive game.
Machine Learning Renaissance
Machine learning is experiencing a renaissance as innovations create new building blocks for creating sophisticated networks. Three major advancements have included:
Learning to learn, also called learning to reinforcement learn, is a family of evolving techniques employing multi-layered networks where lower levels learn about patterns and discern which patterns matter, and higher levels devise strategy given an understanding of how patterns interact.
Adversarial training, where one network is trained to train another network. In a hallmark study, one network was trained to recognize a photorealistic image of a bedroom, and another to generate such images, These networks were then connected in a single system generating superior images of a bedroom.
Convolutional neural networks, which improved computer vision training by creating pattern matching competencies in neural networks. These solutions began with hard wiring for pattern matching and filtering but are likely to evolve into systems which learn to see in new ways through self-learning techniques.
These methods become building blocks, and interconnecting these building blocks helps developers to write increasingly capable, programmable machine learning agents. Elon Musk has described the effect of multiple exponentials - a rise in machine learning manpower meeting increasingly capable algorithms and hardware. Faster-than-real-time automated training combined with intelligent, virtual reality simulators may be the last exponential. Good machine learning models will gain access to the flood of interactive training data needed to become expert learning systems and ultimately useful, intelligent agents.
The use of virtual reality and cloud based simulation engines promises to train automatons in real-world scenarios through simulated trial and error without endangering real people and real things. Developers will create VR simulations in the cloud rather than on a PC, able to run many simultaneous instances speeding the training of machine learning agents.
A foundation for these advancements was the open-sourcing of significant pieces of AI technology from academia, the open-source machine learning community, and technology powerhouses including Google and Facebook. At the same time, game engines have created relatively open architectures that have allowed fast innovation in the construction of virtual reality technologies. Bringing these technologies together is the next step, and over the next two years we anticipate that game engines will be able to execute simulations in the cloud, and AI agents will advance to visually process and learn in VR . Developers will use these tools to create a new training ground for machine learning agents and unlock exponential and disruptive advancements.