“Unity is our mission to power the frontier of AI”- Danny Lange, Senior VP, AI & Ml at Unity Technologies
Recently, Unity held its first-ever AI summit, known as the Unity AI Summit 2020. The summit was a one-day event for machine learning developers, researchers as well as practitioners to learn about the technological innovations at Unity and their real-world applications in the field of machine learning and computer vision.
The free summit included two important tracks — Industrial track and Games track, that featured keynote speeches including several panel discussions, and hands-on workshops and presentations. The event stressed discussions on synthetic data generation, spatial simulation, automated playtesting, ML-Agents, robotics, among others.
Danny Lange, Senior Vice President, AI & ML at Unity Technologies kickstarted the event with his keynote speech — Simulation The New Reality for AI — where he mentioned about the AI technologies and product information that was going to be discussed further by the Unity team during the event.
During the tracks, Lange explained that there are four dimensions to AI — visual, physical properties, cognitive and social aspects that help in crucial development of AI. Further, Lange discussed the Unity Real-time 3D engine that includes a spatial environment, graphical rendering system, a physics engine and multi-sensory.
There were other interesting sessions at the industrial track, such as embracing AI to build the next-gen games, use reinforcement learning in games with ML agents, AI-assisted artistry for 3D material creation, unity perception, among others.
The Industrial track session sheds light on a brief look into 2021. It includes:
- An AI-ready content database and environment generators
- Labellers for additional computer vision tasks
- Integration of different sensor models
- New simplified interface for dataset generation
The Games Track session covers the importance of using ML-Agents and Unity Engine and its usage in games. Currently, in beta version, Unity Machine Learning agents are open-source machine learning product offerings that train intelligent agents with reinforcement learning and evolutionary methods via a simple Python API.
ML-Agents is an open-source project that facilitates games as well as simulations to serve as environments for training intelligent agents. It includes a C# software development kit in order to set up a scene and defining the agents within it. It also consists of an ML library in order to train agents for 2D, 3D as well as VR/AR environments. The latest release of ML-Agents includes significant features and improvements that include:
- The Match3 environment, which uses the new utilities added in com.unity.ml-agents.extensions.
- PyTorch trainers, which are now made default. However, the use of TensorFlow is still available, and one can use the TensorFlow backend by either adding –tensorflow to the CLI or adding the framework, tensorflow, in configuration YAML.
- Utilities, in order to make it easier to integrate with match-3 games.
- The action_probs node is no longer listed as an output in TensorFlow models.
While, in 2017, the first version of Unity Machine Learning Agents Toolkit (ML-Agents) was released, in May this year, the team announced ML-Agents Unity package version 1.0, that provides:
- API stability: With massive advancements in ML-Agents C# SDK, it is now capable of providing a flexible, feature-rich, and stable API that is easy to integrate into any game or Unity environment.
- Ease of installation: One can now get started with the ML-Agents Unity package directly from the Package Manager without the requirement of cloning the GitHub project.
- Verified Unity package: The ML-Agents Unity package is also going to be a verified package for the 2020.2 version of the Unity Editor.