Uber AI is at the heart of AI-powered innovation and technologies at Uber. The research and advancements in artificial intelligence made by the team solve several challenges across the company. For example:
- Uber AI develops packages and open sources them which are built on top of the strong foundations of other open source libraries
- Pyro, a deep probabilistic programming language built on PyTorch, released by Uber in 2017
- Another major open source AI tool Horovod, a framework hosted by the LF Deep Learning Foundation that allows distributed training of deep learning models over multiple GPUs and several machines
At Uber, deep learning models are used for a variety of tasks like customer support, object detection, improving maps, streamlining chat communications, forecasting, and preventing fraud.
Named after one of the greatest physicists, Ludwig Boltzmann, Uber’s new tool Ludwig is poised to make deep learning more accessible with code-free model training.
Increased Deep Learning Accessibility With Ludwig
Ludwig was developed internally at Uber over the past two years to simplify the use of deep learning models in applied projects.
Multi-task learning can be performed along with learning to predict all the outputs simultaneously, a task that usually requires custom code.
Default values of preprocessing, training, and various model architecture parameters are chosen or are adapted from the academic literature, allowing non-experts to easily train complex models.
The new idea that Uber AI introduces with Ludwig is the notion of data-type-specific encoders and decoders, which results in a highly modularized and extensible architecture: each type of data supported (text, images, categories, and so on) has a specific preprocessing function. In short, encoders map the raw data to tensors, and decoders map tensors to the raw data.
The researchers at Uber AI say that each data type may have more than one encoder and decoder. For instance, text can be encoded with a convolutional neural network (CNN), a recurrent neural network (RNN), or other encoders.
The user can then specify which one to use and its hyperparameters directly in the model definition file without having to write a single line of code.
How Is Ludwig Different From Other Open Source AI Toolboxes
Ludwig is built on top of TensorFlow that allows users to train and test deep learning models without writing code.
In this way, Ludwig is unique in its ability to help make deep learning easier to understand and enable faster model improvement iteration cycles for experienced non-experts and researchers alike.
- Tool Integration: Ludwig has a mix of influences from Weka, scikit-learn, and Mlib. Ludwig is a different tool from the usual deep learning libraries that provide tensor algebra primitives and few other utilities to code models, while at the same time making it more general than other specialized libraries like PyText, StanfordNLP, AllenNLP, and OpenCV
- No coding required: No coding skills are required to train a model and use it for obtaining predictions
- Extensibility: Easy to add new model architecture and new feature data types
Deploying Ludwig In 4 Easy Steps
Install
Just run pip install ludwig and it will be ready to use. Some features may require further steps, read Getting Started.
Train
Prepare data in a CSV file, define input and output feature in a model definition YAML file and run:
ludwig train
--data_csv file.csv
--model_definition definition.yaml
Predict
Prepare data in a CSV file and use a pre-trained model to predict the output targets:
ludwig predict
--data_csv data.csv
--model path_to_model
Visualize
Ludwig comes with many visualization options.
ludwig visualize
--visualization learning_curves
--training_statistics train_statistics.json
Know more about Ludwig here.
Future Direction
Ludwig incorporates a set of command line utilities for training, testing models, and obtaining predictions. Furthering its ease-of-use, the toolbox provides a programmatic API that allows users to train and use a model with just a couple lines of code.
By open-sourcing Ludwig, Uber has made AI more accessible and with this, there will be a torrential inflow of ideas and innovations from outsiders who have been left out due to hurdles like coding skills.
