Advertisement

Active Hackathon

Here’s How Uber’s Ludwig Helps Deploy Code-Free DL Models

 

THE BELAMY

Sign up for your weekly dose of what's up in emerging technology.

Uber AI is at the heart of AI-powered innovation and technologies at Uber. The research and advancements in artificial intelligence made by the team solve several challenges across the company. For example:

  • Uber AI develops packages and open sources them which are built on top of the strong foundations of other open source libraries
  • Pyro, a deep probabilistic programming language built on PyTorch, released by Uber in 2017
  • Another major open source AI tool Horovod, a framework hosted by the LF Deep Learning Foundation that allows distributed training of deep learning models over multiple GPUs and several machines

At Uber, deep learning models are used for a variety of tasks like customer support, object detection, improving maps, streamlining chat communications, forecasting, and preventing fraud.

Named after one of the greatest physicists, Ludwig Boltzmann, Uber’s new tool Ludwig is poised to make deep learning more accessible with code-free model training.

Increased Deep Learning Accessibility With Ludwig

Ludwig was developed internally at Uber over the past two years to simplify the use of deep learning models in applied projects.

Multi-task learning can be performed along with learning to predict all the outputs simultaneously, a task that usually requires custom code.

Default values of preprocessing, training, and various model architecture parameters are chosen or are adapted from the academic literature, allowing non-experts to easily train complex models.

The new idea that Uber AI introduces with Ludwig is the notion of data-type-specific encoders and decoders, which results in a highly modularized and extensible architecture: each type of data supported (text, images, categories, and so on) has a specific preprocessing function. In short, encoders map the raw data to tensors, and decoders map tensors to the raw data.

The researchers at Uber AI say that each data type may have more than one encoder and decoder. For instance, text can be encoded with a convolutional neural network (CNN), a recurrent neural network (RNN), or other encoders.

The user can then specify which one to use and its hyperparameters directly in the model definition file without having to write a single line of code.

How Is Ludwig Different From Other Open Source AI Toolboxes

Ludwig is built on top of TensorFlow that allows users to train and test deep learning models without writing code.

In this way, Ludwig is unique in its ability to help make deep learning easier to understand and enable faster model improvement iteration cycles for experienced non-experts and researchers alike.

  • Tool Integration: Ludwig has a mix of influences from Weka, scikit-learn, and Mlib. Ludwig is a different tool from the usual deep learning libraries that provide tensor algebra primitives and few other utilities to code models, while at the same time making it more general than other specialized libraries like PyText, StanfordNLP, AllenNLP, and OpenCV
  • No coding required: No coding skills are required to train a model and use it for obtaining predictions
  • Extensibility: Easy to add new model architecture and new feature data types

Deploying Ludwig In 4 Easy Steps

Install

Just run pip install ludwig and it will be ready to use. Some features may require further steps, read Getting Started.

Train

Prepare data in a CSV file, define input and output feature in a model definition YAML file and run:

ludwig train
--data_csv file.csv
--model_definition definition.yaml

Predict

Prepare data in a CSV file and use a pre-trained model to predict the output targets:

ludwig predict
--data_csv data.csv
--model path_to_model

Visualize

Ludwig comes with many visualization options.

ludwig visualize
--visualization learning_curves
--training_statistics train_statistics.json

Know more about Ludwig here.

Future Direction

Ludwig incorporates a set of command line utilities for training, testing models, and obtaining predictions. Furthering its ease-of-use, the toolbox provides a programmatic API that allows users to train and use a model with just a couple lines of code.

By open-sourcing Ludwig, Uber has made AI more accessible and with this, there will be a torrential inflow of ideas and innovations from outsiders who have been left out due to hurdles like coding skills.

 

More Great AIM Stories

Ram Sagar
I have a master's degree in Robotics and I write about machine learning advancements.

Our Upcoming Events

Conference, Virtual
Genpact Analytics Career Day
3rd Sep

Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep

Conference, in-person (Bangalore)
Machine Learning Developers Summit (MLDS) 2023
19-20th Jan

Conference, in-person (Bangalore)
Data Engineering Summit (DES) 2023
21st Apr, 2023

3 Ways to Join our Community

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Telegram Channel

Discover special offers, top stories, upcoming events, and more.

Subscribe to our newsletter

Get the latest updates from AIM
MOST POPULAR
How Data Science Can Help Overcome The Global Chip Shortage

China-Taiwan standoff might increase Global chip shortage

After Nancy Pelosi’s visit to Taiwan, Chinese aircraft are violating Taiwan’s airspace. The escalation made TSMC’s chairman go public and threaten the world with consequences. Can this move by China fuel a global chip shortage?

Another bill bites the dust

The Bill had faced heavy criticism from different stakeholders -citizens, tech firms, political parties since its inception

So long, Spotify

‘TikTok Music’ is set to take over the online streaming space, but there exists an app that has silently established itself in the Indian market.

[class^="wpforms-"]
[class^="wpforms-"]