Guide To Clarifai - The End To End Platform For AI Lifecycle

Business applications now need a one-stop solution for the entire AI-assisted projects. Starting from data gathering to building the appropriate training dataset to model building, validating and evaluating over various test cases and deployment. With advancements in deep learning over these years, transfer learning has gained preference and helped in automating a lot of stuff for large training datasets. 

Visual content is the most widely used medium in today’s world. To maintain this visual content, manual work can be very tedious. Thus to handle such heavy loaded work automated ML assisted systems are built that provide end to end solutions.

Today we will be discussing one such platform named Clarifai that provides solutions on computer vision(images and videos) and NLP(text) use cases. 

Clarifai

Clarifai is an AI-powered platform that specializes in computer vision solutions. With only a few steps the entire business solution can be built. Founded in 2013, by Matthew Zeiler, Clarifai is headquartered in New York City, USA. Ever since its launch, Clarifai has been a market leader due to its winning solution in the 2013 Imagenet Challenge classification section among the top 5 spots. Clarifai works mainly using convolutional neural networks. 

Some of the benchmark from Clarifai on multiple models trained on the original data plus an additional model trained on 5000 categories, multiple models trained on the original data plus an additional model trained on other 1000 category data, an average of multiple models on original training data, another attempt at multiple models on original training data and a single model trained on original data are 0.11197, 0.11537, 0.11743, 0.1215, 0.12535 respectively. 

Initially, Clarifai had only the API as its service, which offered both free and paid versions. In 2016 they added the custom training and visual search feature into the platform. In 2017 they launched a mobile SDK to run their platform locally on their devices. In 2018 Clariai released their on-premise solution. Provides 1000 free operations a month.

Features

Automatic Tagging - shows all the relevant classes and subclass categories the image falls under.

Content Moderation - Provides all relevant information related to the image using pre-trained deep neural network architecture to predict

Visual Search

Workflow Management

Upload image, video or text data onto the platform in the mentioned formats and start with task descriptions to your projects and then share with team members. They can then start annotating with the efficient tools and features present in the platform. A review will then generate and objects that were not annotated properly can be checked. Model is ready to be trained. Clarifai has a model gallery which pre-trained models which can be used to get your work done. After the model is trained it can be evaluated and misclassifications can be checked and given to the model again for retraining. Finally certain metrics Precision ROC and co-relation matrices provide the model performance which is monitored and improved if required.  

Model Gallery

NLP - text classification, multilingual classification, text moderation, visual text generation. Automate document analysis, gain market intelligence, monitor user comment.

Other features include ready to use models such as people and vehicle detector, demographics(predict age, gender and culture of people by faces), food, colour and textual detectors, face embeddings, text embeddings, NSFW(not safe for work) these are uncensored contents which people would not like to generally view in between their work. Models detecting travel, wedding concepts. Image moderations. 

In 2018, Clarifai claimed to recognise over 11000 different concepts from objects, like mood or theme.

Industry Use Cases

• E-Commerce - use of visual search for better interactivity, use of chatbots(NLP) for better customer experience.

• Brick and mortar - Retail industry improvement with AI monitoring on customer activity

• Aviation - analyse check-in facilities with video surveillance, better customer interface using NLP, speed up check-in formalities with facial recognition

• Tourism - Automate check-in and check-out process, employee access control to make sure employees are in their designated areas. Assign loyalty points by detecting a VIP client. Get customer insights and preferences. Make your brand safe with NSFW.

• Digital Asset Management - helping users tag unstructured data in videos, images, and texts and search more deeply based on media content, colour, location, abstract concepts, and more.

• Streaming Services -  enhanced recommendation system using customer experience. Regulate content to ensure copyright issues.

• Public Sector - use cases in army/military, medical devices and drone surveillance.

• Insurance - vehicle damage assessment, property assessment, underlying risk factors in documents, natural damage assessment.

API Supports

The gRPC clients are auto-generated thus providing the latest features of Clarifai are present. Available are Clarify gRPC Python, Clarify gRPC Java, Clarify gRPC NodeJS, Clarify gRPC C#, Clarify gRPC PHP.

For python:

pip install clarifai-grpc

Setup Client connection

from clarifai_grpc.channel.clarifai_channel import ClarifaiChannel
from clarifai_grpc.grpc.api import resources_pb2, service_pb2, service_pb2_grpc
from clarifai_grpc.grpc.api.status import status_pb2, status_code_pb2
# Construct one of the channels you want to use
channel = ClarifaiChannel.get_json_channel()
channel = ClarifaiChannel.get_insecure_grpc_channel()
# Note: You can also use a secure (encrypted) ClarifaiChannel.get_grpc_channel() however
# it is currently not possible to use it with the latest gRPC version
stub = service_pb2_grpc.V2Stub(channel)
# This will be used by every Clarifai endpoint call.
metadata = (('authorization', 'Key {YOUR_CLARIFAI_API_KEY}'),)

Manually-built Clients - Clarify Python, Clarify Java, Clarify JavaScript, Clarify C#, Clarify PHP.

For python:

pip install clarifai

Initialize API key to authorize client and predict on image

from clarifai.rest import ClarifaiApp
app = ClarifaiApp(api_key='YOUR_API_KEY')
model = app.public_models.general_model
response = model.predict_by_url(url='https://samples.clarifai.com/metro-north.jpg')

Customers

Brandwatch, OpenTable, Tatcha, Trivago, Widen, Tradesy, Stocksy, Othon, Staples, Extensis, Canva, Unilever, BuzzFeed, Ubisoft. In 2017 a report was released which said Clarifai has over 100 customers in different businesses and industries.