Recently, the leading AI and ML company Dataiku released a new version of its data science platform — Dataiku 9. With exciting new features and capabilities, Dataiku 9 will allow business analysts, novice data scientists and other business users to build AI projects. The company aims to make experimenting with AI models easier and reduce the dependency on the data science team.
From support for the popular dash application framework to what-if analysis and support for Python 3.7, Dataiku 9 comes with several new features. The ultimate goal is to involve more users across enterprises to deliver creative AI projects. The company believes as more organisations are investing in AI to make data-driven decisions, business users should be able to create AI with ease.
Opportune Moment
Most companies rely on data science teams for AI implementations. It usually requires specialists to create a series of dashboards and derive insights from data. But with the ever-growing data and AI implementations, business users are needed to dynamically create AI models that enable organisations to respond faster to rapidly changing business conditions.
In the post COVID world, more businesses are looking to adopt AI to keep up with the changing times. For this to happen, more and more users need to have expertise in data science tools, which Dataiku 9.0 will facilitate.
What’s New
Faster and transparent model deployment: The company said with Dataiku 9, business analysts and novice data scientists would have more tools to create high-quality AI and ML models. It provides them with best practices to deal with pitfalls, model assertions to capture and test use cases, what-if analysis to test model sensitivity, and more.
Easier and scalable production management: Dataiku 9 introduces a new unified deployer to move projects from design to production environments. It also enhances the programmatic access allowing for integration with CI/CD systems for project deployment and management.
Advanced capabilities for business analysts: Dataiku 9 delivers new capabilities for business teams for creating AI projects. It makes data preparation tasks easy for business analysts.
Features
The major features of Dataiku 9 include:
Unified Deployer: The new version provides unified environments for fully-managed production deployments of both projects and API services. It allows users to have a central view of the production assets.
Interactive scoring and What-if: These features allow for a better understanding of what impact can changing a feature have on the prediction by displaying in real-time the resulting prediction and individual prediction explanations.
Dash Webapps: The new version has the ability to write, deploy and manage Dash web apps that can help users with simple dashboards and full interactivity to users.
Visual ML Diagnostics: This feature allows to detect common pitfalls while training models, such as overfitting, leakage, insufficient learning and more.
Model assertions: This feature will help streamline and accelerate the model evaluation process by automatically checking that predictions for specific subpopulations meet certain conditions. It ensures the model’s predictions are aligned with the business judgment.
Distributed hyperparameters search: It is now possible to distribute the training of a single model over multiple containers. Distributed hyperparameter search permits increased depth and precision of hyperparameter search while keeping an acceptable time for training.
Model fairness report: This feature evaluates the fairness of machine learning models. It is crucial to learn about how biased a model is, and this new version measures the fairness metrics.
Some of the other features include, fuzzy join recipe to join two datasets, Wiki export to PDF, Git push and pull for notebooks such as Jupyter, smart pattern builder, experimental real-time processing framework that targets Kafka and Spark structured streaming, experimental support for directly reading the latest version of Delta Lake datasets and more.
Enhancements
Other enhancements announced are: Azure Synapse support, new capabilities for date preparation, enhanced formula editor with better code completion, support for Spark 3, which comes built-in, support for Python 3.7, built-in Snowflake driver and native Spark connector, enhanced time-based trigger, enhanced cross-connection and no-input SQL recipes, the addition of individual users to projects and more.
Dataiku will have enhanced machine learning capabilities, visual recipes along with Elastic AI and Kubernetes. It will allow users to create, delete and manage projects freely, manage datasets, list jobs, check job status, manage users and groups, global variables and more.
