Last week,  Atlan announced that it has raised $16 million as part of its Series A round led by global venture capital and private equity firm Insight Partners. “We are excited to partner with Insight as their significant expertise in backing ScaleUp SaaS companies will help accelerate our growth. We will be investing heavily in growing our product & engineering team in India and take great pride in pioneering a new wave of companies in India that are building world-class products from India for the rest of the world,” said Varun Banka, Co-Founder of Atlan.

Founded in 2018, this Indian startup now has its offices in Singapore, Philippines, Nigeria, USA, India – for now, and is catering to customers across the globe. With an investment of over two years across 200 data projects, the company has developed its own unique product.  “Today, data assets are not just tables, but code, models, BI dashboards, and pipelines,” said Prukalpa Sankar, Co-Founder of Atlan. At Atlan, continued Prukalpa, her team is reimagining the human experience with data. “Why can’t data assets be shared as easily as sharing a link on Google Docs, or if Google Analytics can tell you usage on a website, why can’t we do the same for our data?”

DataOps: The secret sauce of Atlan

Data drives businesses growth and provides valuable insights prior to any conclusive decision making. As the enterprises scale, many challenges surface. For instance, working professionals, including data scientists, analysts, engineers, join in with different skill-sets and tools. Different people, different tools, different working styles – all these lead to a major bottleneck.  Business segments are in dire need of data management to create contextual insights, now is the time to improve the quality and speed of data streaming into the organisation and get leadership commitment to support and sustain a data-driven vision across the company. This is where DataOps (data operations) come in handy. 

For instance, users can integrate their tables from Databricks with Atlan in a series of steps. Initially there are some prerequisites for establishing a connection between Atlan and Databricks Account:

• Go to the Databricks console and select “Clusters” from the left sidebar.
• Select the cluster you want to connect with Atlan. The cluster should be in a Running state for the Atlan crawler to fetch metadata from it.
• Click on “Advanced Options” in the “Configuration” tab.
• Select the “JDBC/ODBC” tab and copy the information here:

Image credits: Atlan

• Host: Databricks cluster server hostname
• Port: Port. Typically it is 443.
• Personal Access Token: You can generate the Personal Access Token by following the official guide.
• JDBC URL Suffix: This is the JDBC URL suffix. Highlighted in the image above. Please ensure you do not add the hostname, port or PWD values here.

After these prerequisites, following steps need to be followed.

STEP 1: Selecting the source

• Log into your Atlan workspace.
• On the home screen, click on the “New Integration” button in the top right corner. You will see a dialogue box with the list of sources available on your workspace.
• Select “Databricks” from the list of options, and click on “Next”.

STEP 2: Providing credentials

• You will see an option to either select a preconfigured credential from the drop-down menu or to create a credential. To set up a new connection, click on the “Create Credential” button.
• You will be required to fill in your Databricks credentials.
• Once you have filled in the details, click on “Next”.

Image credits: Atlan

STEP 3: Setting up your configuration

• You will now be asked to fill in the details of your database and table.
• Choose whether to run the crawler once or schedule it for a daily, weekly, or monthly run. You will be asked to specify the timezone for the run.
• Click on “Create”. Your connection is now created.

DataOps should be looked at as a collaboration of people, process and technology to deliver trusted and high-quality data in a step by step process.  The information architecture of any company is at the heart of DataOps. Data curation, metadata management to data governance are few of the many other things that DataOps practices can help with.