Bengaluru-based Awign is a work fulfilment no-code tech platform that focuses on gig workers. Founded by Gurpreet Singh, Praveen Kumar Sah and Anannya Sarthak in 2016, Awign solves end-to-end outcome-based execution, discovery, deployment, and payroll management for gig economy players.
In November 2020, the startup raised an undisclosed amount in its pre-Series B funding round from Unitus Ventures, the Michael and Susan Dell Foundation (MSDF) and Eagle10 Ventures. Previously, a year earlier, Awign picked up USD 4 million in its Series A round from its existing backers Work10M, MSDF, Eagle10 and Unitus Ventures. Awign has recently transformed to include other verticals like SaaS, retail, warehousing and consumer product testing, among others.
Analytics India Magazine caught up with co-founder and CTO Praveen Kumar Sah to understand what went into choosing Awign’s tech stack.
AIM: What are the details of Awign’s tech stack? How did you go about deciding the tech stack? What is the journey of your tech stack along with a timeline?
Praveen: Awign has a microservice architecture, with backend services written in Java, Ruby, and Python, while our web platform is built using React and Android development is handled in Kotlin.
The first tech stack selection was made, keeping our pace of development in mind. Aside from my Ruby on Rails (ROR) expertise, it genuinely gives a methodology to launch any application in the shortest amount of time.
When we first started, most of the backend and even the webpages were developed using ROR because I am an expert in it; however, later, we began to construct backend services in Java and Python since they provide us with efficiency, ease of use, and scalability. In terms of web apps, we have moved all of our ROR projects to React.
We began focusing on data analytics in the past two years. At the time, our problem was to consider whether we needed to confirm options provided by AWS, Adobe Spark, or Hadoop, but the easiest choice we found was Google BigQuery – for all warehousing and visualisation reasons, and we solely used Google Data Studio, which was completely handled by Google Cloud.
Our data is ingested from multiple applications and consumed in our Google BigQuery, and we have our own data pipeline, which we designed using Python. It handles the task of gathering or requesting data from multiple sources and then ingesting it into Google BigQuery.
We now use a lot of analytics, and we’ve all started having data-driven conversations. We make use of LS instead of relying completely on Google Data Studio for visualisation. We have built analytics.awign.com, where teams have access to look at whatever data they want.
In terms of AI, and ML, we are still in the stage where we are relying on third-party services. We do not currently have an in-house staff, even though there’s a requirement for it. For example, we work on image-to-image transfer and OCR image recognition while also transcribing voice-to-voice-to-text. But, for the time being, we are leveraging third-party service APIs, such as Google APIs or AWS APIs, to accomplish all these tasks.
AIM: What are the main challenges that you faced while choosing your tech stack?
Praveen: While we didn’t face many challenges while selecting the tech stack, the main issue we experienced was with hiring top teammates since we use some tech stacks that aren’t very popular, such as Ruby on Rails.
Another challenge we encountered was that we couldn’t discover another company that was dealing with the same issues as us, which would have made it easier to learn from their mistakes. Since there aren’t many gig economy platforms, we had to experiment a lot with the features because we didn’t know what would succeed and what wouldn’t.
AIM: Has Awign ever migrated its tech stack? Did you face any challenges during the migration process?
Praveen: Yes, there have been some tech migrations at Awign.
- We migrated several of our backend services from Ruby on Rails to Hanami. We also launched new backend services in Java and Python instead of Ruby.
- We completely deprecated ROR in frontend technologies and experimented with Angular for a while. However, Angular did not work out for us because development was slower than using React. As a result, all of our web apps are now developed in React.
- We’ve now moved from Java to Kotlin for our Android code.
This whole exercise has been a good learning curve for us to identify the right options. We’ve been using Hanami, which isn’t a frequently used framework and caused us some difficulties at first. However, the web tech stack – React – is one of the greatest frameworks for developing web apps, so it was worth exploring.
AIM: What were the main reasons for migrating Awign’s tech stack?
Praveen: We wanted a more appealing, engaging, and intuitive user interface to engage a diverse variety of gig partners, but with basic JavaScript and HTML, we felt a little restricted. To overcome this, we needed a different framework that would grow faster. React was the right fit as it allowed us to expedite development into its bright, rapidly evolving, SEO-friendly, and solely UI-focused interface.
We migrated from Rails to Hanami since Rails is a large framework that can perform a lot of things that we didn’t require. As a result, we switched to a lightweight framework, which allowed us to speed up our APIs. Now, our API response time has been reduced from 300 milliseconds to 40 milliseconds. Kotlin provided a way to reduce the amount of code, making it easier to read and comprehend. As a result, we switched from Java to Kotlin for Android development.
AIM: How did Awign come up with its data pipeline? What comprises Awign’s data pipeline?
Praveen: Awign concentrates on Google BigQuery for all data warehousing and analytics. We built our Python data pipeline to feed data from various apps into Google BigQuery. This data ingestion occurs in near real-time.
Awign’s data is distributed across multiple databases in both SQL and NOSQL forms. As a result, when a data-driven analytical report needs to be accessed, it becomes important for us to have high data processing and performance ability. This has led us to opt for Big Data solutions. After evaluating many options, including Snowflake, AWS Redshift, and BigQuery, we determined that BigQuery was the best fit for our needs.
Once that decision was made, we needed to migrate our production data from MySQL and MongoDB databases to BigQuery.
We first considered integrating message pipelines by using Kafka and Google PubSub, but we quickly realised that we needed replaying messages, customised transformation, scheduled migrations, bulk processing, performance, cost and other use cases, which were not being resolved in our best interests. The solution was then modified to poll data from the database, alter it and then write it to BigQuery.
Our ETL pipeline connects to multiple production databases and runs scheduled jobs to read data in batches, clean and convert it before writing to BigQuery. It has multiple scheduled jobs, one for each database migration and a few for generating metadata information for projects and leads. In case there is an issue in the migration process, we store failed messages in Redisson for when we might retry migration and transfer data to BigQuery.
In addition to reading data from databases, the ETL pipeline includes APIs for logging events from different services, which are then uploaded to BigQuery. This allows Awign to get more user insights such as user journey, user actions, user behaviour and interests, which can then be leveraged to make data-driven product decisions and create more user-focused features.
AIM: How hard or easy is it to chalk out the costs of a tech stack?
Praveen: The majority of Awign’s services are hosted on AWS. We use Google Cloud, too, for several services, although currently, our monthly costs are lower than they were in 2019. After running all of the optimisations, we were able to cut our IT costs in half while our sales and users increased 20-fold. As a result, even as the business grew, we were able to maintain the cost of our tech.
AIM: What are some of the most valuable lessons that you learned while choosing your tech stack?
Praveen:
- Speed is critical for start-ups. Choose a tech stack that allows you to launch quickly, deploy, and experiment more.
- Not all fancy-sounding frameworks will be applicable. For example, we spent a lot of effort developing Kafka (a data streaming service), but maintaining it has become a task for 2-3 people. Eventually, we opted to employ a fully managed messaging and pub-sub infrastructure.
- Choosing popular tech stacks makes it easier to hire people in the long term.
