Image recognition is used in numerous applications across verticals. And one of the most prominent users for them are stock websites. Shutterstock is the most prominent example of this, that adopted artificial intelligence in their operations late last year.
Their new feature known as Shutterstock Showcase heavily uses AI to perform a variety of tasks that function on the basis of image analysis. To understand how image recognition can be applied to this field, we must first look into how image recognition on the website functions.
A Unique Solution For A Unique Problem
At its heart, image recognition is applying metadata to data that does not have a structure. While many methods have been tried in the past, convolutional neural networks have emerged as a top player in this space. CNNs correlate proximity with similarity, which is a natural fit for image recognition. By further filtering CNN connections by proximity, they provide an efficient and accurate result. This is what Shutterstock aimed to achieve.
Shutterstock is one of the oldest stock image sites on the Internet today, with over 13 years of collected photos. Reportedly, the team went through multiple models before being satisfied with one, testing them over a period of months.
This was due to the vast variety of content on the website, but also due to a more arbitrary reason. As always, humans remained the weakest link, with search queries tending towards abstract concepts or ideas. Moreover, a lot of designers used the service, leading to a need for images with enough white space to insert text or logos.
The inference step was performed on all images in the site’s database, which also included the newer images which were being added on the platform. This was used to create a fingerprint of images on the site that captured their structure.
This was then followed by grouping the fingerprints together and running them through a process called the ‘nearest neighbour search index’. This allowed the researchers to find the closest examples in the library of stock photos.
The Model To Redefine Image Search
Shutterstock has also long been at the forefront of using computer vision technology to provide better search results. This was the culmination of their efforts to make a catch-all image search algorithm.
As mentioned previously, the concepts of language were one of the biggest points of uncertainty 0in the search process. This is why they tweaked their algorithm to treat pixel data inside the images as its language, so as to recognise them more accurately.
The process involves breaking down the image to its principal features, thus recognising what is inside the image as opposed to what the image was made of. This includes characteristics such as shapes, colours, and other minute details.
Using AI, the website was able to implement a reverse image search that found similar images based on the look and feel. This is available in text form as well, with the user selecting the most relevant results to deepen the search. The handy feature also made it into a Chrome extension called Shutterstock Reveal, which finds a similar royalty-free photo to the search query in seconds.
The technology was also extended to include composition search algorithms, which allowed users to place keywords in a canvas to determine the placement of image objects in the frame.
In a boon to designers everywhere, they also created a utility to allow users to pick exactly where and how much white space they needed for inserting text. This feature, known as Copy Space, was released alongside the other features for use by everyone.
The Democratisation Of Easily Accessible CV Algorithms
Shutterstock then proceeded to make this available for purchase by end users, allowing them to utilise the API for this service to allow for the service to be used on any site. This is indicative of the larger growth that is seen in the content creation space for better image recognition algorithms.
The services can also be used by many other types of companies, with other products also being available on the market. A similar service is Clarifai, which also provides image recognition services on an API call. These algorithms can be used in various applications, including providing feature fit images for creative design companies, e-commerce for clothing and identifying human movement for retail analytics.
Computer vision is one of the fastest evolving fields in deep learning today, with many giants such as Amazon and Google duking it out over the leadership role in the field. However, smaller companies such as Shutterstock and Clarifai provide an accessible, light and easy image recognition algorithm, which is sure to increase their adoption.
