The aesthetic and technical quality of images is an important factor for providing users a better visual experience. Image quality assessment (IQA) enables models to build a relationship between an image and a user’s perception of its quality. Many IQA approaches have achieved success by leveraging the idea of convolutional neural networks (CNNs).  Moreover, IQA models based on CNNs are constrained by the fixed-size input requirements in batch training, i.e., the input images that need to be cropped or resized to a fixed size shape. 

Addressing the issue, researchers at Google have introduced ‘Multi-scale Image Quality Transformer (MUSIQ)’ with an aim to bypass constraints of CNN on fixed input size to predict effective image quality on native-resolution images.

The paper was published at ICCV 2021, where the model supports the processing of full-size image inputs with varying resolutions and aspect ratios. This would further allow a multi-scale feature extraction to capture image quality at different granularities. 

“We apply MUSIQ on four large-scale IQA datasets, demonstrating consistent state-of-the-art results across three technical quality datasets (PaQ-2-PiQ, KonIQ-10k, and SPAQ) and comparable performance to that of state-of-the-art models on the aesthetic quality dataset AVA,” the research said. 

Source: Google AI

The proposed model tackles the challenges of learning IQA on full-size images. Unlike CNN-models constrained to fixed resolutions, MUSIQ handles inputs with arbitrary aspect resolutions. 

Initially, a multi-scale representation of the input image was generated, which contained both  the native resolution image and resized variants. To preserve the image composition, the researchers maintained its aspect ratio during resizing. The partitions of the images were then generated into fixed-size patches at different scales, which were fed into the model.

Mainly designed for IQA, the model can be applied to scenarios with task labels that are sensitive to aspect ratio and image resolution. Moreover, the MUSIQ model and checkpoints are made available at the GitHub repository.
To know more about the model, read the whole blog here.