Image & Video Quality Assessment at LIVE
The field of image and video processing generally deals with signals that are meant for human consumption, such as images or videos over the Internet. An image or video may go through many stages of processing before being presented to a human observer, and each stage of processing may introduce distortions that could reduce the quality of the final display. For example, images and videos are acquired by camera devices that may introduce distortions due to optics, sensor noise, color calibration, exposure control, camera motion etc. After acquisition, the image or video may further be processed by a compression algorithm that reduce the bandwidth requirements for storage or transmission. Such compression algorithms are generally designed to achieve greater savings in bandwidth by letting certain distortions happen to the signal. Similarly, bit errors, which occur while an image is being transmitted over a channel or (rarely) when it is stored, also tend to introduce distortions. Finally, the display device used to render the final output may introduce some of its own distortion, such as low reproduction resolution, bad calibration etc. The amount of distortion that each of these stages could add depends mostly on economics and/or physical limitations of the devices.
One is obviously interested in being able to measure the quality of an image or video, and to gauge the distortion that has been added to it during different stages. One obvious way of determining the quality of an image or video is to solicit opinion from human observers. After all, these signals are meant for human consumption. However, such a method is not feasible not only due to the shear number of images and videos that are "out there" but also because we want to be able to embed quality measurement techniques into the very algorithms that process images and videos, so that their output quality may be maximized for a given set of resources.
The goal of research in objective image quality assessment is to develop quantitative measures that can automatically predict perceived image quality. Generally speaking, an objective image quality metric can play an important role in a broad range of applications, such as image acquisition, compression, communication, displaying, printing, restoration, enhancement, analysis and watermarking. First, it can be used to dynamically monitor and adjust image quality. Second, it can be used to optimize algorithms and parameter settings of image processing systems. Third, it can be used to benchmark image processing systems and algorithms.
In short, objective quality measurement (as opposed to subjective quality assessment by human observers) seeks to determine the quality of images or videos algorithmically. The goal of objective quality assessment (QA) research is to design algorithms whose quality prediction is in good agreement with subjective scores from human observers.
Image and video QA algorithms may be classified into three broad categories:
Full-Reference (FR) QA methods, in which the QA algorithm has access to a 'perfect version' of the image or video against which it can compare a 'distorted version'. The 'perfect version' generally comes from a high-quality acquisition device, before it is distorted by, say, compression artifacts and transmission errors. However, the reference image or video generally requires much more resources than the distorted version, and hence FR QA is generally only used as a tool for designing image and video processing algorithms for in-lab testing, and cannot be deployed as an application .
- No-Reference (NR) QA methods, in which the QA algorithm has access only to the distorted signal and must estimate the quality of the signal without any knowledge of the 'perfect version'. Since NR methods do not require any reference information, they can be used in any application where a quality measurement is required. However, the price paid for this flexibility is in terms of the ability of the algorithm to make accurate quality predictions, or a limited scope of the NR QA algorithm (such as NR QA for JPEG images only etc.).
- Reduced-Reference (RR) QA methods, in which partial information regarding the 'perfect version' is available. A side-channel (called an RR channel) exists through which some information regarding the reference can be made available to the QA algorithm. RR QA algorithms use this partial reference information to judge the quality of the distorted signal.