(a) Background

Recently, it has become increasingly important to develop image and video quality/distortion measures that help to evaluate, compare and improve image the video qualities. Currently, the only widely accepted method is the subjective measurement of mean opinion score (MOS). MOS is very tedious, expensive and slow. In addition, it is very difficult to be embedded into a practical image and video processing system because of its impossibility of automatic implementation. Instead, an objective image and video quality assessment method can automatically evaluate image and video qualities. An appropriate objective measurement metric can serve as a useful tool for the design and optimization of various image and video compression (both at the encoder and the decoder), enhancement and restoration systems.

(b) A Universal Image Quality Index

There are basically two classes of objective quality or distortion assessment approaches. The first are mathematically defined measures such as the widely used mean squared error (MSE) and peak signal to noise ratio (PSNR). The second class of measurement methods incorporate human visual system (HVS) characteristics in an attempt to incorporate perceptual quality measures. Unfortunately, none of these complicated objective metrics in the literature has shown any clear advantage over simple mathematical measures such as PSNR under strict testing conditions and different image distortion environments (see VQEG page).

Mathematically defined measures are still attractive because of two reasons. First, they are easy to calculate and usually have low computational complexity. Second, they are independent of viewing conditions and individual observers. Although it is believed that the viewing conditions play important roles in human perception of image quality, they are, in most cases, not fixed and specific data is generally unavailable to the image analysis system. If there are N different viewing conditions, a viewing condition-dependent method will generate N different measurement results that are inconvenient to use. In addition, it becomes the user's responsibilities to measure the viewing conditions and to calculate and input the condition parameters to the measurement systems. By contrast, a viewing condition-independent measure delivers a single quality value that gives a general idea of how good the image is.

We propose a mathematically defined universal image quality index. By "universal", we mean that the quality measurement approach does not depend on the images being tested, the viewing conditions or the individual observers. More importantly, it must be applicable to various image processing applications and provide meaningful comparison across different types of image distortions. Currently, the PSNR and MSE are still employed "universally", regardless of their questionable performance. This work attempts to develop a new index to replace their roles.

Our experiments on various image distortion types show that the proposed quality index exhibits surprising consistency with subjective quality measurement. It performs significantly better than MSE or PSNR.

For demo images and free software download, click here.

Demo 1: "Lena" image with different types of distortions

Demo 2: "Goldhill" image with different types of distortions

Demo 3: "Couple" image with different types of distortions

Demo 4: "Tiffany", "Lake" and "Mandrill" images with JPEG compression

Demo 5: "Woman", "Man" and "Barbara" images with blurring

Relevant Publications

Z. Wang, and A. C. Bovik, "A universal image quality index," IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81-84, March 2002.

(c) Foveated Wavelet Image Quality Index (FWQI)

Because of foveation, the human visual system (HVS) is highly non-uniform in sampling, coding, processing and understanding. Currently, most image quality measurement methods are designed for uniform resolution images. These methods do not correlate well with the perceived foveated image quality. Wavelet analysis delivers a convenient way to simultaneously examine localized spatial as well as frequency information. We developed a new image quality metric called foveated wavelet image quality index (FWQI) in the wavelet transform domain. FWQI considers multiple factors of the HVS, including the space variance of the contrast sensitivity function, the spatial variance of the local visual cut-off frequency, the variance of human visual sensitivity in different wavelet subbands, and the influence of the viewing distance on the display resolution and the HVS features. FWQI can be employed for foveated region of interest (ROI) image coding and quality enhancement. We show its effectiveness by using it as a guide for optimal bit assignment of the EFIC system. The coding system demonstrates very good coding performance and scalability in terms of foveated objective as well as subjective quality measurement.

An illustration of the human visual foveation model is available here. This model is employed by our FWQI algorithm.

Relevant Publications

Z. Wang, A. C. Bovik, L. Lu and J. Kouloheris, "Foveated wavelet image quality index," SPIE’s 46th Annual Meeting, Proc. SPIE, Application of digital image processing XXIV, vol. 4472, July-Aug. 2001.

(d) No-Reference Perceptual Quality Assessment of JPEG Compressed Images

This research aims to develop no-reference quality measurement algorithms for JPEG compressed images.

First, we established a JPEG image database and subjective experiments were conducted on the database. There are 120 test images in the database. 30 of them are original images. The rest are JPEG-compressed. 53 subjects were shown the database; most of them were the students taking the Digital Image and Video Processing course in the Fall 2001 semester in the Dept. of ECE, The Univ. of Texas at Austin. The subjects were asked to assign each image a quality score between 1 and 10 (10 represents the best quality and 1 the worst). The 53 scores of each image were averaged to a final Mean Opinion Score (MOS) of the image.

Second, we show that Peak Signal-to-Noise Ratio (PSNR), which requires the reference images, is a poor indicator of subjective quality. Therefore, tuning an NR measurement model towards PSNR is not an appropriate approach in designing NR quality metrics.

Furthermore, we propose a computational and memory efficient NR quality assessment model for JPEG images. Subjective test results are used to train the model, which achieves good quality prediction performance as shown below.

Demo

Download Free Software.

Relevant Publications

Z. Wang, H. R. Sheikh and A. C. Bovik, "No-reference perceptual quality assessment of JPEG compressed images," accepted by IEEE International Conference on Image Processing, Sept. 2002.

(e) Blind Measurement of Blocking Artifacts

Block transform coding has been widely adopted in many current image and video compression standards. In order to achieve low bit rates, quantization is normally used during encoding to compress the transform coefficients. The quantization process is lossy. As a result, the decompressed image and video exhibit various kinds of distortion artifacts such as blocking, blurring and ringing. The human visual sensitivity to different types of artifacts is very different. The blocking effect is usually the most significant among them, especially at low bit rate compression.

Blockiness is a special kind of image feature in the sense that human eyes can easily perceive it without observation of the original images. This implies that blockiness can be and should be detected and measured blindly. Currently, most published blockiness measurement techniques require access to the original images. We propose blind blocking artifact measurement, which is calculated without the reference images. It would be especially useful for the assessment and design of post-processing algorithms since the original images are not available at the receiver side.

We model the blocky image as a non-blocky image interfered with a pure blocky signal. The goal of the blocking effect measurement algorithm is then to detect and estimate the power of the blocky signal. Such a general model can easily combine with the human visual luminance and texture masking effects.

Relevant Publications

Z. Wang, A. C. Bovik, and B. L. Evans, "Blind measurement of blocking artifacts in images," IEEE International Conference on Image Processing, vol. 3, pp. 981-984, Sept. 2000.

(f) A Practical Video Quality Assessment System and the "Video Compare" Software

We developed a practical objective visual distortion measurement system for compressed video. The model is established by making use of both spatial and temporal human visual system (HVS) features, which include spatial frequency sensitivity, luminance masking, spatial masking, temporal frequency sensitivity, and short-term memory effect. The model is implemented in two stages. In the first stage, the video sequence is considered as a collection of independent still images. The visual distortion is measured using only spatial HVS features and reported frame by frame. In the second stage, temporal HVS features are added and the video distortion measures are given by a gradually time-varying curve as well as an overall distortion value. A ‘Video Compare’ software is developed to demonstrate the system.

Relevant Publications

Z. Wang and A. C. Bovik, "A Human Visual System-Based Objective Video Distortion Measurement System," International Conference on Multimedia Processing and Systems, Aug. 2000