ut ut

Laboratory for Image & Video Engineering

Full Reference Image and Video Quality Assessment

Please go here to download our quality assessment databases and for free software releases of our quality assessment algorithms.

Traditional Error-sensitivity Based Image/Video Quality Assessment

Traditional perceptual image quality assessment approaches are based on measuring the errors (signal differences) between the distorted and the reference images, and attempt to quantify the errors in a way that simulates human visual error sensitivity features. These methods usually involve

  1. A channel decomposition process that transforms the image signals into different spatial frequency as well as orientation selective subbands
  2. An error normalization process that weights the error signal in each subband by incorporating the variation of visual sensitivity in different subbands (often related to a contrast sensitivity function), and the variation of visual error sensitivity caused by intra- or inter-channel neighboring transform coefficients (often related to certain visual masking effects)
  3. An error pooling process that combines the error signals in different subbands into a single quality/distortion value.

While these approaches can conveniently make use of many known psychophysical features of the human visual systems, they are based on some strong assumptions, which are difficult to validate. The limitations include “the suprathreshold problem”, “the natural image complexity problem”, “the decorrelation problem” and “the cognitive interaction problem”. More detailed discussions are given in the publications below.

Relevant Publications

  1. Z. Wang and A.C. Bovik, The Handbook of Image and Video Processing , 2nd ed. New York: Academic Press, June 2005, ch. Structural approaches to image quality assessment, pp. 961–974.
  2. Z. Wang, A.C. Bovik, H.R. Sheikh and E.P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing , vol.13, no.4pp. 600- 612, April 2004.
  3. Z. Wang and A.C. Bovik, The Handbook of video databases: Design and Applications . Boca Raton, Florida: CRC Press, September 2003, ch. Objective video quality assessment, pp. 1041–1078.
  4. Z. Wang, A.C. Bovik and L. Lu, "Why is image quality assessment so difficult?," IEEE International Conference on Acoustics, Speech, and Signal Processing , 2002.

Information Theoretic Approaches to Image Quality Assessment

At LIVE, we have explored information-theoretic approaches to the quality assessment problem, where the quality assessment problem is viewed as an information-fidelity problem rather than a signal-fidelity problem. An image source communicates to a receiver through a channel that limits the amount of information that could flow through it, thereby introducing distortions. The output of the image source is the reference image, the output of the channel is the test image, and the goal is to relate the visual quality of
the test image to the amount of information shared between the test and the reference signals, or more precisely, the mutual information between them. Although mutual
information is a statistical measure of information fidelity, and may only be loosely related with what humans regard as image information, it places fundamental limits on the amount of cognitive information that could be extracted from an image. For example, in cases where the channel is distorting images severely, corresponding to low mutual information between the test and the reference, the ability of human viewers to obtain semantic information by discriminating and identifying objects in images is also hampered. Thus, information fidelity methods exploit the relationship between statistical image information and visual quality. Read more .

Relevant Publications

  1. H.R. Sheikh, M.F. Sabir and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Transactions on Image Processing , vol.15, no.11, pp.3440-3451, Nov. 2006.
  2. H.R. Sheikh.and A.C. Bovik, "Image information and visual quality," IEEE Transactions on Image Processing , vol.15, no.2,pp. 430- 444, Feb. 2006.
  3. H.R. Sheikh, A.C. Bovik and G. de Veciana, "An information fidelity criterion for image quality assessment using natural scene statistics," IEEE Transactions on Image Processing , vol.14, no.12, pp. 2117- 2128, Dec. 2005.
  4. H. R. Sheikh and A. C. Bovik, The Handbook of image and video processing , 2nd ed. New York: Academic Press, June 2005, ch. Information theoretic approaches to image quality assessment, pp. 975–989.
  5. H. R. Sheikh and A. C. Bovik , "A Visual Information Fidelity Approach to Video Quality Assessment" (Invited Paper), The First International Workshop on Video Processing and Quality Metrics for Consumer Electronics , 2005.

Structural Similarity Based Image and Video Quality Assessment

Different from traditional error-sensitivity based approach, structural similarity based image quality assessment is based on the following philosophy:

The main function of the human visual system is to extract structural information from the viewing field, and the human visual system is highly adapted for this purpose. Therefore, a measurement of structural information loss can provide a good approximation to perceived image distortion.

Here, we regard the structural information in an image as those attributes that reflect the structure of objects in the scene, independent of the average luminance and contrast. A universal image quality index that separates the comparison of luminance, contrast and structure was introduced. This approach was generalized and improved, leading to a Structural SIMilarity Index (SSIM), which had shown clear advantages over traditional mean squared error (MSE) and peak signal to noise ratio (PSNR) measures when tested on a database of JPEG and JPEG2000 compressed images. This SSIM index has also been applied for video quality assessment and extended to a multi-scale approach.

Relevant Publications

  1. Z. Wang and A. C. Bovik, The Handbook of Image and Video Processing , 2nd ed. New York: Academic Press, June 2005, ch. Structural approaches to image quality assessment, pp. 961–974.
  2. Z. Wang, A.C. Bovik, H.R. Sheikh and E.P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing , vol.13, no.4pp. 600- 612, April 2004.
  3. Z. Wang, L. Lu and A.C. Bovik, "Video quality assessment based on structural distortion measurement", Signal Processing: Image Communication, Special issue on Objective video quality metrics , vol. 19, no. 2, February 2004.
  4. Z. Wang and A. C. Bovik, The Handbook of video databases: Design and Applications . Boca Raton, Florida: CRC Press, September 2003, ch. Objective video quality assessment, pp. 1041–1078.
  5. Z. Wang, E.P. Simoncelli and A.C. Bovik, "Multi-scale structural similarity for image quality assessment", Proc. 37th IEEE Asilomar conference on Signals, Systems and Computers , 2002.
  6. Z. Wang and Bovik, A.C., "A universal image quality index," IEEE Signal Processing Letters , vol.9, no.3pp.81-84, Mar 2002.

Foveated Wavelet Image Quality Index (FWQI)

The human visual system is highly non-uniform in sampling, coding, processing and understanding. The resolution has the highest value at the point of the fixation and drops rapidly away from that point as a function of eccentricity. Currently, most image quality measurement methods are designed for uniform resolution images. These methods do not correlate well with the perceived foveated image quality. Wavelet analysis delivers a convenient way to simultaneously examine localized spatial as well as frequency information. We developed a new image quality metric called foveated wavelet image quality index (FWQI) in the wavelet transform domain. FWQI considers multiple factors of the HVS, including the space variance of the contrast sensitivity function, the spatial variance of the local visual cut-off frequency, the variance of human visual sensitivity in different wavelet subbands, and the influence of the viewing distance on the display resolution and the HVS features. FWQI can be employed for foveated region of interest (ROI) image coding and quality enhancement. We show its effectiveness by using it as a guide for optimal bit assignment of the Embedded Foveation Image Coding system. The coding system demonstrates good coding performance as well as scalability in terms of foveated objective as well as subjective quality measurement.

Relevant Publications

  1. Z. Wang, L. Lu, and A. C. Bovik, "Foveation scalable video coding with automatic fixation selection," IEEE Transactions on Image Processing , vol. 11, no. 2, Feb. 2003.
  2. H. R. Sheikh, S. Liu, Z. Wang, and A. C. Bovik, “Foveated multi-point videoconferencing at low bit rates,” IEEE International Conf. on Acoustics, Speech, & Signal Processing , May 2002.
  3. Z. Wang, and A. C. Bovik, "Embedded foveation image coding," IEEE Transactions on Image Processing , vol. 10, no. 10, pp. 1397-1410, Oct. 2001.
  4. Z. Wang, A. C. Bovik, and L. Lu, "Wavelet-based foveated image quality measurement for region of interest image coding," IEEE International Conference on Image Processing , Oct. 2001.
  5. Z. Wang, A. C. Bovik, L. Lu and J. Kouloheris, "Foveated wavelet image quality index," SPIE’s 46th Annual Meeting , Proc. SPIE , Application of digital image processing XXIV , vol. 4472, July-Aug. 2001.
  6. Z. Wang, L. Lu, and A. C. Bovik, "Rate scalable video coding using a foveation-based human visual system model," IEEE International Conference on Acoustics, Speech, & Signal Processing , vol. III, pp. 1785-1789, May 2001.

Back to Quality Assessment Research page