Subjective Image Quality Assessment 

LIVE Image Quality Assessment Database

The Quality Assessment research at LIVE is being conducted in collaboration with CPS

Introduction

Quality Assessment research strongly depends upon subjective experiments to provide calibration data as well as a testing mechanism. After all, the goal of all QA research is to make quality predictions that are in agreement with subjective opinion of human observers. In order to calibrate QA algorithms and test their performance, a data set of images and videos whose quality has been ranked by human subjects is required. The QA algorithm may be trained on part of this data set, and tested on the rest.

At LIVE (in collaboration with The Department of Psychology at the University of Texas at Austin), an extensive experiment was conducted to obtain scores from human subjects for a number of images distorted with different distortion types. These images were acquired in support of a research project on generic shape matching and recognition.

We have decided to make the data set available to the research community free of charge. If you use these images in your research, we kindly ask that you reference this website and our papers listed below.

• H.R. Sheikh, Z.Wang, L. Cormack and A.C. Bovik, "LIVE Image Quality Assessment Database Release 2", http://live.ece.utexas.edu/research/quality.
• H.R. Sheikh, M.F. Sabir and A.C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms", IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440-3451, Nov. 2006.
• Z. Wang, A.C. Bovik, H.R. Sheikh and E.P. Simoncelli, "Image quality assessment: from error visibility to structural similarity," IEEE Transactions on Image Processing , vol.13, no.4, pp. 600- 612, April 2004.

Please scroll to the end of the page to download the data set.

Subjective experiments are cumbersome to design and expensive. Although great care was taken to ensure that the testing environment was as close to the "real-world" as possible, we cannot claim that our subjective experiments were exhaustive, comprehensive, or precise in every respect. Hence, any research conducted with the database should consider the limitations imposed by the scope and methodology of our experiments.

Please contact Hamid Rahim Sheikh (hamid.sheikh@ieee.org) if you have any questions.
The investigators on this research were:
Dr. Hamid Rahim Sheikh (hamid.sheikh@ieee.org) -- Formerly a student at the Department of ECE at UT Austin
Dr. Alan C. Bovik (bovik@ece.utexas.edu) -- Department of ECE at UT Austin
Dr. Lawrence Cormack (cormack@psy.utexas.edu) -- Department of Psychology at UT Austin
Dr. Zhou Wang (zhouwang@ieee.org) -- Formerly a student at the Department of ECE at UT Austin

What's new in Release 2

Release 2 has the following differences from Release 1:

1. More distortion types: 

• JPEG compresses images (169 images). More subjects than in Release 1

• JPEG2000 compressed images (175 images)

• NEW! Gaussian blur (145 images)

• NEW! White noise (145 images)

• NEW! Bit errors in JPEG2000 bit stream (145 images)

1. More subjects for JPEG distortion

2. DMOS values instead of MOS values for distorted images

3. Different processing of raw scores than in Release 1 

4. Individual subject scores and source code for processing raw scores will be released later. 

Please read the readme.txt in the database release for more details about the database.

Download Release 2

Subjective database Release 2. Please email Hamid R. Sheikh (hamid dot sheikh at ieee dot org) for password request (mentioning Release 1 or 2), briefly describing the intended use of the database and your affiliation. If you download the database, it is assumed that you agree to the copyright notice

Update to Release 2

Download realigned subjective quality data here. This data was obtained by running realignment experiments on Release 2 data. The details of the experiment can be found in the paper.

H. R. Sheikh, M. F. Sabir, A. C. Bovik, "A Statistical Evaluation of Recent Full Reference Quality Assessment Algorithms", IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440-3451, Nov. 2006.

Download Release 1

Subjective database for JPEG2000 - README.TXT. Please email Hamid R. Sheikh (hamid dot sheikh at ieee dot org)  for password request (mentioning Release 1 or 2), briefly describing the intended use of the database and your affiliation. If you download the database, it is assumed that you agree to the copyright notice

Subjective database for JPEG - README.TXT. Please email Hamid R. Sheikh (hamid dot sheikh at ieee dot org) for password request, briefly describing the intended use of the database and your affiliation. If you download the database, it is assumed that you agree to the copyright notice

Back to Quality Assessment Research page