Picture Quality and Comfort
More than that, our algorithms have found very wide industry use. For example, the Visual Information Fidelity (VIF) model developed in LIVE is a core engine of the Netflix VMAF video quality prediction engine that is used to quality-assess all Netflix encodes. This represents 35% of all US bandwidth usage, and a significant and growing percentage of global bandwidth use. VIF is also a standardized video quality tool in the ISO/MPEG standard HDR Tools for ongoing High Dynamic Range (HDR) video compression development.
The Emmy-winning Structual Similarity (SSIM) has found even more widespread use. SSIM and its reletives/versions, such as MS-SSIM, UQI, Fast-SSIM, CW- SSIM, SSIMPlus etc are relied on by most manufacturers of broadcast encoders and statistical multiplexers that compress and distribute television, including, for example Cisco, Motorola-Arris, Ericson, Harmonic, Envivio, RGB Networks, Intel, Texas Instruments (and many others) to daily ensure their products deliver the best possible video quality to the mass consumer. SSIM is used by U.S. broadcast, cable, and satellite program originators (such as Netflix, DIRECTV, AT&T, Comcast, Discovery, Stars, NBC, FOX, Showtime, Turner, PBS and many others) to test and/or monitor the quality of the picture content that they deliver to tens of millions of US Television viewers around the clock. SSIM is used by international terrestrial, cable and satellite broadcasters to monitor and control the quality of their channel line-ups. These include British Telecom, the Sky companies in Brazil, Italy, the UK, and India, Nine and Telstra in Australia, and Oi and TV Globo in Brazil, among many others. For example, British Telecom continuously quality-controls 30 live HD channels using SSIM. SSIM is also part of the ISO-MPEG H.264 video compression standard JM reference software.
The high-performance MOVIE index developed in LIVE is also globally marketed and used to assess Television picture content on a global scale. Overall, the picture quality delivered daily to hundreds of millions of global Television viewers is monitored and maintained using picture quality models and algorithms first developed in LIVE. LIVE continues to develop top-performing Reference image, video, and 3D picture quality models and algorithms, including the remarkably robust ST-RRED model described here.
The above models are all reference models, but the development of No-Reference picture, video, and 3D quality models in LIVE has also grown quite significant. Models such as BIQI, DIIVINE, BRISQUE, NIQE and 3D-BLINQ, all of which are based on natural scene statistics models first developed in the context of picture quality in LIVE, are heavily referenced and are being integrated/adapted into a wide variety of industry solutions, e.g., for video source inspection at Netflix, and for puicture quality control in digital cameras. Some of the key papers follow:
N. Damera-Venkata, T.D. Kite, W.S. Geisler, B.L. Evans and A.C. Bovik, "Image quality assessment based on a degradation model," IEEE Transactions on Image Processing, vol. 9, no. 4, pp. 636-650, April 2000.
S. Lee, M.S. Pattichis and A.C. Bovik, "Foveated video quality assessment," IEEE Transactions on Multimedia, vol. 4, no. 1, pp. 129-132, March 2002.
Z. Wang and A.C. Bovik, “Why is image quality assessment so difficult?,” IEEE International Conference on Acoustics, Speech, and Signal Processing, Orlando, Florida, May 13-17, 2002.
Z. Wang and A.C. Bovik, “A universal image quality index,” IEEE Signal Processing Letters, vol. 9, no. 3, pp. 81-84, March 2002.
Z. Wang, E. Simoncelli and A.C. Bovik, “Multi-scale structural similarity for image quality assessment,” Thirty-Seventh Annual Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, California, November 9-12, 2003.
Z. Wang, L. Lu and A.C. Bovik, “Video quality assessment based on structural distortion measurement,” Signal Processing: Image Communication, vol. 19, no. 2, pp. 121-132, February 2004.
Z. Wang, A.C. Bovik, H.R. Sheikh and E.P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, April 2004 (Winner of IEEE Signal Processing Society Best Paper Award for 2009) (Winner of IEEE Signal Processing Society Sustained Impact Paper Award for 2017, limited to papers published in an IEEE Signal Processing Society journal at least ten years prior to the award, thereby recognizing sustained impact over many years). It is the most-cited paper ever published in any IEEE Signal Processing Society publication.
H.R. Sheikh, A.C. Bovik and L.K. Cormack, “No-reference quality assessment using natural scene statistics: JPEG2000,” IEEE Transactions on Image Processing, vol. 14, no. 11, pp. 1918-1927, November 2005.
H.R. Sheikh, A.C. Bovik and G. DeVeciana, “An information fidelity criterion for image quality assessment using natural scene statistics,” IEEE Transactions on Image Processing, vol. 14, no. 12, pp. 2117-2128, December 2005.
H.R. Sheikh and A.C. Bovik, “Image information and visual quality,” IEEE Transactions on Image Processing, vol. 15, no. 2, pp. 430-444, February 2006. Recipient of a 2017 Google Scholar Classic Paper Citation (for Computer Vision and Pattern Recognition). Google Scholar Classic Papers are highly-cited papers that have stood the test of time, and are among the ten most-cited articles in their area of research published ten years earlier.
Z. Wang, G. Wu, H.R. Sheikh, E. Simoncelli, E. Yang and A.C. Bovik, “Quality-aware images,” IEEE Transactions on Image Processing, vol. 15, no. 5, pp. 1680-1689, May 2006.
H.R. Sheikh, M.F. Sabir and A.C. Bovik, “An evaluation of recent full reference image quality assessment algorithms,” IEEE Transactions on Image Processing, vol. 15, no. 11, pp. 3440-3451, November 2006. Recipient of a 2017 Google Scholar Classic Paper Citation (for Signal Processing). Google Scholar Classic Papers are highly-cited papers that have stood the test of time, and are among the ten most-cited articles in their area of research published ten years earlier.
S.S. Channappayya, A.C. Bovik, C. Caramanis and R.W. Heath, “Design of linear equalizers optimized for the structural similarity index,” IEEE Transactions on Image Processing, vol. 17, no. 6, pp. 857-872, June 2008.
S.S. Channappayya, A.C. Bovik and R.W. Heath, “Rate bounds on SSIM index of quantized images,” IEEE Transactions on Image Processing, vol. 17, no. 9, pp. 1624-1639, September 2008.
A.C. Bovik, “Video quality is in the eye of the beholder,” IEEE Communications Systems and Integration and Modeling Electronic Newsletter, Invited Article, vol. 3, no. 1, pp. 11-16, December 2008.
Z. Wang and A.C. Bovik, “Mean squared error: Love it or leave it? ― A new look at signal fidelity measures,” IEEE Signal Processing Magazine, vol. 26, no. 1, pp. 98-117, January 2009 (Winner of the IEEE Signal Processing Magazine Best Paper Award for 2013).
A.K. Moorthy and A.C. Bovik, “Visual importance pooling for image quality assessment,” IEEE Journal on Selected Topics in Signal Processing, Special Issue on Visual Media Quality Assessment, vol. 3, no. 2, pp. 193-201, April 2009.
A.C. Bovik, “Meditations on video quality,” Invited Distinguished Position Paper, IEEE Multimedia Communications E-Letter, vol. 4, no. 4, pp. 4-10, May 2009.
M.P. Sampat, Z. Wang, S. Gupta, A.C. Bovik and M.K. Markey, “Complex wavelet structural similarity: a new image similarity index,” IEEE Transactions on Image Processing, vol. 18, no, 11, pp. 2385-2401, November 2009.
C. Li and A.C. Bovik, “Content-weighted video quality assessment using a three-component image model,” Journal of Electronic Imaging, Special Section on Image Quality, vol. 19, no. 1, 011003, pp. 011003-1 - 011003-9, January 7, 2010.
K. Seshadrinathan and A.C. Bovik, “Motion-tuned spatio-temporal quality assessment of natural videos,” IEEE Transactions on Image Processing, vol. 19, no. 2, pp. 335-350, February 2010 (Winner of the IEEE Signal Processing Society Young Author Best Paper Award for 2013).
A.K. Moorthy and A.C. Bovik, “A two-step framework for constructing blind image quality indices,” IEEE Signal Processing Letters, vol. 17, no. 5, pp. 513-516, May 2010.
K. Seshadrinathan, R. Soundararajan, A.C. Bovik and L.K. Cormack, “Study of subjective and objective quality assessment of video,” IEEE Transactions on Image Processing, vol. 19, no. 6, pp. 1427-1441, June 2010.
M.A. Saad, A.C. Bovik and C. Charrier, “A DCT statistics based blind image quality index,” IEEE Signal Processing Letters, vol. 17, no. 6, pp. 583-586, June 2010.
C. Li and A.C. Bovik, “Content-partitioned structural similarity index for image quality assessment,” Signal Processing: Image Communication, Special Issue on Image and Video Quality Assessment, vol. 25, no. 7, pp. 517-526, July 2010.
C. Yim and A.C. Bovik, “Quality assessment of de-blocked images,” IEEE Transactions on Image Processing, vol. 20, no. 1, pp. 88-98, January 2011.
C. Yim and A.C. Bovik, “Evaluation of temporal variation of video quality in packet loss networks,” Signal Processing: Image Communication, vol. 26, no. 1, pp. 24-38, January 2011.
K. Seshadrinathan and A.C. Bovik, “Automatic prediction of perceptual quality of multimedia signals – A survey,” International Journal of Multimedia Tools and Applications, Special Issue on Survey Papers in Multimedia by World Experts, vol. 51, no. 1, pp. 163-186, January 2011.
A.K. Moorthy and A.C. Bovik “Visual Quality Assessment Algorithms: What Does the Future Hold?” International Journal of Multimedia Tools and Applications, Special Issue on Survey Papers in Multimedia by World Experts, vol. 51, no. 2, pp. 675-696, February 2011.
C. Li and A.C. Bovik, “Blind image quality assessment using a general regression neural network,” IEEE Transactions on Neural Networks, vol. 22, no. 5, pp. 793-799, May 2011.
M.-J. Chen and A.C. Bovik, “No-reference image blur assessment using multiscale gradient,” EURASIP Journal on Image and Video Processing, Special Issue on Quality of Multimedia Experience, 2011:3 doi:10.1186/1687-5281- 2011-3 [open access], July 2011.
M.-J. Chen and A.C. Bovik, “Fast structural similarity index algorithm,” Journal of Real- Time Image Processing, vol. 6, no. 4, pp. 281-287, December 2011.
A.K. Moorthy and A.C. Bovik, “Blind image quality assessment: From natural scene statistics to perceptual quality,” IEEE Transactions on Image Processing, vol. 20, no. 12, pp. 3350-3364, December 2011.
A. Mittal, G.S. Muralidhar, J. Ghosh and A.C. Bovik, “Blind image quality assessment without human training using latent quality factors,” IEEE Signal Processing Letters, vol. 19, no. 2, pp. 75-78, February 2012.
R. Soundararajan and A.C. Bovik, “RRED indices: Reduced reference entropic differencing for image quality assessment,” IEEE Transactions on Image Processing, vol. 21, no. 2, pp. 517-526, February 2012.
M.A. Saad and A.C. Bovik, “Blind image quality assessment: A natural scene statistics approach in the DCT domain,” IEEE Transactions on Image Processing, vol. 21, no. 8, pp. 3339-3352, August 2012.
A.K. Moorthy, L.K. Choi, A.C. Bovik and G. de Veciana, “Video quality assessment on mobile devices: Subjective, behavioral, and objective studies,” IEEE Journal of Selected Topics in Signal Processing, Special Issue on New Subjective and Objective Methodologies for Audio and Visual Signal Processing, vol. 6, no. 6, pp. 652-671, October 2012.
C. Charrier, K. Knoblauch, L.T. Maloney, A.C. Bovik and A.K. Moorthy, “Optimizing multi-scale SSIM for compression via MLDS,” IEEE Transactions on Image Processing, vol. 21, no. 12, pp. 4682-4694, December 2012.
A. Mittal, A.K. Moorthy and A.C. Bovik, “No-reference image quality assessment in the spatial domain,” IEEE Transactions on Image Processing, vol. 21, no. 12, pp. 4695-4708, December 2012.
J. Park, S. Lee and A.C. Bovik, “VQpooling: Video quality pooling adaptive to perceptual distortion severity,” IEEE Transactions on Image Processing, vol. 22, no. 2, pp. 610-620, February 2013.
A. Mittal, R. Soundararajan and A.C. Bovik, “Making a ‘completely blind’ image quality analyzer,” IEEE Signal Processing Letters, vol. 21, no. 3, pp. 209-212, March 2013.
R. Soundararajan and A.C. Bovik, “Video quality assessment by reduced reference spatio- temporal entropic differencing,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 23, no. 4, pp. 684-694, April 2013 (Winner of the IEEE Circuits and Systems for Video Technology Best Paper Award for 2016).
R. Soundararajan and A.C. Bovik, “Survey of information theory and visual quality assessment,” Signal, Image, and Video Processing, Special Section on Human Vision and Information Theory, Invited Paper, vol. 7, no. 3, pp. 391-401, May 2013.
M.-J. Chen, L.K. Cormack and A.C. Bovik, “No-reference quality assessment of natural stereopairs,” IEEE Transactions on Image Processing, vol. 22, no. 9, pp. 3379-3391, September 2013.
A.C. Bovik, “Automatic prediction of perceptual image and video quality,” Proceedings of the IEEE, Invited Paper, vol. 101, no. 9, pp. 2008-2024, September 2013.
A.K. Moorthy, C.-C. Su, A. Mittal and A.C. Bovik, “Subjective evaluation of stereoscopic image quality,” Signal Processing: Image Communication, vol. 28, no. 9, pp. 870-883, September 2013.
M.-J. Chen, C.-C. Su, D.-K. Kwon, L.K. Cormack and A.C. Bovik, “Full-reference quality assessment of stereopairs accounting for rivalry,” Signal Processing: Image Communication, vol. 28, no. 10, pp. 1143-1155, October 2013.
A.K. Moorthy, A. Mittal and A.C. Bovik, “Perceptually-optimized blind repair of natural images,” Signal Processing: Image Communication, vol. 28, no. 10, pp. 1478-1493, November 2013.
M.-J. Chen, A.C. Bovik and L.K. Cormack, “Study of distortion conspicuity on stereoscopically viewed 3D images,” Journal of the Society for Information Display, vol. 21 no. 11, pp. 491-503, November 2013.
W. Xue, L. Zhang, X. Mou and A.C. Bovik, “Gradient magnitude similarity deviation: A highly efficient perceptual image quality index,” IEEE Transactions on Image Processing, vol. 23, no. 2, pp. 684-695, February 2014.
T. Kim, J. Kang, S. Lee and A.C. Bovik, “Interactive continuous quality evaluation of subjective 3D video quality of experience,” IEEE Transactions on Multimedia, vol. 16. no. 2, pp. 387-402, February 2014.
M. Saad and A.C. Bovik, “Blind prediction of natural video quality,” IEEE Transactions on Image Processing, vol. 23, no. 3, pp. 1352-1365, March 2014.
L. Liu, H. Dong, H. Huang and A.C. Bovik, “No-reference image quality assessment in the curvelet domain,” Signal Processing: Image Communication, vol. 29, no. 4, pp. 494-505, April 2014.
C. Chen, L.K. Choi, G. de Veciana, C. Caramanis, R.W. Heath, Jr. and A.C. Bovik, “A model of the time-varying subjective quality of HTTP video streams with rate adaptations,” IEEE Transactions on Image Processing, vol. 23, no. 5, pp. 2206-2221, May 2014.
J. Park, S. Lee and A.C. Bovik, “3D visual discomfort prediction: Vergence, foveation, and the physiological optics of accomodation,” IEEE Journal on Selected Topics in Signal Processing, vol. 8, no. 3, pp. 415-427, June 2014.
Y. Zhang, A.K. Moorthy, D.M. Chandler and A.C. Bovik, “C-DIIVINE: No-reference image quality assessment based on local magnitude and phase statistics of natural scenes,” Signal Processing: Image Communication, vol. 29, no. 4, pp. 725-747, August 2014.
L. Liu, B. Liu, H. Huang and A.C. Bovik, “No-reference image quality assessment based on spatial and spectral entropies,” Signal Processing: Image Communication, vol. 29, no. 8, pp. 856-863, September 2014.
Q.B. Sang, H.X. Qi, X.J. Wu, C.F. Li and A.C. Bovik, “No-reference image blur index using singular value curve,” Visual Communications and Image Representation, vol. 25, no. 7, pp. 1625-1630, October 2014.
W. Xue, X. Mou, L. Zhang, A.C. Bovik, and X. Feng, “Blind image quality prediction using joint statistics of gradient magnitude and laplacian features,” IEEE Transactions on Image Processing, vol. 23, no. 11, pp. 4850-4862, November 2014.
Q.B. Sang, X.J. Wu, C.F. Li and A.C. Bovik, “Blind image quality assessment using a reciprocal singular value curve,” Signal Processing: Image Communication, vol. 29, no. 10, pp. 1149-1157, November 2014.
T. Eerola, L. Lensu, H. Kalviainen and A.C. Bovik, “Study of no-reference image quality assessment algorithms on printed images,” Journal of Electronic Imaging, Special Section on Image Quality and System Performance, vol. 23, no. 6, pp. 061106-1- 061106-12, November/December 2014.
S. Gunasekar, J. Ghosh and A.C. Bovik, “Face detection on distorted images augmented by perceptual quality-aware features,” IEEE Transactions on Information Forensics and Security, Special Issue on Facial Biometrics in the Wild, vol. 9, no. 12, pp. 2119-2131, December 2014.
T. Oh, J. Park, K. Seshadrinathan, S. Lee and A.C. Bovik, “No-reference image sharpness assessment of camera-shaken images by analysis of spectral structure,” IEEE Transactions on Image Processing, vol. 23, no. 12, pp. 5428-5439, December 2014.
M.H. Pinson, L.K. Choi, A.K. Moorthy, and A.C. Bovik, “Temporal video quality model accounting for variable frame delay distortions,” IEEE Transactions on Broadcasting, vol.
no. 4, pp. 637-649, December 2014.
C. Chen, X. Zhu, G. de Veciana, A.C. Bovik and R.W. Heath, “Rate adaptation and admission control for video transmission with subjective quality constraints,” IEEE Journal on Selected Topics in Signal Processing, Special Issue on Visual Signal Processing for Wireless Networks (VSPWN), vol. 9, no. 1, pp. 22-36, February 2015.
J. Park, H. Oh, S. Lee and A.C. Bovik, “3D visual discomfort predictor: Analysis of disparity and neural activity statistics,” IEEE Transactions on Image Processing, vol. 24, no. 3, pp. 1101-1114, March 2015.
C.-C. Su, L.K. Cormack and A.C. Bovik, “Oriented correlation models of distorted natural images with application to natural stereopair quality evaluation,” IEEE Transactions on Image Processing, vol. 24, no. 5, pp. 1685-1699, May 2015.
L. Zhang, L. Zhang and A.C. Bovik, “A feature-enriched completely blind local image quality analyzer,” IEEE Transactions on Image Processing, vol. 24, no. 8, pp. 2579-2591, August 2015.
L.K. Choi, J. You and A.C. Bovik, “Referenceless prediction of perceptual fog density and perceptual image defogging,” IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 3888-3901, November 2015.
L. Liu, Y. Hua, Q. Zhao, H. Huang and A.C. Bovik, “Blind image quality assessment by relative gradient statistics and Adaboosting neural network,” Signal Processing: Image Communication, vol. 40, no. 1, pp. 1-15, January 2016.
T.R. Goodall, A.C. Bovik and N.G. Paulter, Jr., “Tasking on natural statistics of infrared images,” IEEE Transactions on Image Processing, vol. 25, no. 1, pp. 65-79, January 2016.
A. Mittal, M. Saad and A.C. Bovik, “A ‘completely blind’ video integrity oracle,” IEEE Transactions on Image Processing, vol. 25, no. 1, pp. 289-300, January 2016.
D. Ghadiyaram and A.C. Bovik, “Massive online crowdsourced study of subjective and objective picture quality,” IEEE Transactions on Image Processing, vol. 25, no. 1, pp. 372- 387, January 2016.
H. Oh, S. Lee and A.C. Bovik, “3D visual discomfort prediction: A dynamic accommodation and vergence interaction model,” IEEE Transactions on Image Processing, vol. 25, no. 2, pp. 615-629, February 2016.
F. Xie, Y. Lu, A.C. Bovik, Z. Jiang and R. Meng, “Application-driven no reference quality assessment for dermoscopy images with multiple distortions,” IEEE Transactions on Biomedical Engineering, vol. 63, no. 6, pp. 1248-1256, June 2016.
K.J. Chen, J. Zhou, J. Sun and A.C. Bovik, “3D visual discomfort prediction using low complexity disparity algorithms,” EURASIP Journal on Image and Video Processing, DOI: 10.1186/s13640-016- 0127-4, vol. 2016, no. 1, pp. 1-10, August 2016.
T. Goodall, A.C. Bovik, Z. Li, A. Aaron and I. Katsavounidis, “Blind picture upscaling ratio prediction,” IEEE Signal Processing Letters, vol. 23, no. 12, pp. 1801-1805, December 2016.
D. Ghadiyaram and A.C. Bovik, “Perceptual quality prediction on authentically distorted images using a bag of features approach,” Journal of Vision, vol. 17, no. 1, art. 32, pp. 1-25, January 2017.
K.J. Chen and A.C. Bovik, “Visual discomfort prediction on stereoscopic 3D images without explicit disparities,” Signal Processing: Image Communication, vol. 51, pp. 50-60, February 2017.
D. Kundu, D. Ghadiyaram, A.C. Bovik and B.L. Evans, “No-reference quality assessment of tone-mapped HDR pictures,” IEEE Transactions on Image Processing, vol. 26, no. 6, pp. 2957-2971, June 2017.
D.E. Moreno-Villamarin, H.D. Benitez-Restrepo and A.C. Bovik, “Predicting the quality of fused long wave infrared and visible light images,” IEEE Transactions on Image Processing, vol. 26, no. 7, pp. 3479-3491, July 2017.
C. Bampis and A.C. Bovik, “Continuous prediction of streaming video QoE using dynamic networks,” IEEE Signal Processing Letters, vol. 24, no. 7, pp. 1083-1087, July 2017.
B. Yan, B. Bare, K. Li, J. Li and A.C. Bovik, “Learning based quality assessment for image retargeting,” Signal Processing: Image Communication, vol. 56, pp. 12-19, August 2017.
T. Kim, S. Lee and A.C. Bovik, “Enhancement of visual comfort and sense of presence on stereoscopic 3D images,” IEEE Transactions on Image Processing, vol. 26, no. 8, pp. 3789- 3801, August 2017.
K. Gu, J. Zhou, G. Zhai, W. Lin and A.C. Bovik, “No-reference quality assessment of screen content pictures,” IEEE Transactions on Image Processing, vol. 26, no. 8, pp. 4005-4017, August 2017.
D. Kundu, D. Ghadiyaram, A.C. Bovik and B.L. Evans, “Large-scale crowdsourced study for tone-mapped HDR pictures,” IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4725-4740, October 2017.
J. Kim, T. Kim, S. Lee and A.C. Bovik, “Quality assessment of perceptual crosstalk on two- view auto-stereoscopic displays,” IEEE Transactions on Image Processing, vol. 26, no. 10, pp. 4885-4899, October 2017.